Metallic Metals Act

The Metallic Metals Act was a fictional piece of legislation included in a 1947 opinion survey conducted by Sam Gill and published in the March 14, 1947 issue of Tide magazine. When given four possible replies, 70% of respondents claimed to have an opinion on the act. It has become a classic example of the risks of meaningless responses to closed-ended questions and prompted the study of the pseudo-opinion phenomenon.

The question

Respondents were asked this question and were given four possible answers:[1]
Which of the following statements most closely coincides with your opinion of the Metallic Metals Act?

• It would be a good move on the part of the US.
• It would be a good thing, but should be left to the individual states
• It’s alright for foreign countries, but should not be required here.
• It is of no value at all

Initial publication and reaction

Sam Gill was a Marketing Research Director for Sherman & Marquette, Inc when he included a question about the fictional Metallic Metals Act in a survey. He reported on the results in the March 14, 1947 issue of Tide magazine in an article titled "How Do You Stand on Sin?", saying that 70% of respondents claimed to have an opinion on the topic.[2][3] Gill also asked respondents if they favored incest, an unfamiliar term to most people at the time, and one third supported it.[1] The article did not include any information on the size or make-up of the sample population, nor how much pressure the interviewer applied to receive a response.[4]

A similar study by Eugene Hartley in 1946 asked college students how connected they felt to students of various nationalities. His questionnaire included three imaginary nationalities, but a majority of students did not question them.[1] Together, these two studies are the earliest publicized examples of opinion surveys on fake subjects, a phenomenon known as a pseudo-opinion.[5][6] At the time, the results of both studies amused laymen but were not immediately taken seriously in the field of public opinion because most professionals felt the studies were ridiculous and reflected negatively on their field.[3] One exception, Stanley L. Payne, wrote about Gill's study in the 1951 The Public Opinion Quarterly journal article "Thoughts About Meaningless Questions" and called for further investigation into this type of non-sampling error.[3]

Legacy

Despite Payne's call to action, pseudo-opinions remained largely unstudied until the 1980s,[6] but in 1970 Philip Converse postulated that answering "don't know" is seen by respondents as an admission of "mental incapacity".[7] In 1981, researchers Howard Schuman and Stanley Presser were unable to locate documentation for Gill's study and concluded it should be taken as an anecdote rather than a true study.[5] Their research found that pseudo-opinions are a significant source of error but not as prevalent as Hartley and Gill's studies suggested.[1]

The Metallic Metals Act is considered a classic example of pseudo-opinions and difficulties with close-ended survey questions[2][8] and continues to be supported by later studies.[9] By 1991, it had become standard practice to include a false question in opinion surveys to gauge the degree of pseudo-opinions.[10] A study by the University of Cincinnati found 20 to 40 percent of Americans will provide pseudo-opinions because of social pressure, using context clues to select an answer they believe will please the questioner.[11] This has occasionally provided a source for jokes on talk shows and comedy shows who air interviews to mock the respondents.[4] Other studies have shown the phenomenon is not limited to the United States.[12] In a 2019 opinion piece written for The Guardian, Richard Seymour speculated that most opinion polls only represent what respondents heard most recently in the news media.[13]

References

1. "How Questions Affect Answers" p55. Retrieved July 17, 2018
2. Bishop, George F. (2005). The Illusion of Public Opinion. Rowman & Littlefield. p. 19. ISBN 9780742516458.
3. Payne, Stanley L. (1950). "Thoughts About Meaningless Questions". The Public Opinion Quarterly. 14 (4): 687–696. JSTOR 2746245.
4. Inglis-Arkell, Esther (June 6, 2014). "The Metallic Metals Act Shows People Always Bluffed About Politics". IGN. Retrieved July 16, 2018.
5. Bishop, G.; Tuchfarber, A.; Oldendick, R. (1986). "Opinions on Fictitious Issues: The Pressure to Answer Survey Questions". The Public Opinion Quarterly. 50 (2): 240. JSTOR 2748887.
6. Bishop, G.; Oldendick, R.; Tuchfarber, A.; Bennett, S. (1980). "Pseudo-Opinions on Public Affairs". The Public Opinion Quarterly. 44 (2): 198. JSTOR 2748428.
7. Schuman, Howard; Presser, Stanley (1996). Questions and Answers in Attitude Surveys. Sage. p. 147. ISBN 9780761903598.
8. Lawless, Harry T.; Heymann, Hildegarde (August 31, 1999). Sensory Evaluation of Food. Springer Science & Business Media. p. 511. ISBN 9780834217522.
9. Somin, Ilya (December 18, 2015). "Political ignorance and bombing Agrabah". The Washington Post. Retrieved July 16, 2018.
10. Baker, Michael J. (1991). "Data collection — questionnaire design". Research for Marketing. London: Palgrave. pp. 132–158. doi:10.1007/978-1-349-21230-9_7. ISBN 978-0-333-47021-3.
11. Daniels, Eugene (December 18, 2015). "The 'Bomb Agrabah' Survey Shows How Problematic Polling Can Be". KIVI-TV. Archived from the original on July 18, 2018. Retrieved July 17, 2018.
12. Somin, Ilya (May 23, 2015). "Survey suggests that one third of US 8th graders believe that Canada, France, and Australia are dictatorships". The Washington Post. Retrieved July 17, 2018.
13. Seymour, Richard (September 20, 2019). "The Polling Industry Doesn't Measure Public Opinion - It Produces It". The Guardian. Retrieved February 20, 2020.