Metallic Metals Act


The Metallic Metals Act was a fictional piece of legislation included in a 1947 opinion survey conducted by Sam Gill and published in the March 14, 1947 issue of Tide magazine. When given four possible replies, 70% of respondents claimed to have an opinion on the act. It has become a classic example of the risks of meaningless responses to closed-ended questions and prompted the study of the pseudo-opinion phenomenon.

The question

Respondents were asked this question and were given four possible answers:
Which of the following statements most closely coincides with your opinion of the Metallic Metals Act?
Sam Gill was a Marketing Research Director for Sherman & Marquette, Inc when he included a question about the fictional Metallic Metals Act in a survey. He reported on the results in the March 14, 1947 issue of Tide magazine in an article titled "How Do You Stand on Sin?", saying that 70% of respondents claimed to have an opinion on the topic. Gill also asked respondents if they favored incest, an unfamiliar term to most people at the time, and one third supported it. The article did not include any information on the size or make-up of the sample population, nor how much pressure the interviewer applied to receive a response.
A similar study by Eugene Hartley in 1946 asked college students how connected they felt to students of various nationalities. His questionnaire included three imaginary nationalities, but a majority of students did not question them. Together, these two studies are the earliest publicized examples of opinion surveys on fake subjects, a phenomenon known as a pseudo-opinion. At the time, the results of both studies amused laymen but were not immediately taken seriously in the field of public opinion because most professionals felt the studies were ridiculous and reflected negatively on their field. One exception, Stanley L. Payne, wrote about Gill's study in the 1951 The Public Opinion Quarterly journal article "Thoughts About Meaningless Questions" and called for further investigation into this type of non-sampling error.

Legacy

Despite Payne's call to action, pseudo-opinions remained largely unstudied until the 1980s, but in 1970 Philip Converse postulated that answering "don't know" is seen by respondents as an admission of "mental incapacity". In 1981, researchers Howard Schuman and Stanley Presser were unable to locate documentation for Gill's study and concluded it should be taken as an anecdote rather than a true study. Their research found that pseudo-opinions are a significant source of error but not as prevalent as Hartley and Gill's studies suggested.
The Metallic Metals Act is considered a classic example of pseudo-opinions and difficulties with close-ended survey questions and continues to be supported by later studies. By 1991, it had become standard practice to include a false question in opinion surveys to gauge the degree of pseudo-opinions. A study by the University of Cincinnati found 20 to 40 percent of Americans will provide pseudo-opinions because of social pressure, using context clues to select an answer they believe will please the questioner. This has occasionally provided a source for jokes on talk shows and comedy shows who air interviews to mock the respondents. Other studies have shown the phenomenon is not limited to the United States. In a 2019 opinion piece written for The Guardian, Richard Seymour speculated that most opinion polls only represent what respondents heard most recently in the news media.