Election Season Overwhelms with Stats, Numerical Literacy Needed

With only 39 days left until the presidential election, news publications and television talking heads are producing more poll results and statistics than ever, each purporting to be the most recent, most accurate information about how President Obama is edging out former Governor Romney in 4 percent of swing state districts where likely voters favored Senator McCain by 3:1 in 2008. All the numerical jargon does little to improve our understanding of how the candidates are polling nationally.

            Political pressure groups and PACs complicate whatever picture of the election we think we have, by conducting studies that we not only have difficulty understanding, but that also have an intentional internal bias toward a particular candidate or point of view. The “largest non-profit non-partisan organization the United States” Generation Opportunity, which claimed that “nearly twice as many young adults say they would rather work on Wall St. than protest against it.” This claim was in part backed up by a Harvard University study conducted in December 2011.

            The e-mail from Generation Opportunity, and the study from Harvard, were forwarded along to Jeremy Wolf, professor of government and doctoral candidate at the University of Massachusetts. Wolf is teaching “Topic: Inequality, Social Policy and the Politics of Method, Who Counts?” as a seminar in American government this semester. The focus of the course is to teach numerical literacy to students of politics, so that they can understand where bias comes from and the ways in which studies and surveys can be made more or less accurate.

            Using the Generation Opportunity e-mail as a jumping-off point, Wolf explained that there’s no easy way to tell if a survey is accurate. “There's no easy, foolproof shorthand,” he said by e-mail. “But, in general, it's a good idea to ask yourself who has shouldered the cost of conducting this survey, and why? Don't let a group off the hook just because you agree with their conclusions, and watch out for question that seem loaded.”

            There are a few things readers should always check when given a statistic claim. The first is the sample size of the survey, which for the Harvard study was over 2000 people, and limited to 18-19 year olds, and had a margin of error of 2.2 percent which “means that if you drew a random sample from this universe 100 times and asked them all the same questions, 95% of the time the answers would fall within 2.2% of the answers on this survey,” explained Wolf.

            But the underlying claim by Generation Opportunity is not backed up by the questions asked in the Harvard study. “The survey does show, as the press release claims, that only 2% of those surveyed had participated in OWS protests, and only 11% knew someone who had participated,” said Wolf, but “The…survey doesn't say anything at all about working for Wall Street.” Instead, the “non-profit non-partisan group” used an online poll they conducted themselves and constructed their press release in such a way that it appeared that this claim was backed by the Harvard study.

Wolf added that Generation Opportunity’s poll, like many political surveys, has three major problems. “Any survey conducted or commissioned by a group with a clear political agenda is automatically suspect. It's not the case that such surveys are never meaningful, but they should get some extra scrutiny,” said Wolf. Secondly, the press release did not include the actual questions used in their online survey, unlike the Harvard study. “If they really asked people "Would you rather be employed by Wall Street than protest Wall Street?" then I have limited faith in this survey,” admitted Wolf. “This is not a real choice; it's not as though anyone anywhere has ever been faced with the dilemma of joining an OWS protest or going to work at Goldman Sachs.”

And finally, “The press release claims to show that OWS hasn't really produced much enthusiasm among young people because it doesn't speak to their concerns. However, simply showing that 47%... would answer that they would rather work for Wall St than protest it doesn't show this it all. 26% would rather protest, and another 27% presumably don't know or don't care. Either way, it's not doing a very good job of demonstrating anything,” stated Wolf.

            Although this particular press release was misleading, the study itself was solid. Such claims are not uncommon. Remember to check sample size, the source of funding, the percentage of error and how the survey was conducted. Then you can truly be an informed voter.