We are getting polls about voters wrong in an important election year
It’s a fun comedic line: “If the election were held today, some 89% of voters would be very surprised.” The kernel of truth in the joke is that polls measure the opinions of voters when many voters are not ready to be measured. That early poll panic is only one of many mistakes that some politicians and many news organizations commit when trying to understand public opinion.
'Precision journalism' and ethical practices
It may be time for a quick refresher course on the intersection of journalism and public opinion measurement. There actually is a term we can use. Philip Meyer in a 1973 book crafted “precision journalism” to mean the use of polls, polling and social science research techniques in news stories.
The world was very different when Meyer wrote his book. Nearly everyone lived in a household with one landline phone. Best practices meant that journalists were on the lookout for polls that did not take steps toward randomized selection, used too small a group, phrased questions badly or did not fully list methods used. The American Association for Public Opinion Research later codified these ideas into a good list of ethical practices in reporting polls. A very smart editor or reporter might even know what is meant by a margin of error and how to calculate it.
Relying on landline owners skews polls
Current conditions make it a bigger, but not unsurmountable, challenge for reporters using public opinion. Refusal rates are through the roof. Most people now have cellphones. Landlines are rare and declining. Relying on them skews any poll toward an elderly demographic willing to answer a call from a stranger. Pollsters now often rely on obtained cellphone lists or even panels of respondents often recruited online and stratified to achieve certain demographic parameters.
Big newsrooms and pollsters often are partners and collaborators on polls: Fox News/ Opinion Dynamics, Siena College Research Institute and the New York Times, CBS News and YouGov, Gallup/USA TODAY. Thus, it is even more important that newsrooms understand what the partner pollsters are providing and can interpret is reasonably.
One example of how things can go wrong came from a recent Siena College/New York Times Battleground States poll. Those who relied only second-hand on reporting of the poll would think that Donald Trump is leading Joe Biden in five key states, and that young and minority voters are drifting away from Biden. Online political columnist Jay Kuo did a great public service when he did a deep dive into the small print of the poll’s methods.
Kuo found that the poll under-sampled Democrats, liberals and Spanish-speaking Hispanics. He further noted that a lot of the coverage of the poll breathlessly relays findings from registered voters, but when one looks at likely voters the numbers are much closer. He also took issue with the poll’s very broad definition of likely voters and its sampling of young people.
Polls this far out are, at best, a snapshot of opinion at one moment. This one is a blurry image with the cat filter left on. Polls have had a rocky record in the past few election cycles, partly from excess reliance and inexact reporting, but also from inadequate adjustment to the changes in the current difficult-to-measure environment.
Mark Harmon is a professor of journalism and media at the University of Tennessee, Knoxville. He has presented public opinion research at numerous academic conferences.
This article originally appeared on Knoxville News Sentinel: We are getting polls about voters wrong in an important election year
