Trust the numbers? What to think when the polls don't measure up | St. Louis Public Radio

Trust the numbers? What to think when the polls don't measure up

Nov 11, 2016

How did the polls get it so wrong?

This is a question that has dominated headlines in the days following Tuesday’s historic election.

Professors Matt Carlson and Terry Jones joined Don Marsh to discuss polls and media coverage in the 2016 election.
Credit Kim Oswalt | St. Louis Public Radio

On Friday’s St. Louis on the Air, we discussed the role of polls and their use in media coverage of the 2016 election with guests Terry Jones, Founders Professor of Political Science and Public Policy Administration at the University of Missouri-St. Louis, and Matt Carlson, Associate Professor of Communication at Saint Louis University.

While Jones explained it would likely take three-to-nine months to find out exactly what went wrong with polls this year, he said the American Association of Public Opinion Research has already formed a taskforce to address the issue. He said investigators will probably focus on four main areas:

1. The “Shy Trump” Vote

This hypothesis is based on the idea that people who were embarrassed or scared to say there were voting for Donald Trump either avoided polls entirely or gave false answers.

2. Turnout

Overall voter turnout was much lower this week than it was in 2012. Jones and Carlson emphasized the importance of remembering that people who answer poll questions are not guaranteed to cast their ballots early or on election day.

3. Third-Party Vote

Jones explained that third-party candidates ultimately received fewer votes than polls originally indicated. Some of those disappearing votes could have been redistributed to Trump.

4. Anti-Establishment Voters’ Rejection of Pollsters

Jones explained that the strong anti-establishment sentiments of even Trump’s most ardent supporters might have made them more likely to reject requests to participate in polls, regardless of how outspoken they chose to be in other arenas.

Did the polls actually let us down?

While many of the polls did ultimately offer inaccurate predictions, Carlson explained that polls are often misinterpreted as guarantees of victory or loss.

“Things that happen 30 percent of the time, happen,” Carlson said. “And so I think that we saw 70 percent as certainty, and that’s not certainty.”

Journalists, Jones said, tend to misunderstand the margin of error in probability measures like those seen in polling data.

“It’s a normal curve,” he said. “It’s sloping down; it’s not a rectangle.”

Do polls reveal media biases?

“Polls are in the accuracy business, not the advocacy business,” said Jones.

Carlson explained that accusations of media bias have been around for over 50 years and are not new to this election season. He suggested that rejecting the media as liberally biased offers a convenient argument for discounting information.

“I think that there are biases in the media, but they’re more about sensationalism, about horse race, about excitement than they are about having a liberal leaning,” Carlson said.

Pollsters in this year’s election did not rely solely on landlines to reach potential voters. Jones said that about 50 percent of responses were gathered from cellphones and 50 percent from traditional landlines. This was intended to help prevent any skewing of the data in favor of older voters who are more likely to own a traditional phone than are millennials.

While Jones explained that the polls have never been off to this degree before, Carlson noted that we often forget that they were wrong in 2012 as well. In that election, polls predicted Barack Obama would win the presidency, but they underestimated the margins by which he would do so.

“I think that polling error was interesting,” Carlson said. “It allowed people to think that the [Hillary] Clinton voter turnout machine would be similar to the Barack Obama turnout machine, and that turned out not to be the case. So I think there was an assumption built in that maybe the polls were not picking up Clinton voters rather than not picking up Trump voters.” 

St. Louis on the Air brings you the stories of St. Louis and the people who live, work and create in our region. St. Louis on the Air host Don Marsh and producers Mary Edwards, Alex Heuer and Kelly Moffitt give you the information you need to make informed decisions and stay in touch with our diverse and vibrant St. Louis region.