Polls and Overconfidence in Election Outcomes

My piece on the shock of the 2016 election results is up at the Washington Post’s Monkey Cage blog. There’s a lot that was edited out for space, and I’ll save most of that for a lengthier treatment on this subject at a later date, ideally once the 2016 ANES data is available.

The main takeaway concerns what’s going on in the right-most set of responses on the graph below. There was a huge and unexplained uptick in voters convinced that BOTH candidates were going to win in 2012 “by quite a bit”—far outside what you would expect given the actual margin that year. And unlike in past close races, more educated voters were the ones most likely to demonstrate this overconfidence. We don’t yet know whether the same dynamic applied in 2016, but it certainly seems plausible given the shock so many of us well-educated political observers expressed about the results last week.


Source: ANES cumulative data for survey responses.

Couple of things that have come up since publishing the piece that I think are worth emphasizing:

(1) Were the 2012 results unusual for reasons mainly having to do with the ANES? This is something I wondered about. The ANES shifted to a mix of online and face-to-face interviewing that year. But in supplementary analysis, the survey mode made no difference in the percentages. Still, I plan to examine other versions of these questions on other surveys to make sure I’m not making too much of a single set of survey results.

(2) The headline suggests there’s a casual argument here, but the evidence presented is admittedly circumstantial. Based on my interviews with journalists and pollsters, I do think the proliferation of polling aggregators and forecasting sites is a likely explanation for the results in 2012—and what I expect will be similar results in 2016. That said, it’s not the only possible explanation; for example, mass polarization more generally is a factor. Nor is it clear whether the public is selectively misinterpreting poll results or whether coverage of the election is what matters and the impact of the polls is more indirect: influencing newsroom decisions in ways that differ from the past. I suspect a combination of both effects, but that’s difficult to parse empirically.

I will add, and this is mentioned only briefly in the blog post itself, there has been a marked decline in “Don’t know” or “Depends” responses on the first question where voters are asked to offer a prediction about who is likely to win the presidential election. Presumably the widespread availability of polling data is a major reason for this dramatic drop-off.


Source: ANES cumulative data.

Lastly, it is worth pointing out that even though there is a good chance that voters’ expectations were off this year, it is not as though voters have a perfect track record in that regard prior to the proliferation of media polling. That is, even if voter forecasts in general have been some of the best available predictors of actual results (as Andreas Graefe shows in a recent analysis in Public Opinion Quarterly), sometimes events overtake projections. In 1980, the ANES failed to capture the late movement toward Reagan in voters’ predictions about the outcome of the race. The difference that year was that most voters on both sides were still predicting a close race, making the outcome somewhat less of a shock.





Comments are closed.

Follow me on Twitter

%d bloggers like this: