Princeton Election Consortium

A first draft of electoral history. Since 2004

Politics & Polls #8: How To Interpret Polls

August 18th, 2016, 9:36pm by Sam Wang


On Politics & Polls (SoundCloud, PodOmatic, and the slower-to-update iTunes): Julian Zelizer and I offer up a basic primer on how to make sense of the onslaught of polls. What should we think of a really surprising result? Are polling numbers twisting and turning with every week’s news…or are polls the best empirical measure we can count on in a crazy year? Want to learn a trick for aggregating polls that is so simple that it does not even involve arithmetic? All this and more…listen now!

Tags: 2016 Election

23 Comments so far ↓

  • Latichever

    What’s the deal with the LA Times poll?

    It’s a moving average of the same 3,000. Is it more a giant focus group than a poll?

    It’s always been much closer this cycle than virtually every other poll. Trump is +2 in its latest iteration.

    Is it worth watching for trends rather than as a snapshot of “if the election was today.”

  • J-D

    In the talk you mentioned that you got a better estimate of results from aggregates of pre-election polls than came from the actual early returns on election night.

    I’m used to the election (and pre-election) coverage from Antony Green, elections analyst for the Australian Broadcasting Corporation, and his election-night computer-modelling has (for a long time) systematically compared early returns with the results from the same polling place at the previous election, and systematically produced excellent forecasts on that basis. Of course the fact that this is feasible under Australian conditions doesn’t mean it would be feasible under US conditions, but I thought the comparison might interest you.

    • Sam Wang

      Yes, that is possible. However, it requires knowing something about past voting by specific location. I did that a few years ago for a major news organization’s election-night projections. I believe it is fairly common. However, this level of analysis is out of the reach of casual consumers of news.

  • Chip

    Interesting (if simplistic) poll-based analysis of House races finds 24 vulnerable Republican incumbents (less than the 30 needed to flip the House):

    http://www.dailykos.com/stories/2016/8/20/1561671/-View-from-the-left-Let-s-look-at-the-House

    • Michael Coppola

      My first question about that sort of analysis would be whether there is a credible Democratic challenger for each of those supposedly vulnerable GOP seats. You can’t win if you don’t play.

    • Sam Wang

      See this Twitter thread from earlier today, in which the topic came up with some real aficionados of individual districts.

  • Greg Gross

    I chuckle at how little I understand of the statistical analysis; folks, I was an English major! That’s why I am always thankful for the baked-cake metaphor Sam has used….

  • Jay Sheckley

    Best podcast yet. Lots of rivetting subtopics. Although, speciously, now I’m a bit fearful of both weighted and unweighted polls. But still more afraid of losing this best source of aggregated polls.
    The 19% of undecideds– that was for Congress not POTUS ? If so, what the % of Presidential undecideds, and how does that compare with this time last election?

    Concerning that most of interesting poll stats –PEC’s– Trump’s 11% is not nothing. So it troubles me that friends say _these_ are Trump’s odds: http://goo.gl/lDMd1q . So I researched the likehood of _that_, and was surprised to see instances like this: http://goo.gl/iQ3DQC and this http://goo.gl/8551AF . More importantly, I’d like to see voting made mandatory, and more publicized fact checking including during debates. I also wish commenters would comment about the podcast content.

  • Eric Walker

    Perhaps OT, but the Senate “Power of One” numbers haven’t budged in a long time now, though I am pretty sure that there are quite a few new polls. I know they are medians, but I would have thought that at least one or two might have moved in all that time. Is the updating proceeding OK?

  • Nicholas Warino

    Fwiw I don’t see episode 8 on iTunes (or overcast)

  • Justin

    Sounds like a great episode, but I won’t be able to listen for a couple of days–can anyone tell me what the polling aggregation trick is that’s “so simple that it does not even involve arithmetic”? Dying to know… :)

    • Sam Wang

      Wait for 3 polls, calculate the margin, and take the median. All you have to do is be able to sort numbers in order. No averaging.

    • David Fry

      “Calculate the margin”. Arithmetic! :-)

    • Sam Wang

      Good point. Can’t escape that devil, subtraction.

    • pechmerle

      It’s true, no arithmetic required! In this simplest of all possible worlds, that’s already been done for you. Huffposter, on its Clinton vs. Trump polls page http://elections.huffingtonpost.com/pollster/2016-general-election-trump-vs-clinton, also linked on right column of this site as polls: Clinton vs Trump national, shows the spread for each poll that it lists. (Many other poll listing sites and articles also already show the spread from each listed poll.) Subtraction by you not required. Last three — as of this minute — are Clinton +5, +4, +15. Ergo, pick the middle number, median is Clinton +5 and “Bob’s your uncle.”

  • George

    Re polls, etc. I noticed that your random drift and Baysian numbers each bumped up 1 point without any new poll numbers that I could discern. So – the question is …. if numbers stay stable, how does the length of time to election factor in to the odds calculation? Is it linear over time or some other pattern?

    • Michael Tiemann

      The random effects are uniformly random (just like a spherical cow of uniform density has uniform density). So if the number stay stable, the apex of the orange triangle and the left side of the yellow trapezoid just move to the right, one day at a time. This makes it look like the numbers are tightening, but that’s only because every day of certainty removes one day of random drift. On the eve of the election, the EV estimator will be exactly the meta-margin, and uncertainty boundaries will be the EV’s margin of error.

    • George

      To Michael Tiemann – that’s what I figured, but wasn’t sure if the removal of each day of uncertainty was a linear function, as depicted in the chart. You seem to be saying, yes, linear. Thanks.

    • Jay Sheckley

      I like what Michael Tiemann said. But PEC does add in new state polls whenever theyre available, outright replacing numbers on polls which are done over. Trump’s popularity numbers have been so low that the candidate said yesterday he “regrets some” of his statements and didn’t mean to offend people. Watch the header here to see if that helps him.