Princeton Election Consortium

A first draft of electoral history. Since 2004

Weekend At Bernie’s (no, not that Bernie)

October 12th, 2016, 4:55am by Sam Wang

To the extent that the most statistically stable Presidential race in 65 years can be made to look alive, here it is. Right-click to adjust the looping playback.

Animated Median EV

I have a question for readers. To me, it is plain that this year’s race is statistically highly stable, i.e. it hasn’t moved up and down very much. I am occasionally met with incomprehension or disbelief, even after showing a graph like the one able. Is the difficulty simply the emotional nature of this year’s race? Or is something else at work?

Thanks to PEC reader David Elk!

Tags: 2016 Election · President

98 Comments so far ↓

  • Amanda G

    Sam, thank you so much for this site and for your hard work. I’m an economics professor, so I consider myself “in the business.” I will be using your site to teach my students about election polling and statistics this November.

    I think the best argument explaining doubts about the polling predictions is based on expected value (did you say this already?). Because Trump would be such a huge negative hit, even a small probability makes folks nervous, and we start to come up with all kinds of neurotic reasons to doubt the stats. I agree with you that this election has been remarkably stable. There’s no news here. (Actually, I’ve been irritated with Nate Silver for tweeting out small perturbations that rile folks up. I thought the whole point of his exercise was to get us past the area of the horse race based on the popular vote, which doesn’t matter anyway).

  • Kevin King

    Whoa! Alaska?!

  • Matt McIrvin

    Concerning Gary Johnson: Looking at historical polls (from Gallup at least), it seems to me that when a third-party candidate is polling as well in mid-October as Johnson is polling now, they don’t fade. Stein, maybe, will. But I don’t think most of Johnson’s voters are going to break for the major-party candidates; I think that by and large they will vote for Johnson.

  • km

    Maybe you need to show an example with high fluctuation and then mid fluctuation for comparison? (And, of course, keep your graphs with same y-axis scale.)

  • Meg

    OK…I tried to Tweet at you because I feel terrible about making you sad, especially since it seems to be due to my slow learning.
    But, here’s a question: If, due to our increasingly polarized electorate, there are only a very small number of folks who are true swing voters, then aren’t even small fluctuations bigger ones? Maybe this is a philosophical question, but I guess I’m asking what the definition of a volatile election would Be. Would it have to include swings that went beyond the margin of error? I understand that it is very stable compared to historical elections, but it seems that we’re unlikely to ever return to those kinds of elections. And, just as a point of gossip, I happen to have a whole bunch of Mormon friends, and in the last two-three days they have almost all announced they’re voting for McMullin. I realize my fb feed isn’t a scientific sampling, but I wouldn’t be surprised to see him win Utah at all. That’s it…I’ll try to be less dense in future comments (or go back to sitting on my hands). Thanks, Sam! You’re the best!

    • Sam Wang

      Hmmm. If you define a big swing as “a big fraction of the voters who are swayable,” then by definition all elections could plausibly be called volatile. So that seems to be inadequate as a definition. “Margin of error” only really applies to single polls. Once polls get cleaned up and averaged, the margin of error becomes very small.

      To me, a good yardstick for the size of a fluctuation is the percentage point. I agree that elections vary less than they used to…which is my main point. At some level, if they start varying more, that might be an indication that we have busted out of this trap we are in as a nation.

  • Meg

    Well, because to non-stats people it seems like the polls swing widely-HRC was up a lot in August; then her lead evaporated; then she went way back up after the debate….it seems like the polls have swung quite dramatically in response to particular events (I believe this is the esssence of Nate Silver’s argument). I think your answer to that is something about regressing to the mean, but to non-math people, it is hard to get. I should say that I’m speaking strictly for myself here….since you’re specifically asking for the confused to comment, I felt like I finally had reason to do so! lol, as the kids say….

    • Sam Wang

      Actually, my non-technical answer is that the race has not varied that much. I do not know how to express this more clearly. There. Is. No. Wild. FLUCTUATION.


      It just seems that way because you are worked up about the race and watching closely.

      So…when I post these graphs that show Kerry/Bush moving up and down, and Obama/McCain and Obama/Romney doing the same but less so, and then Clinton/Trump not moving much, basically this makes no impression. This makes me so very sad.

  • Lorem

    So, what I was thinking is that it would be nice if we could explicitly derive volatility from/for optimizing predictions. Here’s a rough sketch of an idea for doing so:

    Split predictions into state-by-state predictions (possibly popular vote predictions), then score each prediction by looking at the full time series of win probability (or popular vote prediction?) Maybe just average Brier scores (or some other metric if popular vote) from every day of predictions. So, with that scoring metric, we can numerically try different volatility assumptions, and see which one gives the best average score (assuming one volatility number for the whole season for all states).

    Then, we combine the state-level volatilities to generate the expected volatility for nation-level. (My statistics knowledge isn’t nearly strong enough for this, but it seems like in principle it should be possible.)

    I realize the whole process is pretty complicated and potentially assumption-dependent, but it seems like it could be interesting?

    • Sam Wang

      This sounds so desperate to me.

      Your assignment is to read about data dredging and p-hacking.

    • Lorem

      I guess it is kind of desperate. It just feels more closely connected to what our objective is than just taking standard deviation (though honestly, I suspect the answer would be similar).

      I feel a little unhappy with your decision that ~”recent election SD has been 2.2%, so let’s set 3% to be safe and use a fat-tailed distribution”. It feels like this is a symptom of predictions being to some (relatively small) extent conservative for the sake of being conservative, rather than for the sake of making accurate predictions – which is why I think having an target metric for what kind of prediction is “best” would be nice. (Mind you, every other site seems to be significantly worse in this regard, and I’m not that confident that being more certain would give better predictions, so it’s not a huge complaint.)

  • Jackson

    Trump surrogates have been calling out the polls very recently about not being accurate because they are not capturing the “secret” Trump supporters that will rise up and turn the tide. Seems you have mentioned this in the past – is this indeed possible?

  • Mike Z

    Sam, it is a terrific question. Some of it is how horrified many of us are at the idea of a Trump win, since he is so beyond the pale of a civil society. For others, such as those who make a living based on ratings or clicks, such as one of your best known counterparts, there is a vested interest in portraying the race as razor close, if only to keep readers coming back. It is not a terribly interesting story for aggregators to say the race is stable and Trump is done. Even on your excellent site, there has been quite a variance in probabilities this season, reflecting the polling data at the moment. The truth is, as you say, this is the most stable race since the Eisenhower years, but that does not make for a sexy story. Couple it with this being the most consequential election since at least 2000, and one of the three most of the past 70 years, and that is why people have trouble understanding a simple chart, imho

  • C Cain

    Yes, it’s been stable in retrospect. What if the HRC’s dive in September – when the outer likelihood band went below 270 EV – had continued? Then it wouldn’t have been stable. What I have found so unnerving is the extent of Trump’s support and the fact that it seems to fly in the face of what I see as rationality. If it is possible for so many people to buy into the idea of Trump as president, what’s to stop a few million more shifting over voting him in?
    The alleged rigidity of voter preferences and the alleged stability of the race are the same contention – one doesn’t explain the other. It stands until it’s contradicted, and this of all years, with the alt right turning on the “GOPe”, shows that things can shift suddenly and shockingly.

  • Matt McIrvin

    A lot of analogies have been made to the examples of large polling errors in things like the Brexit referendum and the Colombia FARC referendum. And, of course, there have been many years when the most prominent national polls missed by several points.

    Sam has good reasons why those analogies aren’t applicable to his projections, but I still do wonder every time whether the extraordinary success of state poll aggregation in Presidential elections from 2004 on isn’t a chance small-number effect. Every election is a test.

  • Runner

    I think the answer to Sam’s question to this site’s readers is found, in large part, in the theory of cognitive dissonance.

    Many following this election are seeing PEC’s (and others) stats showing how highly stable (statistically) this presidential election has been (in HRC’s favor). Yet, when they turn on the TV or radio, they hear about horse-race polls, large rallies, undecided voters, denials that the polling is accurate, or statements that the polling is skewed, and talk of October surprises! This leads to an instability in thought processes and cognitive reasoning which can be very uncomfortable.

    I like to explain the 97% Bayesian win probability for HRC in this manner, using legal terms:
    It is clear beyond a reasonable doubt that, if the election were held today, HRC would be elected president. It is also extremely and highly unlikely that anything between now and Nov 8 will change that result. Whether one is for HRC or DJT matters little—it is the math and statistics which tell the tale.

  • Kim Dick

    I’m wondering if current likely voter models are vastly underestimating just how discouraged Republican voters are. This is a very unreliable indicator, but NC records how many people of each party affiliation submit absentee ballots:

    Early voting by Republicans is 55% of its 2012 value. Democrats and independents are essentially unchanged.

  • Colin Wyers

    Sam, I have wondered the same question. I fit a LOESS to the HuffPost pollster data in R, and it looked like a remarkably stable fit line to me. Now, I just went with the defaults in ggplot2, and I should probably investigate that further if I want to make any kind of a conclusion from it — I’m sure you could make a smoother look more like, say, the 538 weighted average over time. But I’d like to see someone who claims the race is unstable actually demonstrate that you should. Every wild swing we’ve seen so far has mostly swung back — every time the race seems to swing towards Trump, it swings back eventually. That to me feels like people are just overinterpreting the noise, not finding real changes in the race.

    • Sam Wang

      The problem with this question is that curve-fitting inherently smears out the temporal aspects. Regression/lowess/etc are absolutely the wrong tool for understanding this question.

      I am forming deep doubts that evidence will be produced for this idea. This is turning into people *wanting* to see volatility where there is none. It is a mistake to be blinded by your biases. Just let the data talk to you.

    • Dave Rodland

      If only the data would speak up a little – it’s hard to hear over the noise. Besides, the noise is possibly the most interesting part of this right now.

      It is almost certainly too late in this election season to tackle a large, unfamiliar field in refinement of the political fix, but during the off-season you might take a look at the climate change and paleoclimate literature, Sam. They have been tackling the problem of extracting trends and sensitivity estimates from time-series data with multiple sources of variation for decades – there have to be some statistical approaches in there that you might find useful.

  • josh f

    Outside of the factors that allowed trump to succeed in primaries, the most interesting thing to me is the degree to which the MSM has actually helped trump .

    While hard-core gop-ers would prefer to talk about benghazi and emails for the rest of their lives, the # of trump scandals which have not been covered at all- much less in depth- is enormous. Granted, clinton has been investigated for decades and had millions if not tens of millions of public money spent which makes all the results mostly public. Trump’s investigations and evidence is often confidential or under seal, but still the sheer # of stories that have been skipped is astounding. In only a year they can only do so much, but it’s interesting to me that they felt they could attract more viewers presenting trump as near the same level as clinton… even if it made the viewers uncomfortable.

    I wish there was a way to quantify/verify this more explicitly beyond ratings. It’s easy to say it’s a profit motive, which it probably is too but it seems like it says something deeper than that.

  • Amitabh Lath

    It’s important to remember the only reason to care about a volatility metric is when the polls are used to project forward to Nov 8. High (low) volatility assumptions leads to a larger (smaller) red and yellow band on the right of the estimator history plot.