Princeton Election Consortium

A first draft of electoral history. Since 2004

Why did the polls seem so variable this week?

October 27th, 2016, 8:45pm by Sam Wang


Why do the polls seem so variable this week? The basic answer is that there were a lot of them. Outliers are an inevitable consequence.

Trump is going on about three polls that he likes best. They are all favorable to him – IBD/TIPP, L.A.Times/USC/Dornsife, and Rasmussen. They show a tie or a small Trump lead. Meanwhile, for some reason he is ignoring polls that show Hillary Clinton 12 to 14 points ahead. Huh, it’s so weird that he would do that.

Anyway, this amount of variation is totally natural. It’s kind of like watching dogs in the park. If you wait for ten of them, you’re more likely to see a big or small one. But what we want here is the median dog.

The more polls there are, the wider the range of outcomes that you’ll see. With the election so near, we’re getting megadoses of polls: I count around 17 in a seven-day period. With that many, of course you will get outliers. As a rough rule of thumb, when the frequency of polls goes way up, you should expect the overall range of outcomes to double, more or less.

In this case, imagine that among pollsters, the standard deviation of the Clinton-Trump margin among pollsters is 3.5%. If there are 5 polls available, about half the time* one of them will be more than one standard deviation away from the median. Under current conditions, a typical range would be Clinton +2% to Clinton +9%.

With 17 polls available, the range will be more like 2.2 standard deviations in either direction. In that case the likely range is a whopping 16 points – Trump +2% to Clinton 14%. Which is just about what we saw this week.

Don’t be like journalists who run off after the most extreme report – that is ridiculous. The only honest thing to do is to take a median or average of all the polls you can get your hands on. Right now, the collected wisdom of all the poll is that Hillary Clinton is ahead by a median of 6 percentage points nationally. Not 4 points, and not 8 points. Her lead is 6.0 +/- 0.9 % (n=12 polls over the period October 20-26, median +/- estimated one-sigma standard error).

Do I have any criticisms of those pollsters? No. They are experts in their field (except maybe Rasmussen – they need a Truth and Reconciliation Commission to repair their methods). There’s nothing wrong with being an outlier – every year it has to be somebody. The consolation prize is that lots of people pay attention to an outlier.

*For total nerds: I’m using a t-distribution with parameter 3.

>>>

I’ll be honest – sometimes I get a little impatient with readers who show up here mainly looking for reassurance in the Presidential race. I understand this is an attraction of the site. But occasionally when I run into people who say they come to my site and feel relieved, I detect a hint of implied complacency. That would be antithetical to why I do this!

I view PEC as a tool for directing your positive energies to make a difference. Of course we are interested in the Presidential race…but there is little that can be done to affect it. It is basically decided. A good way to work off any anxiety is to work on races where we genuinely don’t know what will happen. Senate control is totally up in the air. There is Nevada, North Carolina, Pennsylvania…so much to do. Donate your time and money to the side that you favor. Or, use the Competitive House Race Finder in the left sidebar.

Tags: 2016 Election · President

146 Comments so far ↓

  • anonymous

    I am just curious. Does your model account for this possible input? Thanks.

    https://twitter.com/chucktodd/status/793617765215170560

    • Sam Wang

      No, but we do have to wait longer than I would like to get updates.

      Also, nobody likes a complainer. If NBC wants to save money, they could demote Chuck Todd to part-time red/blue map duty and get some nonprofit-foundation types to talk about issues, for cheap.

  • Davey

    I am totally reassured by this. Not the polls, but that Dr. Wang is frustrated with people who use his data like some sort of mental opiate.

    People use polls to pretend what they want to happen has already occurred. They’re not for this. They’re like radar images of a hurricane. Is it going to hit us or miss us? We can guess, but who knows? And in the end, the unpredictability of hurricanes and elections is part of what it means to be human. Who wants to live in a world where you always know what’s going to happen next week?

    For me, the presidential polls are a really exciting live experiment. We have a ground game experiment going on – is that built into the polls or not? We have a demo experiment going on. Are there hidden minority voters, or hidden ashamed white voters? We have a gender experiment underway. How much more open is the electorate to a woman leader? The polls are creating blurry, uncertain insight into these questions. It will be exciting to have actual results to analyze.

    • Sherry

      You sound very young. For some of us, this is not an experiment, this is real human lives that can and will be affected for the next four years. It is, however, nice to see a site that helps me make sense of the polls. Well done.

  • Frank

    I just want to echo what others said about the importance of time. Election Day is in 10 days. If you take the biggest 10-day decrease in meta-margin and EV in the past 5-6 mos., and apply it to today’s nos., you get Clinton +1.68% and 288.

  • Marco

    Re: Fbi
    I don’t think it will have a huge effect. Are there really that many undecided that this would have an impact? I don’t think so. Will there be another “late October” surprise that is so late that is November? According to a British newspaper, the Daily Telegraph, Trumps campaign has been accepting donations from the Chinese. That is crime too, ain’t it? http://www.telegraph.co.uk/news/2016/10/24/exclusive-investigation-donald-trump-faces-foreign-donor-fundrai/

  • Roger

    Sam, based on how stable the election has been so far. Do you see much potential for a change catalyst, like the FBI director letter, to even affect the election?

    • Josh

      No.

      1. 10-15% of voters (and closer to 20% in some battleground states) have already voted. This doesn’t seem like a lot, but if you think about it, it is.

      Let’s say a state’s electorate would ordinarily go 55-45 for the D candidate, and early voting bears this out–with 20% of the vote in, the D candidate has 11 and the R candidate has 9. Of the remaining 80, then, the R candidate would have to win 42, or about 53%, to win the election

      This doesn’t *seem* like a huge stretch…but then you remember that it would involve flipping something like 6 of the 44 expected D votes, which is the equivalent of getting ~13% of remaining D voters to not only abandon their party but to then go and vote for an R candidate. In our era of extreme and entrenched polarization, getting 13% of expected D and D-leaning voters to turn on a dime and vote R is pretty much a pipe dream.

      2. So if flipping D voters is out of the question, what about undecideds and third-party voters flipping the election to Trump by choosing him by a wide margin because of this news? This, too, is extremely unlikely.

      First, there are almost no undecided voters left (and whether these voters are truly undecided, or they simply don’t know whether or not they will vote, is another story). The median from the last week’s worth of HuffPollster undecideds is something like 4.5%. Even if we make the assumption that all 4.5% of these voters both vote AND vote for Trump, Hillary would still lead nationally by about 2%. The race would be significantly closer, but she’d still be in the lead.

      Second, there are now far fewer voters who still plan to vote for Gary Johnson or Jill Stein who could theoretically abandon their third-party candidate to support Trump. Johnson, who was polling about 9% over the summer, is now down to about 5%, and Stein is down to around 2.5%. The vast majority of Stein voters are far too liberal/left to consider voting for Trump. And of Johnson’s support, about half would rather vote for Clinton than for Trump. So even assuming Trump could steal a disproportionate share of Johnson’s support away, at best, the race might be tied.

      At this point, there are no other voters left–everyone who has chosen to support Clinton or Trump has made their choice, and there are no more undecideds or third-party voters whose votes could be reallocated. In other words: even in this highly extremely scenario, the race might be tied. This is why Clinton today has a roughly 99% chance of winning.

      3. Now, given all of this, there’s still one outcome that could POSSIBLY tip the race to Trump: the FBI decides to, say, charge Clinton with an actual crime. The odds of this happening are vanishingly small, for several reasons:

      A) The FBI already thoroughly investigated Clinton for months and concluded in July that nothing criminal had happened.

      B) There’s no evidence that Clinton sent any of the emails recently uncovered.

      C) Even if Clinton did send some of these emails, there’s no evidence they had anything to do with her previous investigation, which was closed, and is still closed.

      D) Even if Clinton did send some of these emails–and even if they DID have something to do with her previous investigation–there’s still a very good chance there’s nothing criminal about them.

      E) Even if Clinton did send some of these emails–and even if they DID have something to do with her previous investigation, AND there’s a whiff of something criminal in them–there’s almost no way the FBI could figure all of this out and charge Clinton with a crime in the next 10 days.

      F) And even if scenario E did come to pass, what percentage of Americans would already have voted/made up their minds that this was all a witch hunt, anyway?

      ***

      Sorry to ramble, but sometimes it’s good to just take a deep breath and chill. Barring something TRULY extraordinary (read: NOT this email thing), Clinton will be the 45th Prez. Use some of that nervous energy to donate time and money to downballot races.

    • Shawn Huckaby

      My take is that it only hardens positions. Comey has already said they won’t even analyze the emails until after the election, which of course begs the question, why say anything at all?

      So this reinforces what both sides already thought. Could there be some small impact on the margins where more undecideds go to the ‘a pox on both of your houses’ position and stay home? Maybe, but I doubt it’s significant.

      Intetestingly, Republican activist, and Never Trumper Rick Wilson has been strongly hinting at one more major Trump bombshell that the Clinton team is waiting to release at the most opportune time. https://twitter.com/TheRickWilson/status/792210310572732416?s=09

      We’ll see how it plays out, but I would expect everything cancels out and we end up right at the same trend line.

  • Ed Wittens Cat

    Is Comey’s unsubtle political activism part of the increasing polarization of America?
    eg, dropping a “bombshell” on friday designed to let low information partisans cement falsehoods over the weekend.
    These emails are more nothingburger on observation.
    Same as Wikileaks– if Podesta and DNC got hacked where is Hillary’s hacked server?
    Praps Guccifer/GRU has hacked data, but theres no smoking guns in it– so better to not release it.
    And Wikileaks doesnt hack– only leak other ppls hacks. So who is the source? Is the FBI investigating that?

  • Roger

    Here is a good article, by one of the polling companies explaining why its polls are tightening.

    For that poll at least, and assume others, it is almost completely based on who in the sample are qualifying as likely voting and the reason for the tightening is previously Republicans and Republican leaning independents were somewhat depressed on Trump and therefore measurement in the poll were picking up that attitude causing some of them to be measured as unlikely voters. Now that bad news for Trump has waned, their outlooks are on the rise and therefore getting picked up as likely voters again.

  • Elizabeth

    Sam, sorry, I am one of those people fixated on the daily update. I am NOT complacent even after seeing your predictions, I am still terrified. But looking at that map keeps me off the Xanax.

    I am driven crazy by various Facebook groups who keep insinuating that the election is in the bag. We all need to go out and vote, because “It isn’t ovah till it’s OVAH!”

    • Sam Wang

      Nobody said you shouldn’t vote! Make sure to leave time for doing what you can in swing districts and Senate races.

    • Elizabeth

      There are 3 Democratic voters at my house this year. My timing was bad when I had my kids–like her sister, my younger one will have to wait till 2020 to vote in her first presidential election. I just hope that will be a less scary election!
      And I also hope that here in Virginia the blue trend will move towards our state government, which is still dominated by “The South Won the War” types. (!)

  • Barb

    I know we can be annoying, but please hold us, Sam. We’re so scared.

  • abe the scared dem

    sorry but going to ask even if this comment gets deleted, how do you think this email story will affect the outcome

    • Sam Wang

      It won’t affect anything. It’s a great example of how emotional the race is this year – but from a polling standpoint, it’s been an unchanging race. Why would some new addition to the oldest story of the campaign do anything at this point?

    • Matt McIrvin

      I can see it having some effect just by putting the old story top-of-mind.

      The last time Comey made a big splash, it looks like it was worth about one point of Meta-Margin and 20 electoral votes–it’s the little drop back in the middle of the summer. Now, on the one hand, a lot of people weren’t paying attention to the campaign then. On the other hand, people’s minds are more made up now. So, worst-case, I figure this could be comparable to that.

    • Matt McIrvin

      …though there were some other stories around the same time that put crime and terrorism in the news (the Dallas shooting and the Nice terrorist attack), so the effect of the Comey presser may be hard to separate out from those.

    • Ruth Rothschild

      Sam,
      Re: how the national House race is trending: The latest HuffPost national House poll shows that the national race is trending toward Democrats (46% to 41%). Yet, other polls show the race trending toward a Republican majority. And, it seems that in one of your articles at this site about the down-ballot races, you had indicated that the national House race is trending toward Republicans. Do you know what meta-analysis Huffpost uses compared to what you use that would show this disparity?