Princeton Election Consortium

A first draft of electoral history. Since 2004

Election thread #2

November 8th, 2016, 9:34pm by Sam Wang


11:44pm: The business about 65%, 91%, 93%, 99% probability is not the main point. The entire polling industry – public, campaign-associated, aggregators – ended up with data that missed tonight’s results by a very large margin. There is now the question of understanding how a mature industry could have gone so wrong. And of course, most of all, there is the shock of a likely Trump presidency. I apologize that I underestimated the possibility of such an event.


11:12pm: Using the projections of the NY Times, Donald Trump is outperforming his pre-election polling margins by a median of 4.0 +/- 2.6 percentage points (the 8 states in the Geek’s Guide). In Senate races, Republicans are outperforming by 6.0 +/- 3.7 percentage points. A five-percentage-point polling miss would be a tremendous error by modern polling standards. Undecided or minor-party voters coming home to Trump? Shy Trump voters? I don’t know.


10:38pm: At the Senate level, the polling error is looking pretty substantial at the moment, maybe 5 points toward Republicans. A polling error of this size would be the largest on record, at least in a Presidential year. I was wrong to downplay this possibility.

We still have to see what will happen at the top of the ticket. But obviously, with a Meta-Margin of only 2.2%, an equally large across-the-board polling error at the Presidential level would suggest a Trump win of the Electoral College.


9:31pm: NYT presidential tracker showing things very close. Looks like a late night. And perhaps bug cookery for me.

Tags: 2016 Election

163 Comments so far ↓

  • Graham

    Remember that comment about how it irritated you, people coming to your website for reassurance….? I was one of them.

  • BB

    I’ve eaten a chocolate covered cricket. It wasn’t bad.

  • MB

    Sam, wtf?

  • Froggy

    At this point a bug is the least of our worries.

  • kim

    Nope, you just can’t be eating bugs, Sam, the country cannot afford it!

  • Barbara Edelman

    Looks like opinion polling has gone the way of shorthand and mapreading.

  • Paul

    Hi. Watching this nervously from Europe, as Florida seems to go to Trump, with not enough Broward vote left to turn it to Clinton anymore.

    Looking at exit polls, I notice that if Trump wins Florida in particular and Presidency in general, he may have James Comey to thank for.

    Comparing those who decided before last week to those who decided last week, Clinton had 6% drop from 50% to 44% and Trump 3% rise from 47% to 50%. This is only 10% of constituency, but in tight race enough to turn it to Trump.

    Nationally, exit polls show a 8% drop for Clinton from 50% to 42% in those who decided last week compared to those who decided before it, but in those who decided in the last few days a 5% rise to 44% from those who decided before that in last week (39%), probably as a result of Comey’s second letter.

    Looks to me that without Comey’s first letter, this would have probably been a non-contested win for Clinton.

  • codetaku

    99%, huh, Sam? Your unscientific feud with Nate Silver is horrifying. Polling error is real and you should take average polling error into account next time.

  • KT

    Holy shit. Florida is gone.

  • Amitabh Lath

    Perhaps Sean Trende was right about all those white voters that didn’t vote for Romney.

  • Paul

    Just to add to my comment about Comey effect in the exit polls, what made me think he may have decided the election is that NYTimes live presidential forecast gives 59% chance for Trump presidency now.

    That said, I find it absolutely ridiculous that a letter about FBI having a Clinton’s aide’s ex’s computer, without any substance from what there is, may have decided the vote. It’s hard to believe that that was what made people’s minds.

  • hk

    NYT now projects Trump wins with 275 EV

  • bks

    Betfair (UK betting site) just flipped to Trump.

  • Bruce Sands

    Well, the therapists will do very well for the next four years if the crazy man wins,

  • mike

    Even if Clinton pulls out a win, right now that 99% number is looking spectacularly bad.

    And that 82% for a senate call that’s pretty much over already, not much better.

    • Jay Sheckley

      “Even if Clinton pulls out a win, right now that 99% number is looking spectacularly bad.”
      No,99 wasn’t the margin, it was for the win.
      If she wins, PEC was correct on that.
      And since this site was the outlier, and Sam told us and told us to discount outliers, if there’s any face ala egg it’s on all of us.
      Luckily, I don’t even own a bug.

  • Scott J. Tepper

    Nate Silver is looking pretty good about now. No matter how it comes out.

  • Dave

    Any thoughts on why the polls would be this far off?

    • George H

      Maybe there is a silent voter for T after all. Stunning.

    • Sebastian

      Frankly, they weren’t that far off.

      Go to any polling aggregator site and look at the state polls for FL, MI, NC, VA.

      Yes, Clinton was leading in the average, but there was a lot of volatility and there plenty of polls that showed Trump up in all those states. Only in Wisconsin were there none, but they had fewer polls than most states.

      Basically, it seems this unfortunate experiment proved Nate Silver right. There is always a baseline chance that the polls are off in one direction (by a moderatly small amount, it didnt take that big a swing) for all states. The uncertainty was higher than most models accounted for and we walked right into one of those scenarios where they were off by 2% or so on avager, but in one direction and with particular focus (higher turnout and Trump performance among rural and uneducated white voters).

      Hillary Clinton is still on track to win the national popular vote, but Trump had hidden strengths in the electoral College (OH, WI, MI).

  • Indomitable Ted

    Comey. Comey messed up the early voting. This is not good.

  • JSchmoe

    “>99%”

    … Hubris

  • Digger

    I feel like I’ve been had. This is nothing short of terrifying.

  • Josh

    Must be nice to have tenure. If I screwed up that bad in a real job in the real world there would actually be consequences.

    • Sam Wang

      Perversely, it would be the best thing for me because then I can abandon the hobby!

    • Mikey

      Please be fair to Sam, he has publically put his reputation on the line and provided an interesting website for you for free. Right now it looks like he will have egg on his face so he’s paying the price. He’s not the only one that got it wrong (if Don wins), all the famous analysts made the same mistake. Not me though- I did my own analysis and bet bigly on Trump, I win £££ if it comes through. But you should respect Sam more than ppl like me that only come out to claim they were right all along after the fact.

    • Omg

      Seriously what are the chances of this election actually being rigged? If these results hold that is a huge error in polling. How can that happen?

    • Tim

      Academics….I am one as well. Everyone is so aligned with the left in academia they can’t see what is happening with working class population. Wang was incredibly wrong and should suffer reputationwise.

      Silver, again, was the most right.

    • Rachel Findley

      Sam Wang’s tenure is in neuroscience. Election work is extracurricular. So tenure is not involved.

      What about polling as a science? It appears that the ground shifts about as quickly as it can be surveyed.

      Now I’m going to bed. I’ll wake before dawn to see how it all came out.

    • Stephen R. Diamond

      “then I can abandon the hobby!”

      Perhaps you should, if said hobby is based on the premise that the mathematical analysis of polls alone suffices for prediction. Maybe some historical and sociological analysis is necessary.

    • Mary B.

      How can this be? His own campaign didn’t think he was going to win with the polling they had. Members of the GOP didn’t think he would win. I feel sick and scared. Somebody hold me.

    • Arod

      Did realize this was just a hobby. The whole “Princeton Electoral Consortium” title and Princeton.edu address made it seem like a genuine academic endevour. Maybe retitle it “Sam Wang guesses at stuff” with a .com address so people know not to give any credence to it.

    • Josh

      so OMG, immediately you use the logic that people have been ridiculing Trump for all month. If she loses it must have been chicanery.

  • Stephen R. Diamond

    The prediction markets also will take a big reputational hit. It was .9 for Clinton on Predictwise. David Rothschild hasn’t commented since that!

  • Anonymously

    Brings to mind Millerism, and the Great Disappointment.

    https://en.m.wikipedia.org/wiki/Millerism

  • Daniel

    Why are polls off? Fear of women having power has powered Trump from the beginning.

  • George H

    nyt 77% T. 538 66% C.

    • Pat

      Any insight into why such a large margin between the two? Is it different interpretation of exit polls?

  • Solomon Kleinsmith

    Which demographics underperformed and overperformed vs the models most pollsters were using?

  • Josh

    First of all, none of this changes the quality of Sam’s work, and “hubris” has nothing to do with it.
    If the polls are systematically off nationwide by 4-6% I have to wonder if the election was actually hacked.

    • George H

      totally

    • mike

      This absolutely changes the quality of Sam’s work. Based on polling, there can be mistakes which will give bad predictions. But there’s no excuse for 99% certainty that is wrong or even a tiny win.

    • JSchmoe

      My ‘hubris’ comment was not applied to the broader body of the work here at PEC or Sam’s work.

      It was specifically aimed at allowing the “>99%” figure to ever find its way into the top-line.

      Just that …

    • Eric

      How do you “hack” a decentralized voting system that is essentially off the grid?

      No, this is about the silent Trump vote, the underestimated impact of Comey’s letters and lots of poll genies who lean left anyway.

      I’m furious with Comey. His actions were instrumental in costing Hillary the election.

    • Arod

      That’s bs. It doesn’t cast doubt on his whole life’s work or anything, but if the polls are all off by that kind of margin the there is a systematic error in the methods and the pollsters are responsible for that.

    • kim

      agreed. Trump was so sure he would win.

    • Jeff

      What about a systematic error in polling model and execution? All the polling models build samples based upon expected voter profiles. If they miss the model, and do so systematically, then the poll is off. That’s why we don’t determine elections based on polling. We do it at the ballot box. Oh, wait, we don’t use ballot boxes any more. No need to stuff. So maybe you’re right. It has never been easier to hack democracy. What are we thinking about as a nation?

  • Graham Strouse

    I may have to pay a visit, Doctor Wang. We might need to discuss some things.

  • Robert M.

    You can’t be cavalier about eating bugs any more. If Trump really does win, we’re going to have to ration those out as currency and/or food.

  • counsellorben

    Bad news, Sam. The median Trump EV count just went to 244.

  • GM

    As a Sandersnista, it feels like 2004 to me, when lefty Dems were told that we had to settle for a centrist candidate because ours was ‘unelectable’, only to lose the election. Maybe the superdelegate thing gets fixed before 2020 (Warren?).

    On the bright side, my wife and I are already expats, so we don’t have to spend the night drunk wondering where we will move to.

    Sam, as badly as I feel, you must feel 1000x worse. Hang in there, guy.

    • Michael B.

      I’m still hoping Bernie will run again!

    • fred flint

      Bernie would have lost even worse.

    • Scott

      Superdelegates did not decide the primary.

    • Ruth Rothschild

      Lucky you, GM. I’m looking to become an expat, too. Where’s a good place to move to to be an expat?

    • Jeff

      Sanders was sold down the river by the “Guardians” of the Democratic process. Between his story and wiki-leaks revelations on the collusion between Clinton’s team and the leadership of the Democrat Party, it looks like the “fix was in”.

    • GM

      Ruth, it’s a tough call. The entirety of the Western world is dealing with the resurgence of racialized nationalism under the guise of populism – look at Britain w/ Brexit/UKIP. I don’t think anywhere is safe.

  • Dissenter

    I told you bro! I told you about the polls! It keeps happening!

  • Pseudonym

    If Trump gets 300 electoral votes, does the bug eat you?

  • KT

    Trump has won Ohio. Seems like Trump will take Michigan as well. The Blue Wall has fallen.

    At 538, Clinton’s chance of winning has dropped to 44%, and Trump is now at 55%.

    I think… it’s over. It’s the beginning of the end, at least for all the so-called polling and statistical forecasting. All of them were wrong.

  • bks

    The Comey letter had a profound effect. It was too late in the process to be picked up by the polls.

  • GuyWhoShowsUpEvery4Years

    I really don’t care about Sam having to eat a bug now…we may all be eating s**t for 4 years…

    • Al

      Fours years? Don’t forget about the Supreme Court, the economy, our relations with innumerable countries just to mention a few things.

  • Ken L

    Very tasty with your favorite beer or tequilla: http://www.exploringoaxaca.com/gastronomy,oaxaca-city,chapulines/

    Hope in doesn’t come to it.

    • Ruth Rothschild

      Ken L.,
      Looking more and more like Sam will have to eat those bugs unfortunately. NY Times is now giving Trump > 95% chance of winning. Only a miracle will reverse this into Hillary’s favor. That’s likely not gonna happen. Stock Market is tanking. I fully expect a recession worse than we had in 2008. We’re basically screwed. I’m wondering if the election has been hacked. This is just a disaster and absolutely terrifying. We’re in for at least 4 years of a dictatorship. Hard to believe that with the misogyny, bigotry, and racism that there were so many hidden Trump voters. Looks like any checks and balances will be the checks that are the hard-earned money of the middle class will go into the bank balances of Trump and the 1%. :-(

  • George H

    for the conspiracy, during Comey letter 24 millions votes cast, if I remembered AP report

  • Wongstein

    Does anyone understand how NYT’s electoral vote predictions work? Because even if Trump does win Florida and North Carolina and New Hampshire and Iowa and Nevada it still would not add up to their totals…

    Sam would have to eat a bug though

  • Larry G

    I guess if we lose Michigan it’s over, put your money on war stocks. We are going back to war. This time it’s not going to be good.

  • Adam

    Even Nate Silver isn’t looking good with this. She is getting smacked in so many states that were supposed to be close.

    Everything is down to PA, MI and WI, and I dont think she wins any of them

  • Jay Bryant

    I think no one anticipated the enthusiasm of the rural folks, and polls didn’t pick pick it up. That’s not Sam’s fault.

  • Jeff

    Sam,

    It’s not cooking bugs that you have to worry about. It’s your career as a pollster. 99%? Ha. You totally missed the boat here and led us all down a path to . . .

  • Dave Kliman

    when the results are simply so far away from ALL the polls, isn’t that when to suspect a rigged election?

    • Omg

      That’s what I was thinking. How could polling be this far off? There has to be an explanation.

  • Mark Buckley

    99% lol – time for a new hobby

  • Al-Zamar

    Given what we’re seeing, Sam was obviously wrong to so cavalierly dismiss Trump’s chances of winning. However, given the empirical evidence at his disposal at the time he made his prediction, there’s really no possible way Sam could have predicted or even fathomed a 5 point polling error. It’s the biggest polling error in a presidential election year in history. It’s simply the nature of statistics that really unlikely events are hard to predict with any precision.

    • mark-m

      Sam gave his prediction based on data. It was a prediction and he is looking at models based on polls. It is clear that the polls were systematically off.

    • 538 Refugee

      Correct. This is a polling miss. You can only aggregate the data available. Even Republican pollsters didn’t see this one coming.

  • 538 Refugee

    At the moment I feel like choking down a handful.

    https://en.wikipedia.org/wiki/Pogonomyrmex_maricopa

  • George H

    nty has given Trump is near certain win.

  • Lou

    Maybe you were never wrong, Sam. Maybe Trump just won with a 5% chance. Remember how you said in interviews that Trump might win if weathers storms suppressed the vote? I know you were being sarcastic BUT…maybe this is similar…a different KIND of storm. Facism. Facism is a perfect storm of a lot of things wrong in a society… a cancerous or viral form of government. No one could stop that, not if the country just lost its mind. Just think about it. After the Trump tape it should’ve been impossible for him to win. But that was one of many horrible things that should’ve disqualified him. And yet here he is…winning. Fascism in Italy and Germany was the fault of the people, and let’s be clear on that. Tonite is a choice, born of madness. And a perfect storm of ego and insanity could only make that possible. Thank you for your fine work. You did NOTHING wrong. Cheers

    • Anje

      And the Supreme Court slaughtering Voters Rights, there was massive suppression, Trump’s team didn’t even try to hide it. They flat out requested voter registrant info based on ‘race,’ then closed polling sites in those specific areas. NC was particularly obvious. There was a shooting in LA, there were people walking around armed intimidating people, Muslims being harrassed, Trump’s team trying to get voter ballots & poll worker info in Nevada via law suit. The insanity is unreal. Thankfully in that last case the judge kept her head, because the last thing any of those voters & volunteers need is for Trump’s ilk to get their info. The U.S. has gone insane.

  • James

    This is why it is foolish to make such a confident prediction >99% based on such a small sample size. Not enough data to know how likely events at the tails of distributions really are

    • George H

      don’t think so. A model is a model. it has worked for 3 elections. If there is a new undercurrent many people overlooked, and the polls did not capture, it did not capture.

    • AA

      Sam has a aggregation model. In the end it is using data spewed out by various states poll. I think it has a strong underpinning. A model is not helpful if the input data is off. It seems the polls were off.

    • Some Body

      But the win probability is distinct from the aggregation model. James is absolutely right. Predicting future drift based on MM history alone was never the right thing to do. It essentially ignores the chance of systematic polling error, and we’ve seen such errors crop up regularly in other elections worldwide, so this chance could not be relegated to the tail of the distribution, even if it’s a t distribution.

Leave a Comment