11:44pm: The business about 65%, 91%, 93%, 99% probability is not the main point. The entire polling industry – public, campaign-associated, aggregators – ended up with data that missed tonight’s results by a very large margin. There is now the question of understanding how a mature industry could have gone so wrong. And of course, most of all, there is the shock of a likely Trump presidency. I apologize that I underestimated the possibility of such an event.
I agree w/@NateSilver538 that there was high uncertainty, much more than I assumed. Median polling error 4% Presidential, 6% Senate so far.
— Sam Wang (@SamWangPhD) November 9, 2016
The American people have rejected the person they thought was qualified in flavor of the one they thought was not.
— Greg Dworkin (@DemFromCT) November 9, 2016
11:12pm: Using the projections of the NY Times, Donald Trump is outperforming his pre-election polling margins by a median of 4.0 +/- 2.6 percentage points (the 8 states in the Geek’s Guide). In Senate races, Republicans are outperforming by 6.0 +/- 3.7 percentage points. A five-percentage-point polling miss would be a tremendous error by modern polling standards. Undecided or minor-party voters coming home to Trump? Shy Trump voters? I don’t know.
GOP polls weren’t predicting this night either. Senate, House, Gov sources I talked to all expected Clinton would win per their polls
— Jessica Taylor (@JessicaTaylor) November 9, 2016
10:38pm: At the Senate level, the polling error is looking pretty substantial at the moment, maybe 5 points toward Republicans. A polling error of this size would be the largest on record, at least in a Presidential year. I was wrong to downplay this possibility.
We still have to see what will happen at the top of the ticket. But obviously, with a Meta-Margin of only 2.2%, an equally large across-the-board polling error at the Presidential level would suggest a Trump win of the Electoral College.
Trump owns the party whether he wins or loses
— Gideon Resnick (@GideonResnick) November 9, 2016
9:31pm: NYT presidential tracker showing things very close. Looks like a late night. And perhaps bug cookery for me.
Remember that comment about how it irritated you, people coming to your website for reassurance….? I was one of them.
I’ve eaten a chocolate covered cricket. It wasn’t bad.
Sam, wtf?
At this point a bug is the least of our worries.
Nope, you just can’t be eating bugs, Sam, the country cannot afford it!
Looks like opinion polling has gone the way of shorthand and mapreading.
Hi. Watching this nervously from Europe, as Florida seems to go to Trump, with not enough Broward vote left to turn it to Clinton anymore.
Looking at exit polls, I notice that if Trump wins Florida in particular and Presidency in general, he may have James Comey to thank for.
Comparing those who decided before last week to those who decided last week, Clinton had 6% drop from 50% to 44% and Trump 3% rise from 47% to 50%. This is only 10% of constituency, but in tight race enough to turn it to Trump.
Nationally, exit polls show a 8% drop for Clinton from 50% to 42% in those who decided last week compared to those who decided before it, but in those who decided in the last few days a 5% rise to 44% from those who decided before that in last week (39%), probably as a result of Comey’s second letter.
Looks to me that without Comey’s first letter, this would have probably been a non-contested win for Clinton.
99%, huh, Sam? Your unscientific feud with Nate Silver is horrifying. Polling error is real and you should take average polling error into account next time.
Holy shit. Florida is gone.
Perhaps Sean Trende was right about all those white voters that didn’t vote for Romney.
Just to add to my comment about Comey effect in the exit polls, what made me think he may have decided the election is that NYTimes live presidential forecast gives 59% chance for Trump presidency now.
That said, I find it absolutely ridiculous that a letter about FBI having a Clinton’s aide’s ex’s computer, without any substance from what there is, may have decided the vote. It’s hard to believe that that was what made people’s minds.
NYT now projects Trump wins with 275 EV
Betfair (UK betting site) just flipped to Trump.
Well, the therapists will do very well for the next four years if the crazy man wins,
If Paul Ryan didn’t have one, he sure will be looking for one now.
Even if Clinton pulls out a win, right now that 99% number is looking spectacularly bad.
And that 82% for a senate call that’s pretty much over already, not much better.
“Even if Clinton pulls out a win, right now that 99% number is looking spectacularly bad.”
No,99 wasn’t the margin, it was for the win.
If she wins, PEC was correct on that.
And since this site was the outlier, and Sam told us and told us to discount outliers, if there’s any face ala egg it’s on all of us.
Luckily, I don’t even own a bug.
Nate Silver is looking pretty good about now. No matter how it comes out.
Just pretty good?
Any thoughts on why the polls would be this far off?
Maybe there is a silent voter for T after all. Stunning.
Frankly, they weren’t that far off.
Go to any polling aggregator site and look at the state polls for FL, MI, NC, VA.
Yes, Clinton was leading in the average, but there was a lot of volatility and there plenty of polls that showed Trump up in all those states. Only in Wisconsin were there none, but they had fewer polls than most states.
Basically, it seems this unfortunate experiment proved Nate Silver right. There is always a baseline chance that the polls are off in one direction (by a moderatly small amount, it didnt take that big a swing) for all states. The uncertainty was higher than most models accounted for and we walked right into one of those scenarios where they were off by 2% or so on avager, but in one direction and with particular focus (higher turnout and Trump performance among rural and uneducated white voters).
Hillary Clinton is still on track to win the national popular vote, but Trump had hidden strengths in the electoral College (OH, WI, MI).
Comey. Comey messed up the early voting. This is not good.
“>99%”
… Hubris
I feel like I’ve been had. This is nothing short of terrifying.
Must be nice to have tenure. If I screwed up that bad in a real job in the real world there would actually be consequences.
Perversely, it would be the best thing for me because then I can abandon the hobby!
Please be fair to Sam, he has publically put his reputation on the line and provided an interesting website for you for free. Right now it looks like he will have egg on his face so he’s paying the price. He’s not the only one that got it wrong (if Don wins), all the famous analysts made the same mistake. Not me though- I did my own analysis and bet bigly on Trump, I win £££ if it comes through. But you should respect Sam more than ppl like me that only come out to claim they were right all along after the fact.
Seriously what are the chances of this election actually being rigged? If these results hold that is a huge error in polling. How can that happen?
Academics….I am one as well. Everyone is so aligned with the left in academia they can’t see what is happening with working class population. Wang was incredibly wrong and should suffer reputationwise.
Silver, again, was the most right.
Sam Wang’s tenure is in neuroscience. Election work is extracurricular. So tenure is not involved.
What about polling as a science? It appears that the ground shifts about as quickly as it can be surveyed.
Now I’m going to bed. I’ll wake before dawn to see how it all came out.
“then I can abandon the hobby!”
Perhaps you should, if said hobby is based on the premise that the mathematical analysis of polls alone suffices for prediction. Maybe some historical and sociological analysis is necessary.
How can this be? His own campaign didn’t think he was going to win with the polling they had. Members of the GOP didn’t think he would win. I feel sick and scared. Somebody hold me.
Did realize this was just a hobby. The whole “Princeton Electoral Consortium” title and Princeton.edu address made it seem like a genuine academic endevour. Maybe retitle it “Sam Wang guesses at stuff” with a .com address so people know not to give any credence to it.
so OMG, immediately you use the logic that people have been ridiculing Trump for all month. If she loses it must have been chicanery.
The prediction markets also will take a big reputational hit. It was .9 for Clinton on Predictwise. David Rothschild hasn’t commented since that!
Brings to mind Millerism, and the Great Disappointment.
https://en.m.wikipedia.org/wiki/Millerism
Why are polls off? Fear of women having power has powered Trump from the beginning.
nyt 77% T. 538 66% C.
Any insight into why such a large margin between the two? Is it different interpretation of exit polls?
Which demographics underperformed and overperformed vs the models most pollsters were using?
First of all, none of this changes the quality of Sam’s work, and “hubris” has nothing to do with it.
If the polls are systematically off nationwide by 4-6% I have to wonder if the election was actually hacked.
totally
This absolutely changes the quality of Sam’s work. Based on polling, there can be mistakes which will give bad predictions. But there’s no excuse for 99% certainty that is wrong or even a tiny win.
My ‘hubris’ comment was not applied to the broader body of the work here at PEC or Sam’s work.
It was specifically aimed at allowing the “>99%” figure to ever find its way into the top-line.
Just that …
How do you “hack” a decentralized voting system that is essentially off the grid?
No, this is about the silent Trump vote, the underestimated impact of Comey’s letters and lots of poll genies who lean left anyway.
I’m furious with Comey. His actions were instrumental in costing Hillary the election.
That’s bs. It doesn’t cast doubt on his whole life’s work or anything, but if the polls are all off by that kind of margin the there is a systematic error in the methods and the pollsters are responsible for that.
agreed. Trump was so sure he would win.
What about a systematic error in polling model and execution? All the polling models build samples based upon expected voter profiles. If they miss the model, and do so systematically, then the poll is off. That’s why we don’t determine elections based on polling. We do it at the ballot box. Oh, wait, we don’t use ballot boxes any more. No need to stuff. So maybe you’re right. It has never been easier to hack democracy. What are we thinking about as a nation?
I may have to pay a visit, Doctor Wang. We might need to discuss some things.
You can’t be cavalier about eating bugs any more. If Trump really does win, we’re going to have to ration those out as currency and/or food.
Bad news, Sam. The median Trump EV count just went to 244.
not just bad news for predictors. bad news for our country and for the world.
As a Sandersnista, it feels like 2004 to me, when lefty Dems were told that we had to settle for a centrist candidate because ours was ‘unelectable’, only to lose the election. Maybe the superdelegate thing gets fixed before 2020 (Warren?).
On the bright side, my wife and I are already expats, so we don’t have to spend the night drunk wondering where we will move to.
Sam, as badly as I feel, you must feel 1000x worse. Hang in there, guy.
I’m still hoping Bernie will run again!
Bernie would have lost even worse.
Superdelegates did not decide the primary.
Lucky you, GM. I’m looking to become an expat, too. Where’s a good place to move to to be an expat?
Sanders was sold down the river by the “Guardians” of the Democratic process. Between his story and wiki-leaks revelations on the collusion between Clinton’s team and the leadership of the Democrat Party, it looks like the “fix was in”.
Ruth, it’s a tough call. The entirety of the Western world is dealing with the resurgence of racialized nationalism under the guise of populism – look at Britain w/ Brexit/UKIP. I don’t think anywhere is safe.
I told you bro! I told you about the polls! It keeps happening!
If Trump gets 300 electoral votes, does the bug eat you?
Trump has won Ohio. Seems like Trump will take Michigan as well. The Blue Wall has fallen.
At 538, Clinton’s chance of winning has dropped to 44%, and Trump is now at 55%.
I think… it’s over. It’s the beginning of the end, at least for all the so-called polling and statistical forecasting. All of them were wrong.
The Comey letter had a profound effect. It was too late in the process to be picked up by the polls.
I really don’t care about Sam having to eat a bug now…we may all be eating s**t for 4 years…
Fours years? Don’t forget about the Supreme Court, the economy, our relations with innumerable countries just to mention a few things.
Very tasty with your favorite beer or tequilla: http://www.exploringoaxaca.com/gastronomy,oaxaca-city,chapulines/
Hope in doesn’t come to it.
Ken L.,
Looking more and more like Sam will have to eat those bugs unfortunately. NY Times is now giving Trump > 95% chance of winning. Only a miracle will reverse this into Hillary’s favor. That’s likely not gonna happen. Stock Market is tanking. I fully expect a recession worse than we had in 2008. We’re basically screwed. I’m wondering if the election has been hacked. This is just a disaster and absolutely terrifying. We’re in for at least 4 years of a dictatorship. Hard to believe that with the misogyny, bigotry, and racism that there were so many hidden Trump voters. Looks like any checks and balances will be the checks that are the hard-earned money of the middle class will go into the bank balances of Trump and the 1%. :-(
for the conspiracy, during Comey letter 24 millions votes cast, if I remembered AP report
Does anyone understand how NYT’s electoral vote predictions work? Because even if Trump does win Florida and North Carolina and New Hampshire and Iowa and Nevada it still would not add up to their totals…
Sam would have to eat a bug though
I guess if we lose Michigan it’s over, put your money on war stocks. We are going back to war. This time it’s not going to be good.
Even Nate Silver isn’t looking good with this. She is getting smacked in so many states that were supposed to be close.
Everything is down to PA, MI and WI, and I dont think she wins any of them
I think no one anticipated the enthusiasm of the rural folks, and polls didn’t pick pick it up. That’s not Sam’s fault.
Sam,
It’s not cooking bugs that you have to worry about. It’s your career as a pollster. 99%? Ha. You totally missed the boat here and led us all down a path to . . .
He’s not a pollster.
when the results are simply so far away from ALL the polls, isn’t that when to suspect a rigged election?
That’s what I was thinking. How could polling be this far off? There has to be an explanation.
99% lol – time for a new hobby
Given what we’re seeing, Sam was obviously wrong to so cavalierly dismiss Trump’s chances of winning. However, given the empirical evidence at his disposal at the time he made his prediction, there’s really no possible way Sam could have predicted or even fathomed a 5 point polling error. It’s the biggest polling error in a presidential election year in history. It’s simply the nature of statistics that really unlikely events are hard to predict with any precision.
Sam gave his prediction based on data. It was a prediction and he is looking at models based on polls. It is clear that the polls were systematically off.
Correct. This is a polling miss. You can only aggregate the data available. Even Republican pollsters didn’t see this one coming.
At the moment I feel like choking down a handful.
https://en.wikipedia.org/wiki/Pogonomyrmex_maricopa
nty has given Trump is near certain win.
Maybe you were never wrong, Sam. Maybe Trump just won with a 5% chance. Remember how you said in interviews that Trump might win if weathers storms suppressed the vote? I know you were being sarcastic BUT…maybe this is similar…a different KIND of storm. Facism. Facism is a perfect storm of a lot of things wrong in a society… a cancerous or viral form of government. No one could stop that, not if the country just lost its mind. Just think about it. After the Trump tape it should’ve been impossible for him to win. But that was one of many horrible things that should’ve disqualified him. And yet here he is…winning. Fascism in Italy and Germany was the fault of the people, and let’s be clear on that. Tonite is a choice, born of madness. And a perfect storm of ego and insanity could only make that possible. Thank you for your fine work. You did NOTHING wrong. Cheers
And the Supreme Court slaughtering Voters Rights, there was massive suppression, Trump’s team didn’t even try to hide it. They flat out requested voter registrant info based on ‘race,’ then closed polling sites in those specific areas. NC was particularly obvious. There was a shooting in LA, there were people walking around armed intimidating people, Muslims being harrassed, Trump’s team trying to get voter ballots & poll worker info in Nevada via law suit. The insanity is unreal. Thankfully in that last case the judge kept her head, because the last thing any of those voters & volunteers need is for Trump’s ilk to get their info. The U.S. has gone insane.
This is why it is foolish to make such a confident prediction >99% based on such a small sample size. Not enough data to know how likely events at the tails of distributions really are
don’t think so. A model is a model. it has worked for 3 elections. If there is a new undercurrent many people overlooked, and the polls did not capture, it did not capture.
Sam has a aggregation model. In the end it is using data spewed out by various states poll. I think it has a strong underpinning. A model is not helpful if the input data is off. It seems the polls were off.
But the win probability is distinct from the aggregation model. James is absolutely right. Predicting future drift based on MM history alone was never the right thing to do. It essentially ignores the chance of systematic polling error, and we’ve seen such errors crop up regularly in other elections worldwide, so this chance could not be relegated to the tail of the distribution, even if it’s a t distribution.