Princeton Election Consortium

Innovations in democracy since 2004

Oct 21: Biden 357 EV (D+5.4% from toss-up), Senate 53 D, 47 R (D+4.8%), House control D+5.0%
Moneyball states: President NV AZ PA, Senate AK MT IA, Legislatures KS TX NC

Looking ahead

November 9th, 2016, 12:53am by Sam Wang

Going into today’s election, many races appeared to be very close: 12 state-level Presidential races were within five percentage points. But the polls were off, massively. And so we face the likelihood of an electoral win by Donald Trump. At the same time, Hillary Clinton appears likely to win the popular vote. The Upshot’s model currently projects a Clinton lead of more than 1 percentage point. If that lead lasts, it means that more American voters preferred her to Trump.

At the moment, the NYT is projecting Trump leads of less than 1 percentage point in Pennsylvania and Michigan. Even without these states, Trump has at least 268 electoral votes (depending on some districts in Maine and Nebraska). We will see in the morning how these last few states and districts will be resolved.

In addition to the enormous polling error, I did not correctly estimate the size of the correlated error (also known as the systematic error) by a factor of five. As I wrote before, that five-fold difference accounted for the difference between the 99% probability here and the lower probabilities at other sites. We all estimated the Clinton win at being probable, but I was most extreme. It goes to show that even if the estimation problem is reduced to one parameter, it’s still essential to do a good job with that one parameter. Polls failed, and I amplified that failure.

Update: I believe a likely reason for polling failure is undecided Republicans coming home in the home stretch.

This election is about to create shock waves that will make the last year of campaigning look mild. We are about to see both houses of Congress under Republican control, quite possibly with a President Donald Trump. This comes in the face of a reasonably growing economy and a popular Democratic President about to exit the White House. It is difficult to reconcile these different facts.

Thinkpieces that have been written in the last few weeks have to be re-examined in a new light. Ezra Klein at Vox has written about the weakness in U.S. democracy, in which a weak Republican Party could nominate Trump, and partisan polarization gave him a shot at the Presidency. This one-two punch appears to have landed, hard. I was correct in documenting Trump’s rise in the primaries, an easier task for polling analysis because there, his lead was considerable.

I have written about the role of partisan polarization in getting voters to choose up sides, to the exclusion of even considering a vote for the other side. The chickens have now come home to roost. Exit polls showed that most voters felt that Trump lacked the temperament to be President, and that Clinton was seen as more qualified. Yet Trump seems to have rallied enough support to get overcome these factors. All Presidential nominees have had lower and lower approval ratings, and Clinton was no exception to the pattern.

Now we see where that long trend has led. One consequence is that more voters refused to support either major candidate. Neither Trump nor Clinton is headed for winning a majority of voters in Pennyslvania or Michigan. In Pennsylvania, the NYT projects that over 3% of voters cast their ballots for Gary Johnson or Jill Stein. In Michigan, the minor-party total was over 4%. In both cases, these numbers are considerably greater than the Trump-Clinton margin.

The coming years will be disruptive ones, to say the least. Whether you are Democrat, Republican, or neither, it’s going to be a challenging time ahead. It’s Donald Trump’s Republican Party, and maybe his Presidency too. The nation belongs to all of us. Good night.

Tags: 2016 Election · President

291 Comments so far ↓

  • Eric

    Are we witnessing the beginning of the end of America? Something strange has happened here. There’s no way everyone could have been this wrong. There’s more to this.

    • Tom Gavin

      Nobody reads anymore. It’s all about social media. I guess the sleeping giant of displaced majority whites by globalization and income inequality was awakened, and unfortunately a demagogue like Trump was there to reap the benefits, rather than someone like Warren or Sanders who believed in some other issues like the health of the planet.

    • Avattoir

      I’d really like to say something like, Just look at which party’s state operation was in charge of the election of the swing states that pretty much all swung to Trump. That would suggest possible shenanigans, per Eric.

      But Pennsylvania has a Dem governor, and so does Minnesota, which is on a knife’s edge. And how to explain Nevada, with a Republican governor, and Republican control over both state legislative chambers, yet it went for Clinton & has just voted in the first ever latina U.S. Senator, and a Democrat?

      This isn’t funny. The country has just voted into the White House someone a white nationalist, a xenophobe, a grifter, a thief, an economic moron (What’s going to happen when the Debt Ceiling comes up next? This incoming government is for the disaster of a national balanced budget. And the ACA is now on deathwatch, a terribly retrograde step.), and the person who will be ensuring a right wing SCOTUS for the next quarter century, and with his party controlling both Congressional chambers. He’s scheduled for trial in two weeks on a case of FRAUD, FCOL.

      Despite the anomalies like in Pennsylvania & Nevada, I think we can’t just suppose the pollsters – pretty much ALL of them, outside of cranks – got it wrong. If we don’t seriously & immediately consider election theft over a number of key swing states, it’ll of be of no constitutional value at all if such fraud is ‘concluded’ by academics in years to come.

    • Kevin King

      I think we may well be, in recognizable form.

    • George

      I think you may be correct. So many polls being wrong, though possible, doesn’t seem plausible. Unfortunately I don’t trust the mainstream media to do the necessary investigation into whether or not anything was hinkey.

      Empires fall for many reasons. With Trump in the White House and a GOP Congress that sees little need for investing in infrastructure or science, or addressing the effects of climate change, or continuing to provide health care for Americans, I think the US will be surpassed as world leaders by Europe as a whole, China, Japan, and perhaps India. The US may be on its way to being a second-rate economic power with nuclear bombs, kind of like Russia.

  • William

    Please don’t let this discourage you from doing these models! It’s wonderful work; this election involves a literally unprecedented polling error.

  • PECismyoasisofsanity

    The most damaging, far-reaching election result since the crowd picked Barrabas over Jesus.

  • Adam

    So just like 2000, we will get a dem winning the popular vote and losing the EC. Wow.

    • KT

      I’m watching CNN at 12:54 AM… Donald Trump has also won the popular vote. Which means all the national polls were also wrong.

  • Jake

    The silver lining from my POV:

    Either DJT is somehow (miraculously) a way better POTUS than any of his (many) detractors and even a lot of his supporters have predicted, or he will make GWB’s end of 2nd term ~30% approval seem downright incredible in comparison. I honestly do think he will be the worst POTUS of the modern era.

    But then, I thought HRC had a pretty easy win tonight, too, so I’ve been wrong before.

  • EricK

    I am very sad about this.

    But I’m also very curious and puzzled about the polling errors. The polls were similarly off in 2014, from what I recall as well as for the Brexit vote. What has happened in the last 4 years to make polls so much less reliable?

  • SK

    Oh well.

    A new market opens up: for the best pollsters. Aggregate polling (like here) seems to have used the data well. But the end results are only as good as the input.

    In 2014 and now in 2016, the polls have been way off.

    I did like the analysis, and think when we get better polls, we’ll see the analysis keep up

  • Tyler

    How did the polls get it so wrong? Could it be due to Trump mobilizing large groups of voters that have historically been inactive, and because of that inactivity are typically neglected by polling agencies?

    • Michael Coppola

      It will take a while to get all the data, but early indications seem to be that Trump won higher percentages of the R demographics, rather than turning out more of the R demographics.

    • Mike

      One other thought occurs as a possible explanation: widespread systemic voter suppression made possible by the gutting of the Voting Rights Act. It would be really hard to model an unknown number of medium-to-high propensity minority voters suddenly having their 15th Ammendment rights violated.

    • MarkW

      The polling being so lopsided makes me think that there might psychological/social issues at play. I’m a college-educated white male, but I cringe a little at how so many pundits have referred to the non-college-educated white vote. Sure, it is a demographic of interest. But consider this scenario: You hear all these commentators talking about the non-educated white vote along with racism and misogyny etc. when you get a call from a pollster asking for your non-college-educated white preference. Aren’t you going to pause before saying Trump, even if that is your preference? Embarrassed to admit it, maybe, but it might even act as a get out the vote motivation factor—“let’s prove their highfalutin opinions are worthless”.

      The question is: how to way to account for this or model it?

    • Matt McIrvin

      NC and Florida were close enough that a more voting-sympathetic state government could well have made the difference.

    • (((CassandraLeo)))

      I am almost certain that the gutting of the VRA swung FL and NC. I would be unsurprised if it affected WI and MI as well. Possibly even PA.

  • David Elk

    I’m just shocked. How did polls miss some of these states so badly?

    It’s going to be a long four to eight years.

    I hope PEC is still there anyway.

  • Veronica

    You lied to us, Dr. Wang. You lied to us.

    • fred flint

      No he didn’t. The polls were wrong. He just aggregated and condensed what the polls showed. The polls were all wrong for the first time in several presidential elections.

    • Neal J. King

      Veronica, don’t be ridiculous.

    • mike

      He didn’t lie about what the polls were saying (Clinton lead/win). But he absolutely blew it on his percentage of certainty at 99%. Calling it a sure thing was the “lie”.

    • Jay Sheckley

      You’re kidding, right?

    • Stephen R. Diamond

      I wouldn’t say lied, but given the state of the art, why did he presume to advise people on political activism, rather than treating his work as a purely intellectual exercise?

    • Marc

      Being wrong and lying are two different things.

    • Jeremiah

      @Stephen The statistics absolutely showed where activists should have been putting their resources. The big polling misses were in the rust belt Pennsylvania, Michigan, Ohio, Minnesota and Wisconsin. The polls always showed that Florida and North Carolina were close. The senate races that were identified were won within the margin of error, Hassan, McGinty, Kander and Cortez-Masto. Maybe Ross in North Carolina was a waste of money in retrospect and Russ Feingold got swept up in the rust-belt fever.

    • Rob

      Apparently going into last night’s reveal, even Trump’s people were down. Apparently their polls showed defeat. If true then something went totally wrong with polling in general.

  • Arnold


    It looks like some element of the polling involved very serious, systematic uncertainty. And not the sort that is mitigated by things like taking the median. I’d be interested in your thoughts about how, in principle, one might distinguish elements of a polling methodology that are vulnerable to this sort of uncertainty, as opposed to elements that are just noisy.

  • Al

    There’s an ethical question here. Why does nobody consider the feedback effects that these models might have on voter turnout? A perceived probability of >99% for Hillary may well have kept some voters at home.

    The observation of “wrong inputs lead to wrong outputs, independent of how good the model is”, is secondary to this. The probabilities were widely reported and discussed in the news. People believed in them.

    • Michael Coppola

      All due respect to Sam, I don’t think this site gets nearly enough traffic to depress turnout.

    • Ryan

      If you read this site it actually says confidence inspires people to vote. So I don’t think that would be the case.

    • Stephen

      I think that he may have contributed to the complacency of those Clinton supporters who felt they had the election wrapped up. In all fairness to Sam Wang, the general consensus was that the election part of the process was just a formality. I saw an interview with Mr Wang where he sounded so confident that Hillary would win that I was convinced not to even go and vote. I’m glad there were many other Trump supporters who didn’t buy into the hype.

    • Al

      Michael: I disagree – you only need 1 in 20 democratic voters to stay at home to reach a 2% difference (which would’ve flipped Florida). Between P’ton, 538 and New York Times (which I believe is based on these and others), I have no trouble imagining 1 in 20 people being affected.

  • Will

    Please hear me out. The accuracy of polls aggregators is the entire point of poll aggregators. If you view your poll aggregator as informing activists of where to contribute support, as opposed to mere entertainment for the masses than all the more reason to take it seriously. You picked a standard error based on mere intuition, disregarding the high number of undecided voters, and the new early voting information. 538’s presidential election model was far and away superior. Nate owned up when he was wrong in the primary..

    • Michael Coppola

      538 still had it 2:1 against Trump. How would things have been different if all of the aggregators had converged around 2:1 instead of 10:1, 20:1, or 100:1? Aggregating bad data is always going to give a bad conclusion. It’s on the pollsters to fix that, not the aggregators.

    • mike

      There’s a huge difference between 70% and 99%. Sure thing versus fairly likely but could go the other way. And the lower percentage recognizes that there’s high uncertainty.

      Nate was insisting the polls could be wrong all along, and he was mocked for it. Turns out his warnings were right on the money.

    • Michael Coppola

      My point is just that Silver still projected a Clinton win. “I was wrong, but at least I wasn’t sure that I was right” doesn’t change the fact that the issue was with the data rather than the data aggregation.

    • Lorem

      Honestly, 538’s model is still awful. It still gives unreasonably high probabilities to extremely unlikely outcomes (that still look extremely unlikely despite the actual result). It’s still overly unstable.

      However, it must be granted that it turned out to be awful in the right direction, and perhaps makes an honest effort to account for some things that should be accounted for.

  • fred flint

    If you liked George W. Bush. You are gonna love Trump….yikes.

    • TZX4

      We will likely long for the relative safe sanity of Bush Jr

    • Lorem

      I thought G.W. Bush did relatively okay, especially in his second term. I am utterly terrified of Trump.

  • George H

    By definition, a silent voter is one who is silent. They are not in the target of traditional posters, which I think is the fatal flaw of polling. Trump was able to wake them up, thus create a huge error. Had seen this in Brexit. I did not believe it could happen in America. Most serious people did not believe.

    Another system flaw.

    • Bobby P

      Brexit polls were accurate, though. The punditry all said it was impossible, but the polls showed the races were neck and neck. That’s not what happened here. The polls were all off by at least 4 points in most of the swing states. That’s an enormous, systematic, mistake.

  • Marc

    What’s scary about all this – is he (Trump) was actually right. Brexit 2.0, hidden vote, rustbelt strategy, Florida.

    The opinion polling industry will be laughed out of existence.

    So the big question – what is Obama’s legacy? Two words: Donald Trump.

    Good luck…we’re going to need it.

    • Michael Coppola

      No, what’s scary is that he’s going to have the FBI, CIA, and NSA at his disposal.

    • Nancy

      Not Obama’s legacy. Donald was created and installed by the Repubs and the scared soon to be minority. It won’t turn out well.

  • Evan M.

    What do we tell our children?

    • Jim Balter

      Few people grasp the enormity of this. Someone mentioned GWB … this is far far worse. Trump will have the radically right wing House and Senate and will make the SCOTUS the same … for the remaining years of the viability of humanity. Global warming will be unstoppable, racists, bigots, and misogynists will rule the roost, Russia will have a free reign, NATO will fall apart and probably the UN too, the military will run wild … there’s no end in sight to the disasters. At this point, any analysis of polling etc. is pointless … we have far graver concerns.

    • Jeffrey Davis

      What do we tell them? Paper ballots.

    • Marc

      Tell them to vote in every election when they can.

  • Stan

    99% probability of Clinton victory? Time to get a new job, Wang, far away from statistical modeling. Never before has such incompetence been given such a broad platform.

    • Veronica

      It’s just like I said: I’ll never trust this website–or any other polling site–again.

    • Bridget Blasius

      That type of comment doesn’t help, Stan. The whole country is in disarray and shock, right now. Can we send each other some support and kindness rather than blaming a statistician for just looking at the data? Clinton was consistently ahead in most major swing states. This was an unprecedented upset. Nobody could have predicted it.

      I, for one, am never trusting any polls again.

    • The Vindicator

      I quit trusting polls and a lot of other things a long time ago.

    • Vicki Vance

      I, for one, am never trusting any election again.

    • Rick Jones

      Silver was correct, Sam wasn’t. That happens.

      It’s science. Sometimes you get it right and sometimes you don’t. It doesn’t make Sam stupid, it’s just illustrative of what happens when you don’t have many examples to base your model on.

    • Mary B.

      That’s really uncalled for and unacceptably mean spirited.

  • Marvin8

    I obtained coverage for my pre-existing medical condition because of Obamacare. This is just awesome!

  • Mike

    Trump out-performed his polls in Ohio, Michigan, and Wisconsin by 8-9 points. In Illinois he underperformed by 2 (if I’m reading Sam’s data correctly). Some of these misses are getting into the realm of no longer credible.

    Maybe Trump was right and the election was rigged – but for him, not against.

    • Nancy

      I think what’s most interesting is that Donald kept saying he was doing well in the exact states he needed to win even when polls showed otherwise. Not bragging just stating I’m told I’m doing well there.

      The perfect way to “rig” would be just enough not overboard. Finally a narcissistic sociopath uses projection as a tool. His constant whining about rigging set the dems up to deny it can happen, and prevents them from suggesting it now. Stranger things have happened. If nothing else it’s a novel plot.

      History repeats itself because people tell themselves it’s different this time. We’re a sad species.

  • Walter Manny

    I wondered this morning in a post here what to make of the primary polling errors for HRC in key states. Not wondering any more. Got to get that into future models.

  • Bruce O'Dell

    C’mon folks. As a cybersecurity professional, I have to ask if do you not know when you’re being played? Surely it’s more credible to believe there was a mass systematic failure of polling and all the other reassuring indicators of business as usual elections, rather than contemplate the reality that our ludicrously insecure voting systems – traditionally only manipulated by domestic actors – might this time have been breached by outsiders. Google “Votescam”, “Black Box Voting” and “Avi Rubin”. Welcome, consensus reality commentators, to the America that is powned by Russia.

    • WildIrish

      Exactly what I have been thinking. The variance between the polling and the votes is just too large to believe it could have happened without some kind of “help.” I would like to see a comparison between exit polling and votes.

    • Anthony Shanks

      I want to believe this but I don’t want to be a hypocrite here because I would of called bullshit if Trump lost and claimed the election was rigged. Before I even entertain this possibility I would love to see some credible evidence of vote tampering.

    • Bruce O'Dell

      @Anthomy Shanks- you’re asking for “credible evidence of vote tampering”? Carnegie Mellon cybersecurity professionals executed a mobile code exploit against touch screen voting systems in 2007. Hart Intercivic tabulators have internet connectivity on by default. One brass key (schematics available online) opens all Diebold voting machines in the US. Hari Hursti demonstratef a mobile code exploit against op scan voting machines in 2005. As someone who has designed electronic banking systems: if we designed our voting systems the same way, there would be blood in the streets. You’ve been powned.

    • Ryan

      You can’t wide scale rig an election. We have been over this repeatedly for the past 2 months. It can’t happen. It won’t happen. Don’t give into wild conspiracy theories. It just makes us look crazy.

    • Mike

      Exit polls can provide that. If, say, an exit poll of Illinois is +/-2 but one of Michigan is off by 8, that is evidence that votes in Michigan were tampered with. It says nothing about who did it or how, just that it happened, but that’s still important. And if the Michigan poll is also +/-2, then that’s evidence that they were not.

      This, by the way, is why every democracy needs good exit polling. Whether it’s a country emerging from a dictatorship having its first election, or the oldest democracy in the world, there are still incentives for fraud. Good exit polling can help expose fraud, and/or help the voters have faith in the process.

    • pechmerle

      No need to get all conspiratorial, and leave some plain numbers behind. The biggest immediate difference from 2012 is that the popular vote is down by about 10,000,000 for each major party. Two unliked candidates did not bring out as many people as Obama and Romney did in 2012. This gave room for a much closer result, which is what we have.
      The question of why the great majority of pollsters got their likely voter screens wrong and the result wrong remains to be studied and answered. But their is no need to blame everything on the Russian, or the Illuminati, or whomever. The American people have done this, and they will live — however unpleasantly from my point of view — with the results.
      A small probability that the underdog wins doesn’t mean “never.” Sometimes at the track the 44-1 shot comes home — once in a great while it has for me.

    • Josh Soffer

      David Frum discussed this very thing on Bill Maher the other night. As far as he was concerned, the biggest danger facing the country was not a Trump win but a breakdown in trust, by both sides, of institutions. The potential social chaos that could ensue from losing faith in the glue that holds the system together might be far worse than living under a Trump administration. I might add that a number of analysts have argue that Putin wanted to destabilize the American political environment rather than put Trump in power.

    • fred flint

      Oh come one. Most states asked for the gov’ts help securing there systems. Lots of security experts looked at them. They are all different types of systems run by different types of people and most are not connected to any network. So I seriously doubt there was any hack or conspiracy.

    • Dan Heskett

      The reason we have 50 independent states, with 50 independent elections, is not an accident. The number and variety of voting systems and methods precludes an organized, effective, wide-scale successful attempt to hack the election en mass.

      The polls were wrong, it’s not the end of the world. Lots of ink will be spilled dealing with why and how, but there’s no need to imagine a vast conspiracy when your team loses. That’s the stuff of reality TV stars and nut-jobs.

    • Matthew Waters

      I’m very saddened by the loss, but this is exactly the wrong way to respond. Don’t use unsubstantiated accusations to try to even the score when lack of faith in facts and logic are the reason for Trump’s rise.

      There are several checks, by both parties, of each step. The voting machines are not online, but the vote totals are collated manually. Then they are sent to the central elections office, usually with a police escort. There are monitors by both parties at each step.

      Again, the voting machines are not online. If there was a vulnerability found, it would need to be applied manually. And not all states even use voting machines.

  • Steven J. Wangsness

    We survived the Civil War. We’ll survive this.

  • JSR

    Why is no one mentioning the elephant in the room – that the surprise loser is a woman? People weren’t voting *for* him but *against* her. Misogyny is alive and well and this country is not ready for a woman president. That’s what the polls were hiding.

  • Jeff

    Sam, I have been following you everyday for the past year. Every day. Multiple times a day. I trusted you. I told my friends. You badly screwed this up. Whatever excuse for why, its not enough. My trust is broken. Statistics do lie. I am heartbroken.

  • Pat L

    Thanks Dr Wang for all you did and do. As the President said, “The sun will rise in the morning.” That’s probably 99%.

  • Joel Sohn

    Awhile ago – at the beginning of the primaries I believe – I wrote one comment. “Be prepared for surprises”. I read the wordage almost every day and more frequently as the election drew closer. The American people understood the wonder of the secret ballot.

    What went wrong? At least one of the fallacies of the polling has to do with with the time-dependence (if that is the right word) regression analysis – I think its a thing that a biophysicist might encounter when understanding the mechanics of the folding of proteins or maybe the changing micro biome of the zebrafish or humans over-time. It is in this mathematical sphere that this particular fallacy of the regression lay. I may not be clear here. But I think that it is the same problem. Like an assumption that gets made in a time-dependent principal component analysis. If the assumption were true one would be able to make impossible conclusions – like believing they can predict the future I guess. I am not sure that I’ve made myself clear at all.


  • Harold Bridges

    So, it turns out polling and poll aggregation are not engineering disciplines after all. Back to a world with substantial unknown unknowns.

  • Ruth Rothschild

    Sorry this didn’t turn out as you had predicted, Sam. Thank you for all that you did in hosting this site. Sorry you’ll have to eat some bugs. But, we really won’t hold you to that if you don’t care to eat any. I wonder who Trump paid off to throw the election his way or who he got to suppress and intimidate minority voters. It’s hard to believe that polls and aggregation, etc, could be that far off without there having been some underlying hanky-panky going on. We’re in for a long 4 – 8 years, and I suspect that before that time, our economy will be bankrupted and the middle class American economically raped and pillage within 2 years of his taking office. A very sad and terrifying day in American history.

    But please don’t let this discourage you from continuing to do your stuff at this site. I think that it wasn’t you or the polls, but rather some underhanded, corrupt stuff that we’re not privvy to that Trump and his thugs engineered. Your analyses, etc, made perfect sense.

  • John Gilbert

    What would a Russian hack look like? If Russia did hack a few key states, would we be able to tell?

    The results in some states, such as Nevada, were almost spot on to election morning polls. Were the results in key states (Florida, Pennsylvania, Michigan) about right, or were they significantly different than where the polls had them?

  • J-D

    I mentioned in an earlier comment that after the 2015 UK general election, the British Polling Council and the Market Research Society commissioned an independent inquiry into what the pollsters there might have done wrong.

    Two points arising out of that relate to comments people have been making here.

    1. Some comments here suggest that it’s extremely unlikely that the polls could have been so wrong, and that this suggests that the results today must have been produced by fraud. Obviously I can’t prove that there was no election fraud, but it is worth noticing that the pollsters were just as far off the mark in the UK in 2015 as they were in the US in 2016, and that was in a system which is run in a markedly different way from the US one. So it is possible for pollsters to be way off the mark. It probably doesn’t mean fraud.

    2. One of the specific possibilities that the UK independent inquiry looked into was what in the UK they call the ‘shy Tory’ effect: that there’s a substantial group of people who are ashamed or embarrassed about admitting to a pollster how they’ve decided to vote, and that this systematically distorts polling results. The UK inquiry found that the evidence did not support this conclusion. It is still possible that something happened in the US that did not happen in the UK, with people unwilling to admit to pollsters that they were going to vote for Trump, but it’s only one possibility, not a conclusion to jump to at this point.

  • Neal J. King

    It’s very difficult to wrap my head around the idea that the polls were that bad. In addition to the polls leaning Clinton’s way, other factors seemed to be playing a pro-Clinton role:
    – Trump’s threats against the Mexican immigrants, leading to an increase in the Latino participation, that was expected to be anti-Trump;
    – Increase in early voting, that was expected to be pro-Democrat;
    – The gender gap for Trump, that should have tilted the female vote against him.

    There are all these plausible narratives that went only one way. How do all these trends interact together to produce an opposite result?

    Can anyone build a model of the electorate that can explain what happened? What plausible differences could there be between the electorate of 2016 and that of 2012, that could explain the different results? I am wondering about a demographic model.

    (I am also curious about that factor of 5 in the “correlated error”. What’s that about?)

    • Matthew Waters

      One thing to remember is the national polls were not too far off. Clinton won the national vote by one percentage point. The nationwide vote only moved a couple of points from 2012.

      The statewide polls are where the big misses were. 538 posted that Clinton outperformed polls in richer, whiter and more latino places and then drastically underperformed in with poorer whites.

      Poll weight the respondents so that the response pool has a similar make-up as the general population. So if 5% of respondents are latino, the response will be weighted to 15% or whatever is the population of Latinos. But the 5% who respond may not be representative of all Latinos. If the poll only asks questions in English, then English-speaking Latinos may be more conservative.

      For some reason, there is a similar discrepancy among whites. The whites who voted for Clinton were overweighted, especially in state polls. Trump voters may have fundamentally not picked up the phone as much. Or there is a shy Trump voter.

      The other issue are the screens that were used to determine “likely voter.” If it rains, do some voters become less likely? It’s also imprecise.

      Ultimately there’s no way to give an absolutely precise prediction. The imprecision in election polls was offset by repeating election polling and finding tweaks to the polls to better predict outcome. But the last outcome was Obama vs. Romney. Either the voter pool was very different for Trump, or cell phones have created a response bias.

    • Ed Wittens Cat

      Complex Adaptive Systems Dynamics

    • KT

      Americans who want mass deportation outnumber Latino citizens who are still minorities in this country. I am still watching cable news. And, as I suspected, CNN reported that exit polls (yeah, I know, polls) showed that the number one issue that Trump voters overwhelmingly want from him is the mass deportation of illegal immigrants. This is one campaign promise Donald Trump will have to fulfill.

      Coincidentally, Sheriff Joe Arpaio has lost his re-election. I have a strong feeling that Donald Trump will use this opportunity to appoint Arpaio to be the head of ICE (Immgration and Custom Enforcement) or even Homeland Security. When Arpaio becomes the head of ICE or Homeland Security, illegal immigrants will leave and “self-deport” out of fear – thus fulfilling one of Trump’s campaign promises.

      Donald Trump promised that he will DREAM and DACA. Here is the irony: the list of hundreds of thousands or millions of Dreamers and DACA applicants, which have been collected by the Obama adminstration, will end up being the very instructment for finding and deporting those illegal immigrants. The applicant lists will most certainly be passed on to the new head of ICE/Homeland Security, Joe Arpaio, who will be merciless and relentless in his pursuit and deportation of illegal immigrants, given of what we already know about him.

      Donald Trump also promised he will repel Obamacare. However, I want to be optimistic – and hope that he will repel Obamacare and and replace it with a single-payer system. Maybe the Congress will be more receptive to a single-payer health care proposed by a popular Republican president.

    • KT

      Another benefit I can see coming out of the Trump presidency is the demise of the Trans-Pacific Partnership. Hillary’s support of the TPP is the a major gripe I had against Hillary. I was certain that she would pass the TPP as one of her first act as Madam President, regardless of how she had flipped and flopped on it. Now the TPP is more likely to be permanently dead.

    • KT

      On to the next benefit: maybe we can finally untangle with Saudi Arabia and Turkey, who are our worst enemy instead of real allies, undermining us in our back. Let’s be smart here: the whole mess in the Middle East is really about Sunni’s attempt to wipe out Shia and take over Shia lands. The Sunni alliance (Saudi Arabia and Turkey) have been systematically taking out Shia governments and taking over the Middle East. And, America (i.e., we) have been allowing Saudis and Turkey free rein or even doing their dirty work for them – because they are supposedly our “allies”.

      Let’s be real here: the only reason we are arming and funding Islamists in Syria to overthrow the Assad government is because Saudis and Turkey want to install a Sunni theocracy. And Hillary was gonna to let that happen – and even help it happen. (And with the money she had pocketed from Saudis, she would have to do that favor for them.) Hillary has too much ties with the Saudis. We can never become untangled with the Saudis under her adminstration.

      This is where I agree with Donald Trump: we need to work with Russia and cut a deal with Assad. I do not want another Sunni/Wahabbi government in Middle East. Because Islam is not the problem; Wahabbism is. So, that is another good thing that can possibly come out of the Trump adminstration.

  • KT

    I am not going to mince words here. Last night, Nate Silver and Sam Wang lost whatever credibility they had earned over a decade. I do not see how they will ever be able to recover from this historical blunder for the rest of their lifetime. Even if they are correct on their every prediction for the rest of their lives, people will always remember how wrong they were with this election.

  • Kevin B.

    Don’t beat yourself up. You are not the author of this situation.

  • Ed Wittens Cat

    not all of the polling– the LV screens were simply wrong
    this would have happened with or without Comey
    I bet Trump got a higher percent of white voters than Romney.
    There is a little bit of (Eichman) Nate Silver in all of us– it was unimaginable for us college-educated elites that Trump could win. We never even talked about a Trump wave.
    So be antifragile.
    There is opportunity in disaster.
    What happens when Trump cant bring back the jobs he promised?
    Get ready, get woke.
    Like Dr Wang said its gunna be a bumpy ride.

  • Stephen R. Diamond

    Trump won the popular vote too. Yet even in this last comment, Sam wants to keep his contrary prediction. Hold as much as he can salvage. It’s like he still hasn’t really learned the lesson. Or rather, he thinks the error was some (unexplained) bad statistical assumption rather than a failure to understand the reality.

    I thank Sam for curing me of a confusion of “precise” methods with real science.

    • Ed Wittens Cat

      wait for the debrief before throwing stones
      the Math is good– the LV screens were bad.
      this was a Trump wave– something we dilettante poll-aggregator intellectuals never even imagined.
      a white wave!
      TrumpTeam dint imagine it either.
      so was this our bias reasoning? Like Dr. Wangs post?
      motivated reasoning

    • smartone

      California is still coming in when the dust settles Trump will lose the popular vote

    • Ash

      A telling stats from one of the exit polls says around 10% of voter who approved of Obama voted for Trump. That’s around 5.5% of the popular vote. I would like to blame this disparity on Comey’s October surprise.

    • Kevin King

      No, right this second, Clinton won the popular vote by 140,000 votes.

    • Commentor

      “…he thinks the error was some (unexplained) bad statistical assumption rather than a failure to understand the reality.”

      What does this even mean?

  • Danny

    Long time reader, first time poster.

    Thank you for sharing your time and knowledge with us here Dr. Wang. Please try not to take some of the more personal attacks to heart. I hope that you will continue your work in the future.

  • Karl

    I spent months repeating Sam’s predictions of a clear Hillary victory. I believed him because his calculations were explicit, and hence open to criticism.

    The assumptions were wrong, so the conclusions were wrong. I guess it is like they say. You can work with what you know, but you can’t work with what you don’t know.

  • Overanalyzer

    Is it possible that the polls were right but practically all the undecided went to Trump?

  • Arun


    Just a word of encouragement. I personally still think your methods are sound. You were undone by inaccurate polling. A model is only as good as the data that goes into it. Thanks for the application of the scientific method.

  • Robert Moss

    Tough break Sam, but keep your head up. Thanks for all the work you’ve done over the years! Also, I bought your books

  • Seth Gordon

    My sincerest condolences, and also my congratulations on your hard work.

    An experiment with a negative result is just as valuable to science as an experiment with a positive result, no matter how much the experimenter had wished for a positive result.

    • Ken L

      Absolutely, let’s learn as much as we can about why modern polling was unable to accurately represent the electorate and attempt to understand the root causes of this unexpected result.

    • nhm

      Sam didn’t perform any experiment. He made a model. There’s a difference, and he would agree.

    • Seth Gordon


      The Republican vote from approximately 1972 to 2012 has been based on a coalition of three interest groups: “small government” business-oriented folks, socially conservative Christians, and white populists (the old “Dixiecrats” being white populists par excellence).

      Trump upended this coalition by taking a message that focused on a white populist audience and cranking it up to eleven. His rhetoric horrified the business community (which stands to lose more from Trump’s trade policy than it will gain from his tax policy), but he didn’t care: between his own personal wealth and the free coverage he got on TV, he didn’t need their money. And the social conservatives, I guess, believe so much of the over-the-top rhetoric about Hillary Clinton that Trump’s own un-Christian behavior seemed like a secondary concern.

      In so doing, Trump expanded parts of the Republican electorate, sometimes at the expense of the Democratic electorate, sometimes at the expense of people who were previously not voting at all.

      But the models used within public-opinion polls, to translate the raw data of “this is what we hear from the people who answered our question” to “this is an imitation of a simple random sample from the people who will vote”, were all tuned to the Republican and Democratic electorate from 1972–2012. So (almost?) all of them misinterpreted the data, to an extent that wasn’t clear until the 2016 election returns came in.

  • Silver Defender

    Professor Wang you have many times criticized 538 and Nate Silver. He appears to have been correct or at least closer to correct. One of the criticisms you have often leveled against him and others is that his operation was motivated to try to get clicks/views/advertisers and therefore showed the race closer than it actually was. I think it’s only fair that you examine your own biases now. Unlike Silver, your career has far less to lose when wrong. Therefore you were much less risk adverse than Silver, who did a better job of hedging. It appears that overconfidence and that lack of hedging caused you to be brilliantly wrong this time.

    I would also urge you to read Mr. Silver’s piece from a few days ago: which seems very prescient in hindsight, particularly the paragraph regarding polling error.

  • Mike in DC

    Nate Silver was criticized for saying that this election is unprecedented, therefore the polling is less reliable. He detailed his reasoning on his blog.

    Many of 538’s caveats about the illusory nature of Clinton’s “strength” turned out to have occurred.

    I don’t like this result, but it’s not true to say that no one could have predicted it. It may have been a “lower-probability” event, but it was certainly possible–and for reasons that were foreseen.

  • alum

    Here’s a new problem for you, Sam: to what extent did recommendations to redirect donations towards congressional races affect the top of the ticket?

    While obviously a bad look for your own brand, Sam, I think this has been such an extreme mistake that you’ve likely actually hurt the reputation of the university.

  • Some Body

    I think several commenters on this site owe a big apology to Nate Silver and his colleagues. People said he’s click-baiting, padding his model with gratuitous uncertainty. Well, guess what? The uncertainty was there for a reason. All poll aggregators had to work with s****ty polls. 538 were the only ones who had a good assessment of the quality of the data, and treated a polling miss of the magnitude we got (going in any direction) as a likely event. They could also explain exactly what explained the unusually high levels of uncertainty in their model. They even correctly predicted the number of States their model is going to “miss” (they said 6; it was 5). That’s real value added in an aggregator, and impressive work overall.

  • SickAndTired

    I’m not going to throw stones but I’m quitting this site and cutting my reading of news and politics down to 15 minutes a day. I have no trust in media, polls or anything else. I also have decided that I’m not going to care – on a national level – about the marginalized groups that have been my obsession for so long. I’m not reading anything more about their situations, not even about women. I’m over 60 years old with health issues and I’m not going to let caring about this country and its people spoil the few remaining years of my life. I have never been so utterly demoralized in my life. So bye, bye. I won’t be back.

  • Psych

    Thank you for addressing what parts of the model were shaky.

    There’s reliability and validity for a measure when collecting data. In this case the polls seemed reliable. There was agreement. BUT, how valid was the result?

    An analysis depends on the collected data and the missing data. With these election polls there were the “undecided” & those who’d vote for a 3rd party. What were the pollsters doing with those variables? And, what variable(s) were inaccurately weighted?

    • Former Big Data Flunky

      I write this as a molecular biologist who was imbedded in a bioinformatics initiative for several years. I think Psych is basically correct. The analytics can be superb but if the experiment, the assay, in this case polls, are flawed, it is a failed enterprise.

      I saw this in my biological collaborations on gene expression and in considering the 2014 mid-term election, Brexit and now the 2016 election, I have to consider that polling is not a functional detector for the data needed to run the modeling.

      It’s time to go back and reconsider how this experiment is designed. If one can’t assay public sentiment, how can one proceed?