Princeton Election Consortium

A first draft of electoral history. Since 2004

Factchecking Nate Silver

October 5th, 2014, 4:39pm by Sam Wang


In response to the post below, Nate Silver wrote a longer piece for Political Wire.

I am traveling, and will reply soon. You’ll want to read it. Check back soon!

Tags: 2014 Election · Meta-analysis · Senate

9 Comments so far ↓

  • Amitabh Lath

    During the presidential elections you had a calculation that showed the results if the polling had an X% bias in either direction (X = 1 or 2, I don’t recall). Could this be resurrected?

  • A New Jersey Farmer

    In the competitive states, the polls still have razor-thin outcomes and most candidates are polling under 50%. Silver seems to be sure about his model, or at least dismissive of PEC. I’m not sure how he gets away with that.

  • Always Randy

    My read on Nate Silver is that ESPN expects a lot from him. And because of all the money they have spent, he has to deliver. But not deliver accuracy in his predictions but rather deliver a strong advertiser base. ESPN by its very nature is more male and more Republican and old. If Nate knew 3 months ago that the Dems would win the 2014′s –and said so–he would have lost a huge advertising base because they don’t want to hear bad news. Witness the hangers on to Karl Rove on the evening of the 2012 election. I think Sam Wang’s obligations don’t hold him to a skewed result because of money.

  • Davey

    Dr. Wang,

    Perhaps once the election is over and you’ve returned to teaching tomorrow’s neuroscientists and Mr. Silver has returned to finding America’s best burrito, it would be interesting to hear your thoughts on how we can use the 2014 results to assess the better model.

    In other words, if we hit Election Day and you determine a 60% chance Dems retain the Senate while other sites say they have a 45% chance, does Dems winning or losing tell us anything about the model? Clearly, if one site was predicting a 90% chance of a win and another was saying 10%, election outcomes would give credence to one model over another. But although Mr. Silver has has A LOT to say about your figures, I’m not really clear that if one weatherman predicted a 55% chance of rain and another a 65% chance we would laud or condemn either regardless of whether it rains tomorrow or not.

    No…I’m not asking for more analysis of PEC vs 538 specifically, but it would be interesting to hear thoughts about what makes a model particularly good or bad when they’re all within a very close range.

    In the meantime, thanks for your work!

    • Sam Wang

      I am in agreement with your intuition, and that it’s an interesting question.

      From a mathematical point of view, the topline outcome (i.e. who’s in charge of the Senate) is unlikely to distinguish between NYT, 538, WaPo, or PEC calculations. Other options would include comparisons of individual races including exact medians, Brier-index analysis (which is kinda like what you’re driving at), and predictions made months out. Lots to say on this topic.

  • Kevin

    Present estimated chance of Republican control: PEC 48%; 538 58%. In fact, the difference between the models is narrower, because this doesn’t account for the fact that Silver scores 25% of the instances in which Orman decides in the R column.

    This is starting to remind me of Hillary versus Barack: the more narrow the difference, the more intense the disagreement!

  • Alan Koczela

    Dr. Wang,

    Better get a suit and call the folks. My psychic powers tell me that Nat and you will be on TV soon. This “blood feud” is prime for the Sunday snooze-shows. Twitters afire! (On fire, again!?! You’d think they would’ve installed sprinklers or lined it with asbestos by now.) Please calmly do what you’ve been doing and let the chips fall where they may. You have a wonderful model with a good track record. Just laugh and enjoy the fireworks.

  • Randall

    Two new major polls with massive samples of likely voters out today show the Democrat leading in both IOWA and COLORADO. In other news, Nate Silver complains.

    • 538 Refugee

      Uh-oh. Heading back to “outlier”. Expect more incoming and put on the flak jackets.