Princeton Election Consortium

A first draft of electoral history. Since 2004

Everything is different today…or maybe not!

October 10th, 2014, 9:53am by Sam Wang

At this moment, PEC’s probability of D+I is 49%. Yesterday it was 52%. Obviously everything is different, a volte-face. Right? Um…

Only if you don’t have a clear understanding of uncertainty. This is common among even the most experienced journalists [NPR] [WaPo]. It makes the baby Ronald Fisher cry.

For a refreshingly accurate and insightful look at how to think about knife-edge probabilities, Mark Mellman has written an excellent article for The Hill.

[Some people] exchange the numbers for words. “Republicans,” they say, “are likely to take over the Senate.” Well, that depends on what “likely” means to you — and research tells us it means different things to different people. For some people, likely is akin to 90 percent; for others, it could be 60 percent. Thinking about the Senate forecasts this way results in a loss of information and precision.

What [Senate forecasts] may mean is that it is a bit more likely that the GOP wins control of the Senate this cycle than that Democrats keep it.

But consider the classic illustration of probability: coin flipping. If you flipped a coin a thousand times, it should come out heads about 500 times and tails 500 times. If you weren’t actually counting, you couldn’t possibly notice the difference between that 50-50 chance and, say, 510 heads and 490 tails — equivalent to a 51 percent probability. Which is to say, the difference is imperceptible. You probably wouldn’t even notice 590 heads and 410 tails as being particularly off the 50-50 mark.

It’s good – just go read the whole thing.

For this reason, last week’s fuss over the difference between PEC and other forecasters was overblown. Now that we’re microscopically under 50% for the Democrats, I am steeling myself for a round of “aha, now you agree!” However, anyone who has any understanding of the situation should know that control of the Senate is up in the air. All we know is where activists should go to make a difference: Iowa, Colorado, and a shifting pattern of states – today, Kentucky and Georgia.

Also, I estimate that any poll-based estimate, whether at other sites or at PEC, has at least a 15% uncertainty, i.e. the true probability today is between 35% and 65%. Of course, that does not change the fact that the overall trend over the last month has been toward the GOP.

Let me finish by quoting Alan Koczela from yesterday’s thread: “There are worrying signs, even in PEC’s model. The meta-margin tanked in late September and is bouncing around R territory. And, the likelihood D+I number is below 50%….these should be worrying signs for Democrats. No need to panic, but, if you’re not concerned, you’re a fool.” Also see my reply to him.

Tags: 2014 Election · Senate

20 Comments so far ↓

  • Canadian fan

    Sam commented some time back that he felt that this election resembles 2004 in more striking ways than any other recent election. And more and more, that seems to be the case, as the determination of majority control is becoming more and more difficult to truly ascertain – both in individual races and collectively.

    Nate Silver made a very interesting comment some weeks ago, in that he found that the quality of the polling is markedly less reliable than in 2012. The recent polling in Iowa and Colorado would seem – at least in my mind – to bear this out. The wilder Republican estimates – 6 % ahead, 7 % ahead – seem to come from the more brazenly flamboyant Republican pollsters. Most everyone else ( including both partisan leaning pollsters ) have both of these races quite close. The wild estimates as of late ( Fox comes to mind ) have a way of not only grabbing the headlines and narrative ( their latent intent ? ) but manage to squash the picture that most other pollsters are seeing – and that is that these are extremely close races.

    The ground game of the Democrats cannot be easily dismissed, however, as everything has always been determinant on whether they can bring their voters to the polls. We knew that a month ago. We knew that a year ago. If they can, all these numbers and estimates changes. As Democrats have managed in recent elections – including 2010 – to outperform the polling expectations, there seems to be no reason to assume that will not be the case this time. It also needs to be said that the Bannock Street Project is a truly unprecedented effort for a midterm election.

    This election will contain many surprises. And I believe one surprise – and perhaps the determining one – will be the strength of Democratic base support. What makes these elections so particularly fascinating is that we have right now something that is already unprecedented for a midterm election. We have a map that is decidedly Republican in favour, in a midterm during the sixth year of a president’s term who is saddled with poor polling numbers. Historical logic dictates this should be a cake walk for Republicans. It is not. They are being aggressively challenged on their own turf – Kansas, Kentucky, Georgia. They are being surprisingly challenged in South Dakota. The combinations of all of these factors is very new indeed, and it’s hard to imagine that any Republican would have imagined any of these just two months ago. And yet they are a part of the current dynamic. Everything will hinge on whether the Democrats can get their base to the polls.

    • atothec

      That and voter suppression. Georgia Dems did the footwork but it’s apparently perfectly legal for those votes to be dismissed. 40,000 of them. Amazing.

      Vote theft right in front of our eyes.

  • CRM


    This particular Iowa poll with Braley up is from a very partisan Republican internet pollster. A lot like YouGov in their methods, I think.

  • james

    I agree with most of the points here, but I think it’s a bit disingenuous to only compare today’s 49% to yesterday’s 52%. I think the bigger issue is the amount of fluctuation in the daily snapshots. Three days ago the snapshot had the Democrats + Independent in the mid 60s I believe. Dr. Wang, I know that you have frequently cited this variance as a feature, not a bug, but does it make sense that there should be a 15-20% movement in only 3 days based on only a few minor polling changes. If the election were held today, do you really think that the democrats chances would be 15% worse than if the election had been held Tuesday? Maybe I am misunderstanding this?

  • CRM

    Dr. Wang,

    Would a couple or more polls showing Ernst trailing flip it a tad back to D & I probability of control?

    Anyway, here’s one:

    • Sam Wang

      You can figure it out for yourself. Our rule is last 2 weeks, don’t accept older surveys from the same pollster. The data feed is here.

      Looks to me like we have six current surveys. One Braley+ survey would tie it up, two would tilt it toward him. In a situation like this, we’re really in the noise. I don’t think it’s at all obvious who is really in the lead, no matter what the aggregation method.

    • Edward G. Talbot


      Was that conducted on behalf of someone or some group, or is that essentially an independent poll? I know the model doesn’t care, just curious.

    • securecare

      Useful to remember no one is in the lead until votes are actually cast.

      Just sayin.

    • securecare

      @Edward G. Talbot

      From the poll announcement.

      “…Morey Group has conducted telephone and online polls “side-by-side”, which has indicated stronger accuracy among online polls as more than 50% of households do not use a land-based telephone.”

  • Violet

    I panicked for second when your forecast went 49%, but being realistic, a 49% chance of anything is pretty good. I wouldn’t take medication with a 49% mortality rate.

    Compared with Obama’s chances in 2012, which were mostly above 90% for a long time, I refuse to panic over these odds . . . yet.

    People do get so easily misled by statistics. If the risk of something goes from .02% to .04%, I guess you could say it’s twice as likely to kill you, so that’s what the news reports, when, either way, it’s still a tiny chance.

  • Art Brown

    The recent oscillation (I don’t know what else to call it) in the meta-margin (with approximately 1 week period and 2 periods completed) is striking (as well as doing my blood pressure no good at all). Has such occurred previously? (The average actually looks encouraging, like a reversion to a mean following a negative impulse.)

    • Bill Mason

      You can imagine patterns in the movement of the meta-margin. But that might be analogous to seeing patterns in the movements of stock market prices. In reality stock prices seem to follow a random walk. I think Sam believes in a meta-margin model he calls “random drift”, which I imagine is the same as a random walk.

      I did look a little bit at the meta-margin data from the senate_estimate_history.csv file. Between June 1 and October 10 there were:
      31 daily upward moves
      41 daily downward moves
      58 daily sideways moves.
      The average upward move was 0.35
      average downward move was 0.29

      If the meta margin really moves randomly, though, I must admit I have a hard time understanding why the long term, pre-October predictions used the average of the meta margin values instead of the latest value. Of course one answer might be that we’ve used the average in the past very successfully. It’s hard to argue with success.

    • Sam Wang

      The answer is that the autocorrelation in the Meta-Margin went away for intervals longer than 35 days. So the current snapshot might be just fluctuation, until we’re at <35 days. Which of course we are now.

      An alternative would be to look for biased drift. Problem is, with this few states in play that seemed like a dicey approach.

  • Davey

    The media really should get off the “strongly favored to win” kick. And that’s not a political comment, it’s a business comment. They should make every story “these races are all neck and neck and it couldn’t possibly be closer” for two reasons. First, it’s true. Second, you can use that to generate excitement and keep viewers/readers/users coming back. I’m not sure how the “it’s basically over” stories serve them well from a business standpoint.

    On a side note…wow…KY vote power moving on up the list. Who would’ve thunk? Whomever you send to the Senate, Kentucky, it will have been a deliberative choice, that’s for sure!

  • Art Brown

    “… all forecasts, including PEC, have at least a 15% uncertainty.” 1) Why 15%? 2) How could something be more uncertain than 50%? 3) What does 50 (or 49)% +/- 15% mean? Enquiring minds want to know!

  • Alan Koczela

    Thanks for the quote. It made my day. Your points concerning my comment are valid and thanks for clarifying that the D+I only recently dipped below 50%. As for the ground-game, I was venting my spleen a little. Too often, the ground-game is mention as just a throw-away, feel-good line without realizing that follow through is required.

    Sam, you’re in my prayers. For me, I can’t wait for the deluge. I recommend laughing. There’s bound to be a lot of silliness coming your way. These folks who will rain invectives and lamentations were never serious about analysis. They only want to stick it to someone on the other side or read warm and fuzzy rah-rah stories confirming their world view.

    As always, all the best.

  • Alex

    Great post.

    Unfortunately, the media and prominent media figures love to sensationalize everything at the cost of accuracy, clarity, and integrity.

    In general, the media tends to focus on the “horse-race” aspect of elections and fail to adequately report on the POLICY IMPLICATIONS an election may trigger. In the end, it is what actually gets PASSED post-election that substantially impacts our country.

  • Zeke Hunkaburning

    Thanks for pointing us to that Mellman read. Insightful stuff. The great unwashed masses, and by that I mean “the media”, “the chattering classess” however you want to refer to those people, should sit up, take notice, and start speaking and writing as if they have a clue about probability: that may give their followers a better understanding of the closeness of the struggle for the Senate instead of giving the impression that it will be a run-a-way victory for the GOP’s winning the Senate majority.

    I like Mellman’s term for the Bayesian’s outlook: a 50% certainty, which kinda, sorta sounds like an oxymoron. The power of suggestive language. Nothing certain about a 50% chance of anything.

    But in an earlier piece, Finding the Forecast, concerning the House race, Mellman seems to suggest that in this particular nerd-off we may never know who was right. In the House race, the inside the beltway prognosticators using an analytic approach v the modelers, Mellman gives the nod ( a slight nod) to the analytics. Then he closes with this: “In the end though, unlike with the probabilistic models of Senate control, we will know who was more and less right.”

    • Felix

      Not only the ‘great unwashed masses’ should read this, but every experimental scientist. The blatant misuse of statistics in scientific journals of any color is shameful. That journalist don’t get it, that they will say Wang has given in to Silver is almost excusable.