Princeton Election Consortium

A first draft of electoral history. Since 2004

Brief update

August 30th, 2014, 7:33am by Sam Wang


Thanks for your feedback. The comment thread from the previous post clarified my thinking. Drew Linzer dropped in! Also, welcome to Rachel Maddow viewers.

The banner at the top of the page gives the poll-based “snapshot probability” for who would end up controlling the Senate based on an election held today. Based on PEC’s track record since 2004, this will move toward the final outcome in the coming weeks.

Long-term forecasting is based on methods that worked nearly perfectly in 2010-2012. We will soon install code to display the long-term forecast, which is also based purely on polls. Currently, the long-term forecast for Democratic control on Election Day is 65%, about 2-1 odds in favor.

Tags: 2014 Election · Senate

44 Comments so far ↓

  • SFBay

    It seems that you and Nate at 538 are coming up with pretty much the exact opposite prediction. Based on how I understand your model, as a betting person I would place my bet with you.

  • Bill

    It will be interesting to see how the model changes now that Chad Taylor has dropped out of the race in Kansas.

    • Kenny

      Just saw that today and came here to see if anyone was commenting on tit yet. Curious to see Sam’s take.

  • Bob Grundfest

    Has anyone seen this interactive in the NYT? Interesting that if you click on Polls Only and uncheck the “secret sauce” boxes on the left, the Democrats win. Add fundamentals and the Republicans win.

    http://www.nytimes.com/interactive/2014/upshot/elections-2014-make-your-own-senate-forecast.html?rref=upshot&abt=0002&abg=0

    • Sam Wang

      That is a very cool tool. Good catch!

      One note: play with the “house effects” correction. This correction moves all the probabilities toward Republican candidates. I wonder why all races are affected in the same direction. It seems to me that requires knowing ground truth in a manner independent of polls. Hmmm.

    • DaveM

      HuffPost/Pollster’s new polls-only model also has the Democrats retaining control, albeit by a single Biden.

  • Rieux

    Sam:

    Is the substantive dispute you’re having with these various other quant sources something that is likely to be resolved this year (or anytime soon)? Which is (I think) to say: is there any particular outcome of D/R Senate seats in the 2014 election that would suggest strongly that you are now approaching this correctly, or else, conversely, that those you disagree with are?

    My stats knowledge is meager, and (thus?) I’m having a hard time figuring out how we can evaluate the soundness of an evaluation method that yields a 60% likelihood of 2015 Democratic control of the Senate–or conversely a 40% likelihood. Wouldn’t we have to run the same algorithms in a bunch of elections–i.e., through a large number of election cycles–before the sample size of trials would be large enough to make analysis of those forecasts meaningful?

    • 538 Refugee

      The polls are of likely voters now. You ask them who they would vote for and they tell you. The point Sam seems to like to make is that you just go with that because anything else is just overthinking the problem and hasn’t proven any more successful. In some cases, less successful. I think most, if not all, of those that aggregate polls drop all but poll data as the election draws closer. So, the end result may be the same but how far out were they correct? Bottom line? It ain’t broke. At least yet. Will this be the year that breaks Sam’s, lets call it, non-model? Stay tuned.

    • Sam Wang

      Problem is, I’m only showing 65%. So a GOP takeover would not be a “breakage.” If the GOP gets 52 seats…then I’d have a little ‘splainin to do.

    • Sam Wang

      Here are two tests of the long-term prediction. Both could be done after the election:

      (1) A 52-48 split in either direction would be a bit of a surprise to the PEC calculation. A split of 51D/49R or 52D/48R would be a surprise to the other analysts.

      (2) One could calculate the deviation of long-term probabilities from the final outcome. For instance, subtract the PEC snapshot from the final outcome, then sum up the deviations over time. Do the same for the other analysts. The smallest sum indicates the greatest predictive power.

    • Rieux

      538R, I’m afraid you didn’t grapple with my questions at all. For example, the line “how far out were they correct” simply ignores my basic point–how can we know that a projection was “correct” (or incorrect) if the outcome for which it gave a 60% likelihood ends up happening? Or 40%?

      Sam, test (2) is beyond my meager stats chops; sorry. Possibly it answers my question thoroughly, but I’m not grasping it.

      Wrt (1), though–really? The bar graph you provide as “today’s histogram” lists a ~8% chance of the Dems (+ Sanders + King) ending up with 52 Senators and a ~7% chance of the Repubs getting that number. So, according to today’s projection, there’s a ~15%, or 1-in-6.67, chance that one party or the other will end up with 52. That’s not all that likely, but is it such a black-swan-ish outcome that you’d really “have a little ‘splainin to do”? On average, your model (presuming it’s precisely correct) is going to make that kind of a “mistake” every seven election cycles or so, right?

      This is what has me confused. (And I hope that it’s clear that that confusion applies to all of the quants doing these kinds of forecasts–not just PEC!)

    • Sam Wang

      Answer (2) is most of what I have to say for now.

      From a statistical standpoint, you are right about the ~8%. However, the histogram is less precise than reality because it contains uncorrected house effects. Therefore I would personally look askance at a 52-48 result. Besides, in the public-relations battle I would surely take a drubbing.

      This is great thread but I’ll have to stop chatting soon. It’s like an unending Reddit AMA…

  • Michael K

    So if Senate control is “Bush v. Gore close”, and multiple Senate races appear to be toss ups….
    …that begs the question: what happens if one or more Senate races — upon which control hinges — get mired in “Franken v. Coleman”-style recounts and litigation into January 2015 and beyond?

    Would control be determined by the certified races [as of the start of the 114th Congress in Jan '15] only? And then would control change (conceivably multiple times thereafter) as additional races are certified?

    • Craigo

      Yes to both, basically.

      If a Senator hasn’t been sworn in, then the 99 (or 98, or 97) Senators will vote to organize. And if a new Senator could swing the balance, the new majority would certainly hold a new vote.

      (One Senate in the 1880s had three different presidents pro tem as the balance shifted through the term.)

    • ArcticStones

      And this, my friend, is precisely why the District of Columbia has a non-voting representative in the House, and absolutely no representation in the Senate.

  • riddler

    Sam,

    I see that almost everyone’s model has a 50 R 48 D 2 I outcome of the election as one of the high-probability results. In your projection, for instance, it is currently the highest.

    What I don’t understand is why Angus King (I) is considered a lock to caucus with the democrats, and how his possible “defection” ought to be handled in the projected outcomes.

    Shouldn’t there be a probabilistic assessment of his likely behavior, which assigns some portion of that election outcome on the side of the Republicans with respect to answering the question “will the Republicans ‘take’ the Senate ?”

    Eyeballing your snapshot graph, it looks like you are ‘predicting’ about a 40% chance of results of 50 R, 48 D, 2 I.

    I’m not seeing this accounted for in anyone elses projections either, which means I’m probably missing something that ought to be obvious :-)

    • Froggy

      We’re all assuming that Angus King will caucus with the Democrats if he is the deciding vote because his voting record is more in line with the Democrats. He is considerably more liberal than each and every Republican, and in fact has a more liberal voting record than several Democrats.

  • ArcticStones

    Sam,
    Do we have reliable information/studies of how much GOTV operations has swung state-level elections?

    I am, of course, asking this while keeping in mind that control of the Senate is decided by a number of separate state elections.

  • Wendy Fleet

    Sam, Sam, one of my furry feet is over the edge of The RoofTop. Loathe despair but the notion of McConnell’s Smirk turns all the illustrations in my head to Hieronymus Bosch. Also as MoveOn’s Best Phoner in the Nation a few years back (Got the teeshirt . . .), I do need Genuine Hope to keep spending my very very few pennies giving to Hagan, Grimes, etc (The Six) 3 bucks at an impulse on ActBlue’s handy safe donating site. And to spend the huge time phoning in those races. (If it hadn’t been for R Maddow’s piece on you, I might have already fallen to splat dread.) (For those of you who phone, I always say, “John Kennedy beat Richard Nixon by one vote a precinct — you could be THAT vote.”) I know that the wonderful probably sainted John Lewis says We Don’t Get To Give Up, but your candle at the end of the tunnel has really helped.

    • Froggy

      It might help also to recognize that even if the Republicans take control of the Senate in 2014, there is a excellent chance that the Democrats will take it back in 2016. The 2016 map is very favorable for Democrats, with every seat currently held by the Democrats in a state that Obama won twice, and with Republicans trying to retain seven seats in states that Obama won twice (Illinois, Iowa, Wisconsin, Ohio, Pennsylvania, New Hampshire, and Florida).

    • Amitabh Lath

      If it came to pass, Republican Senate control would be fleeting (as Froggy says) and shallow. The p-value for Republican control peaks for 51 R seats, anything more falls off pretty quickly.

    • Craigo

      At this point the only real noticeable different between a Reid Senate and a McConnell one would be that appointments of federal judges and agency officers would essentially come to a halt. Legislation is still subject to filibuster, and can’t pass the House anyway.

      (That’s still pretty bad, as judicial appointments are one of the President’s most important and least visible powers. And Obama’s first term was constantly undermined by the fact that many executive and independent agencies had no leadership – resulting in unfavorable FEC and NLRB rulings, among other things.)

  • Bert

    Hello Sir.

    I have a question about the Senate race in Mississippi. I’ve been reading through the comments on Chris McDaniel’s Facebook page. A number of Republicans are openly threatening to vote for Childers or 3rd Party out of anger at Cochran and the supposed “betrayal” of McDaniel by the GOP establishment in the state. Write ins will not be counted, but nevertheless a number of McDaniel supporters are saying they will write him in no matter what the law says. Do you think this has any potential to affect that Senate race in November? I think the Cochran camp feels that enough of these voters will fall in line to negate any threat from Childers. And what McDaniel decides to do now that his legal options are exhausted will be revealing. But I still feel that the Cochran camp should be at least a little uneasy. The anger at Cochran among Tea Partiers in Mississippi is still palpable.

  • Art Brown

    1. Um, isn’t the confidence level already included in a 60% number? What does 60 +/- 20% mean? (What would 50 +/- 20% mean?. What racetracks include confidence levels?) 2. I get your point that 40% vs 60% is a very small actual difference. However, in the weeks to come, some estimates will swing into meaningful territory, one way or the other (while others may stay in the middle). For example, your snapshots have been moving Democratic (and now you’re posting an election prediction). We can also watch the others’ evolutions, and see real differences emerge, culminating with the actual results as final arbiter. So, from my perspective, it’s great to see even small differences “on opposite sides”. If and when someone crosses over, they’ll need to explain why. (Also more fun than just agonizing over whether the Dems can hold on.)

    • Amitabh Lath

      The confidence level (CL) is different from the expectation value of a measurement. For instance, if I had a (fair) six-sided dice, my estimate for the probability of rolling a “3″ would be 16.7%, but CL on that estimate would be 99% or so.

      In the case of polls there are statistical uncertainties (how many people called) as well as systematic ones associated with how they do their weighting (what type of people called).

    • Art Brown

      In response to Amitabh Lath:
      Yes, and the win probability depends not only on the expectation value, but also on the standard error (SEM). I think the CL is “built in” to the probability via this SEM. For example, using PEC’s methods, if the median margin is 2 and the SEM is 2 (i.e. margin is 1 SEM), the win probability is 84% , but if the SEM is 4 it drops to 69%.

    • Amitabh Lath

      Art, I don’t know what the SEM is. I presume it is the uncertainty reported by the individual pollsters.
      In other words, their statistical uncertainty?
      I recall looking at Sam’s code a while back, and that is indeed taken into account. (I didn’t chase down into the various subroutines…)

      In figuring out the confidence level one would look at the scatter of the various polls. Too much scatter would lower the confidence one would have that polls know how to measure that state. And I suppose too little scatter might indicate the polls are adjusting themselves to agree with each other, which would be just as bad.

  • Chris McLaughlin

    Just from a business point of view–pollsters need as big an audience as they can muster. If 538 was calling this for the Dems right now they would lose all the GOP viewers—they only want to hear good news. So it is good biz right now to lean to the GOP side for 538.

  • AThornton

    Any plans to analyze the House elections?

  • Amitabh Lath

    Bill has a good point. Maybe you could start putting in the sigma value along with the percentage and meta-margin.

    Most people know that 3 sigma is the “huh, interesting” threshold, and 5 sigma is an observation. (don’t they?)

    Right now the CL is hovering around 1 sigma.

    • Sam Wang

      Ballpark, I estimate that the error bar is about 0.5% in units of Meta-margin. Since probability is a sigmoidal function of Meta-margin, it’s not a good parameter for reporting sigma.

      How about we just give this a few weeks and see if the probability gets over 80%?

    • Froggy

      “How about we just give this a few weeks and see if the probability gets over 80%?”

      Sam, are you crazy? It’s almost September. We want to know the outcome of the election — NOW!

    • Sam Wang

      OK, OK.

      My current thought is that the uncertainty is dominated by us not knowing how much all polls, as a population, differ from true voter opinion. Based on past experience, I would put that at 0.3-0.5%. Let’s say 0.5%. (I think that’s 1 sigma, though I’d want to think about whether I really think that). Therefore the Meta-Margin is D+0.7±0.5%.

      Converting that to win probability gives about 75±15% for the snapshot, 60±20% for the November prediction. In both cases the uncertainty is actually a little smaller in the upward direction.

      Oh, and as for your desire for 3-sigma of certainty…I’ve been saying all along that it’s super-close this year. For November we’re at 0.4 sigma. I’d feel a sense of the probable at 1.0 sigma. It’s like, Bush v. Gore close. Actually, closer since two states are tied at the moment. I think that people will understand *that*.

      Logically this means I think the other guys have uncertainties in the vicinity of 40%.

    • Joseph

      @ Sam: “Logically this means I think the other guys have uncertainties in the vicinity of 40%.”

      IOW, you think you’re effectively twice as certain of the outcome as the other guys, but you are far from certain yourself. Which means the other guys are just this side of wild guesses….

      BTW, I really appreciate the “power” ranking. Helps me decide where to spend my admittedly limited resources. For example, I just chipped in a few bucks for Ms. Grimes. I’m thinking that the forced retirement of Mr. McConnell’s campaign manager will have a positive impact for Ms. Grimes and I want to help take advantage of any temporary weakness.

  • DaveM

    I can’t help thinking that the tendency of folks to take to the rooftops is exacerbated by the simple fact that the Senate has 100 members. So “60%”—when it expresses someone’s projection of the likelihood of a Republican takeover—evokes an extremely short-lived but heart-palpitating panic at the prospect of 60 Republican *seats*.

    In this sense, each knife edge race is a metaphor for control of the Senate as a whole, and neither party has reached the magic 50% threshold.

  • RioVistan

    Thanks for the update. Looking forward to the long term forecast for the Senate.

  • Bill

    Maybe post a confidence interval for the probability. That might help clarify a bit what the percentage means.

  • Amitabh Lath

    Rachel Maddow may be the first in a series of media people who are starting to question the conventional wisdom (peddled up by 538, Upshot, etc) of a Republican takeover of the Senate in light of the polls pointing to the opposite.

    • Sam Wang

      So torn here. It’s been hopeless to get anyone to see that anything in the 20-80% range is too close to call a clear winner. Seeing as how “60%” has people on the rooftops, probability is such a terrible measure. But I feel stuck with it…I wonder if I can sell people on the Meta-margin next.

  • SFBay

    I’m looking forward to the long term forecast. Based upon your forecasts for 2010 and 2012 I expect I’ll have a very good idea about the make up of the 2014 Senate. Though I actually can’t follow the statistical logic behind your model, it’s pretty clear you have an excellent model for forecasting election outcomes.

  • Barth

    Yes. A TRMS viewer directed to you by her. Pleased to meet you. Without any data to prove my thesis, I think that some consideration has to be given—perhaps not the way it is usually done—to figuring out who the real LVs are.

    • Sam Wang

      Welcome! Based on my experience, the likely vs. registered voter distinction makes no significant impact on the accuracy of our estimates. We do use LV over RV when both are available. Generally, I take the view that PEC’s aggregation methods remove the need to chew over the details of individual polls, an activity that is subject to the reader’s biases.