The Election Day prediction for Senate control is now fully live. There are parts that may require further tinkering, especially concerning how the results are displayed. But it’s basically in place.
Those of you who followed PEC in 2012 will recognize all the components. Basically, I have adapted the Presidential model for the Senate. Here I will document key components that are needed for the Election Day prediction.
First, let me summarize the core principles of any model at the Princeton Election Consortium.
- Polls are the only inputs, and are never “corrected.” Median-based statistics are used to reduce the influence of outliers.
- Fundamentals based in political science have a minimal role.
- The most important output of the model is the Meta-Margin (and not the win probability).
- Polls from earlier in the year are used to predict future outcomes.
- PEC’s results let you direct your attention and activism to the races that matter most.
- All the code is open-source.
Today I will focus on principles 2, 3, and 4. They involve assumptions that should be examined critically.
Principles 2 and 4 are the biggest differences between us and NYT/FiveThirtyEight, which use models that are based on polls, but also draw upon non-polling fundamentals to nudge expectations for where the race ought to be. The general gist of the fundamentals is that political conditions this year favor the Republicans. Since Democrats are currently outperforming those expectations, the other models predict that in the next two months, Senate polls will drift toward the Republicans. At the core, this is why those models lean more than we do toward the possibility of Republican control of the Senate in 2015.
However, this year a fundamentals-based correction is fraught with difficulty. The reason is that the Senate is likely to split fairly close to 50-50 between Democrats+Independents and Republicans. In this circumstance, even a small amount of imprecision in the non-polling-based estimates can send a prediction awry. In fact, I estimate that FiveThirtyEight’s consideration of fundamentals has the effect of shifting the polls toward Republicans by about 2.0 percentage points. That may not seem like much, but it is enough to flip the sign of the prediction.
To illustrate the difficulties associated with a fundamentals-based prediction, let’s take the House of Representatives as an example. In midterm elections, on average, the Congressional ballot preference moves away from the President’s party from Labor Day to Election Day. However, in the last four midterm elections (2010, 2006, 2002, and 1998), the ballot preference has gone toward the President’s party, away from it, or not changed at all. That is a demonstration of the fact that fundamentals-based models are quite variable in their predictions, perhaps too much so to help.
This year, the data so far indicate that fundamentals aren’t matching the polling data so far. On average, in midterm elections, opinion usually goes against the President’s party. But here is what the generic Congressional ballot looks like.
Most of this graph is not very far from the blue line, which labels the national House vote in the 2012 election. In that respect, the generic Congressional ballot is looking rather different from the campaign of 2010, when opinion favored Republicans all year. In other words, the average fundamentals-based expectation for the House is not being met in 2014.
Now let’s look at Senate polls. Here our best source is PEC’s own Senate Meta-Margin, an extremely valuable tool. The Senate Meta-Margin is defined by how much polls would have to be shifted, across all states, to bring the Democratic/Independent control probability to exactly even odds for the two parties. Since the Meta-Margin is in units of public opinion, it’s a convenient quantity to think about, and to model.
Here is this year’s graph for the Senate Meta-Margin.
The most apparent quality of the Meta-Margin is that it has been above the red line, i.e. favoring Democrats+Independents, for most of this year. (The last big jump at the start of September reflects the withdrawal of Chad Taylor (D) in the Kansas Senate race, which opened the way for independent candidate Greg Orman.) Also, it isn’t moving strongly in either direction. To put it simply: Democratic candidates have steadily outperformed the fundamentals-based models.
Considering the lack of obvious directional drift, I have chosen to model changes in opinion over the coming two months as a random variable that can move in either direction, toward Republicans – or toward Democrats.
The range over which the Meta-Margin is likely to roam can be estimated from the graph above. Since June the Senate Meta-Margin has fluctuated between R+1.0% and D+1.7%. Its average has been D+0.3%, with a standard deviation of 0.6%. Based on this, we would expect polls in late October to have a Meta-Margin of D+0.3±0.6%.
The next question is how to convert this predicted Meta-Margin to a probability of control by one party or the other. The model (MATLAB code here) uses the following assumptions. I note that these are normal, reasonable assumptions, do not add bias to the outcome, and are neutral.
Systematic bias in polls. On average, polls will differ from true election outcomes. This difference arises from pollster biases and misjudgments (“house effects”) and other sources of noise. Systematic bias is the main source of uncertainty in interpreting the Senate Meta-Margin. I have assumed that the systematic bias has an average (rms) value of 0.7%.
The Orman offset. Since Greg Orman (I-KS) was not a factor from June through August, his candidacy is not reflected in the history of the Senate Meta-Margin. His viable candidacy effectively shifts the Senate Meta-Margin toward the Democrats+Independents. If we give him a 50-50 probability of a November win, this effectively moves the Meta-Margin toward Democrats+Independents by 0.47%.
What if the fundamentals-based models are right after all? Despite the absence of evidence thus far, future movement remains a possibility. I would loosely refer to this as a black-swan event that is not predictable from past polling data. The exit of Chad Taylor (D) from the Kansas Senate race was a black-swan event (though note that I did anticipate this event). The possibility of movement that matches the FiveThirtyEight bias is also a black-swan effect, and can be modeled using a t-distribution.
To put outlier events into perspective, the conversion between Meta-Margin and Senate seats is about 1.0 percent per Senate race. So an outlier event would be the equivalent of more than two seats flipping than one would normally expect. For example, Jay Cost at The Weekly Standard says“a true repeat of the 2010 wave should therefore give the Republicans 54 seats.” According to the PEC model, that would be 4 sigma away from our prediction, a clear outlier event.
What’s the bottom line? With these assumptions, the probability of Democratic+Independent vs. Republican control is expressed by asking: will the Senate Meta-Margin be positive or negative, where its historical mean is D+0.3% and its sigma is 0.92% (the propagated combination of the standard deviation, 0.6%, and the systematic error, 0.7%).
Including the Orman offset, Pr(Meta-Margin>0) is expressed by the MATLAB expression tcdf((+0.3+0.47)/0.92,1)=0.722, which rounds off to 0.70. It’s even possible to put an error bar on this probability. Using the systematic error as sigma, 1 sigma lower gives a probability of 0.524, and 1 sigma higher gives a probability of 0.822.
In other words, without equations: the probability that Democrats and Independents will control 50 or more seats is 70%. Because of uncertainties, this probability has a likely range of 50 to 80%.
When converted to a seat count, the Senate Meta-Margin corresponds to a caucus of 50.3±0.9 Democrats and Independents. The likeliest outcome is a 50-50 split, with Greg Orman having to decide how he will caucus. This prediction is plotted at the top of this post, with the 1-sigma “strike zone” plotted in red, and the 2-sigma zone plotted in yellow.
As the election nears… Fluctuations in the Senate Meta-Margin suggest that the daily snapshot starts to be predictive of the final outcome about 5 weeks in advance. When that day comes, we will start using the daily snapshot to weight the Election Day prediction.