[Note: this is a work in progress. I'm basically seeking comment as I develop a November predictive model. Please give your feedback... -Sam]
I’ve been asked why the PEC Senate poll snapshot is more favorable to Democrats than forecasts you’ll find elsewhere: NYT’s The Upshot, Washington Post’s The Monkey Cage, ESPN’s FiveThirtyEight, and Daily Kos. All of these organizations show a higher probability of a Republican takeover than today’s PEC snapshot, which favors the Democrats with a 70% probability.
Today I will show that in most cases, added assumptions (i.e. special sauce) have led the media organizations to different win probabilities – which I currently believe are wrong. I’ll then outline the subtle but important implications for a November prediction.
If you want to get caught up on the major approaches to Senate models, start by reading my POLITICO piece. There I categorized models as “Fundamentals-based (Type 1)” and “Polls-based (Type 2)”. The major media organizations (NYT, WaPo, 538) have all gone with a hybrid Type 1/Type 2 approach, i.e. they all use prior conditions like incumbency, candidate experience, funding, and the generic Congressional ballot to influence their win probabilities — and opinion polls. What does that look like?
The first data column is the current PEC poll median. The next two columns show what a polls-only win probability looks like. Finally, the last three columns show the media organizations’ win probabilities. All probabilities are shaded according to who is favored, the Democrat (blue) or the Republican (red). “sum6″ is the sum of probabilities (converted to seats) for six key races: AK, AR, CO, IA, LA, and NC.
Let me make some general points.
Senate Democrats are doing surprisingly well. Across the board, Democratic candidates in the nine states above are doing better in the polls-only estimate than the mainstream media models would predict. This is particularly true for Alaska, Arkansas, and North Carolina. In these three states, Democrats are outperforming the expectations of the data pundits (The Upshot’s Leo, Nate Silver, Harry Enten, John Sides, etc.). Why is that, and will it last?
“Fundamentals” pull probabilities away from the present. For PEC and Daily Kos, the win probability is closely linked to the poll margin. The Daily Kos model was created by Drew Linzer, of Votamatic fame. Both are based on polls alone.
The mainstream media organizations are a different story. They show a general tendency to be more favorable to Republicans. For Alaska (AK), Arkansas (AR), and North Carolina (NC), the discrepancy between PEC/DKos and NYT/WaPo/538 is rather large. Where PEC shows an average of 4.02 out of 6 key seats going Democratic, those organizations show 2.75 to 3.16 seats. This key difference, 0.86 to 1.27 seats, is enough to account for the fact that PEC’s Democratic-control probability is 70%, while theirs is between 32% and 42%.
Longtime readers of PEC will not be surprised to know that I think the media organizations are making a mistake. It is nearly Labor Day. By now, we have tons of polling data. Even the stalest poll is a more direct measurement of opinion than an indirect fundamentals-based measure. I demonstrated this point in 2012, when I used polls only to forecast the Presidency and all close Senate races. That year I made no errors in Senate seats, including Montana (Jon Tester) and North Dakota (Heidi Heitkamp), which FiveThirtyEight got wrong.
In 2014, these forecasting differences matter quite a lot. This year’s Senate race is harder than any electoral forecast that the other forecasters have ever had to make. To be frank, 2008 and 2012 were easy. My own experience is guided by 2004 Presidential race, which was as close as this year’s Senate campaign. In 2004, I formed the view that the correct approach is to use polls only, if at all possible.
The present is more sharply focused than the future. DailyKos, the Upshot, and FiveThirtyEight have win probabilities are closer to 50% than PEC’s in 18 out of 27 cases. This mostly reflects the fact that they are trying to predict November races on an individual basis. Generally, this drags their total expectations toward randomness. This reason also contributes to why their expected number of wins in key races is closer to 3.0 than PEC’s 4.02.
The exception is The Monkey Cage, which shows higher certainty than PEC in 6 out of 9 races – but sometimes in the opposite direction. This suggests that their model must be heavily weighted toward fundamentals. That is very brave of them.
Can a prediction be made from polls alone? This is the question I am currently wrestling with. Using the 2004-2012 campaigns as a guide, I suggest that the Senate campaign’s future ups and downs can be gleaned from looking at what’s already happened since June 2014:
The PEC polling snapshot has mostly favored Democrats. Starting from June 1st, Democrats have led for 61 days and Republicans for 26 days, a 70-30 split. During that period, the Senate Meta-Margin has been D+0.24±0.57%. Assuming that the June-August pattern applies to the future, I can use this Meta-Margin, and the t-distribution with 3 d.f., to predict the future, including the possibility of black-swan events. The result is that the November Senate win probability for the Democrats (i.e. probability that they will control 50 or more seats) is 65%.
Finally…I note that this is all a work in progress. I’m using PEC as a sandbox for kicking around ideas. With that, I invite reader reactions.