At The American Prospect: Tweet In the home stretch, I wrote that midterm polling is far less accurate than in Presidential years. Today, in The...
The Midterm Polling Curse (Morning-after edition)
Pre-election PEC Senate aggregate: 52 Republican seats.
Outcome: 52 or more Republican seats (Alaska is not called, and Louisiana goes to a runoff).
As I wrote in The New Republic, last night’s performance by the GOP was remarkable. In close Senate races, Republicans outperformed polls by an average of 5.3 percentage points. Prime examples of that effect could be seen with Republican wins in Kansas and North Carolina, two races that went against pre-election polls.
In gubernatorial races, Republicans outperformed polls nearly 2 percentage points on average. This was enough to put Paul LePage of Maine (tied), Rick Scott of Florida (tied), and Bruce Rauner of Illinois (Quinn +2.0%) over the top. All in all, Republicans had an excellent night.
Historically, midterm polling is much more prone to large biases than in Presidential years. In 2010, Democrats benefited; in 2014, it was Republicans. In six Senate races that were polling within less than three percentage points, two were won by the lagging candidate. That is entirely in line with past results. Added to the median poll-based snapshot of 52 Republicans, 48 Democrats+Independents, the result could be as large as a convincing 54-46 majority.
Before the election, I pointed out the possibility that polling bias could go in either direction. It is likely that pollsters face a tough challenge in identifying likely voters in an off-year.
With control the Senate so closely fought, even a small bias put into question who would control the chamber. And, as I wrote, it also opened the possibility of a GOP blowout. I said we didn’t know what would happen. Maybe we can call that my Peggy Noonan moment.
Over the weekend I suggested Brier scores as a way to compare predictions. Aggregators and analysts did worse than in 2012, when polls did not miss any races (PEC Brier score, 0.01; scores close to zero are considered good).
I used final probabilities as listed at The Upshot to calculate Brier scores. The lowest (and therefore best) score came from Drew Linzer (DailyKos Elections), who took a Bayesian polls-only approach and ended up with a Brier score of 0.10. Coming in second was The Washington Post with a mostly-polls approach, at 0.12. Next came HuffPost, FiveThirtyEight, and Betfair got 0.14, followed by The Upshot at 0.15. And finally we have PEC, with 0.18. Although the number of “misses” (i.e. being on the wrong side of 50% probability) was no worse than the other sites, we were done in by an across-the-board lack of certainty, which we predicated on the unreliability of midterm polls. Congratulations to Drew Linzer!
Postscript: as pointed out by commenter Paul, Drew Linzer shines even more if his calculation’s performance in the several months prior to the election is included.
P.P.S.: Doug Rivers at YouGov has evaluated his own organization’s miss of actual-voter behavior, as well as that of other polling organizations. The findings seem consistent with what I’ve reported here.