Princeton Election Consortium

A first draft of electoral history. Since 2004

The Midterm Polling Curse (Morning-after edition)

November 5th, 2014, 11:41am by Sam Wang

Pre-election PEC Senate aggregate: 52 Republican seats.
Outcome: 52 or more Republican seats (Alaska is not called, and Louisiana goes to a runoff).

As I wrote in The New Republic, last night’s performance by the GOP was remarkable. In close Senate races, Republicans outperformed polls by an average of 5.3 percentage points. Prime examples of that effect could be seen with Republican wins in Kansas and North Carolina, two races that went against pre-election polls.

In gubernatorial races, Republicans outperformed polls nearly 2 percentage points on average. This was enough to put Paul LePage of Maine (tied), Rick Scott of Florida (tied), and Bruce Rauner of Illinois (Quinn +2.0%) over the top. All in all, Republicans had an excellent night.

Historically, midterm polling is much more prone to large biases than in Presidential years. In 2010, Democrats benefited; in 2014, it was Republicans. In six Senate races that were polling within less than three percentage points, two were won by the lagging candidate. That is entirely in line with past results. Added to the median poll-based snapshot of 52 Republicans, 48 Democrats+Independents, the result could be as large as a convincing 54-46 majority.

Before the election, I pointed out the possibility that polling bias could go in either direction. It is likely that pollsters face a tough challenge in identifying likely voters in an off-year.

With control the Senate so closely fought, even a small bias put into question who would control the chamber. And, as I wrote, it also opened the possibility of a GOP blowout. I said we didn’t know what would happen. Maybe we can call that my Peggy Noonan moment.

Brier scores

Over the weekend I suggested Brier scores as a way to compare predictions. Aggregators and analysts did worse than in 2012, when polls did not miss any races (PEC Brier score, 0.01; scores close to zero are considered good).

I used final probabilities as listed at The Upshot to calculate Brier scores. The lowest (and therefore best) score came from Drew Linzer (DailyKos Elections), who took a Bayesian polls-only approach and ended up with a Brier score of 0.10. Coming in second was The Washington Post with a mostly-polls approach, at 0.12. Next came HuffPost, FiveThirtyEight, and Betfair got 0.14, followed by The Upshot at 0.15. And finally we have PEC, with 0.18. Although the number of “misses” (i.e. being on the wrong side of 50% probability) was no worse than the other sites, we were done in by an across-the-board lack of certainty, which we predicated on the unreliability of midterm polls. Congratulations to Drew Linzer!

Postscript: as pointed out by commenter Paul, Drew Linzer shines even more if his calculation’s performance in the several months prior to the election is included.

P.P.S.: Doug Rivers at YouGov has evaluated his own organization’s miss of actual-voter behavior, as well as that of other polling organizations. The findings seem consistent with what I’ve reported here.

→ 64 CommentsTags: 2014 Election · governors · Senate

Election night liveblogging, 9:00PM

November 4th, 2014, 9:23pm by Sam Wang

See below the fold for older commentary. The most recent comment will appear up top.

12:10am: Tonight’s performance by the GOP has been quite remarkable. In close Senate races, Republicans seem to be outperforming polls by around 5 percentage points. That goes a long way toward explaining what is happening in Virginia. In close gubernatorial races, Republicans are outperforming polls by about 3 percentage points.

I did say that historically, midterm polling can be off in either direction by a median of 3 percentage points – far worse than Presidential years. Tonight is certainly consistent with that.

11:30pm: Ernst will win Iowa. Other than New Hampshire, it’s looking like a sweep of close races by Republicans. Counting CO, GA, IA, KS, and NC gets to 52. Alaska and Louisiana are still outstanding, but that’s icing on the cake for the GOP.

[

→ 131 CommentsTags: 2014 Election

Election Night liveblogging, 7:00PM

November 4th, 2014, 7:09pm by Sam Wang

See below the fold for older commentary. The most recent comment will appear up top.

8:40pm: The Upshot has projected counts. For now, use those for your Geek’s Guide. Shaheen (D-NH) around +5% and McConnell (R-KY) around +13%, both ahead of their pre-election polls. Ambiguous for estimating Delta.

8:27pm: Reader Forrest asked me how The Upshot estimates vote share from partial returns. I can’t say what they are doing, but look at Jay Boice’s HuffPollster calculation. Basically take the prior history of the state, county by county (or whatever level of granularity you have available). Then slide over all the counts in past comparable elections, and see how each county would have to break in order to reach a 50-50 tie. Use that as an over/under, i.e. calculate whether a candidate is over/underperforming that expectation. Then do a weighted average across counties. That is an estimator of the margin between candidates. [Read more →]

→ 84 CommentsTags: 2014 Election

Errors Are Inevitable – But Who Will Benefit?

November 4th, 2014, 2:00pm by Sam Wang

Despite the certainty of pundits, we actually don’t know who will win the Senate! In The New Yorker, I explain.

From 2004 to 2012, only thirteen Senate races have had margins of less than three percentage points in the week before the election. Of these, four were won by the trailing candidate. One more, the Florida 2004 race, was tied in the polls, and was eventually won by the Republican, Mel Martinez, by 2 percentage points. Scoring that one as half correct, the overall rate of wins by a front-runner is 65%, a bit better than chance.

In light of that, the probability that all six close Senate races (AK, CO, IA, KS, NH, and NC) will be won by the candidate in the lead is only 7%. A wrong call is almost inevitable. We should not be surprised to see one to three races to be won by the candidate who trails this morning. This allows us to hazard a guess as to the most probable path to Democratic retention of the Senate (which PEC currently has at 35%). [Read more →]

→ 38 CommentsTags: 2014 Election · governors · Senate

Geek’s Guide To The Election, 2014 edition

November 4th, 2014, 12:00pm by Sam Wang

Download it! It includes instructions for estimating Delta. Liveblogging will start around 8:00pm.

Useful links (will add as day goes on):
HuffPollster: Senate Election Live-Tracker.
DailyKos Elections, hour-by-hour guide.
New York Times, The Upshot, Senate tracker.

→ 4 CommentsTags: 2014 Election · governors · Senate

Senate and House final snapshots, 2014

November 4th, 2014, 1:54am by Sam Wang

Here are final polling snapshots for Senate races:
Put your own predictions in comments! Some more notes… [Read more →]

→ 91 CommentsTags: 2014 Election · Senate

Gubernatorial final snapshots, 2014

November 4th, 2014, 12:54am by Sam Wang

Here are final polling snapshots for gubernatorial races that are either close or likely to switch party control:

Put your own predictions in comments! Some more notes… [Read more →]

→ 20 CommentsTags: 2014 Election · governors

Grading policies

November 3rd, 2014, 8:33pm by Sam Wang

How should PEC be graded? As the last few polls trickle in, let me give a suggestion for how to evaluate predictions after the election. Late tonight I’ll give actual predictions (and give you a chance to record your own predictions).

My preferred measure is the Brier score. As I explain this concept, I’ll refer to some suggestions from FiveThirtyEight and Drew Linzer. [Read more →]

→ 14 CommentsTags: 2014 Election

Reality check: Obama net approval/disapproval

November 3rd, 2014, 7:00pm by Sam Wang

Coming into the home stretch, President Obama’s net approval/disapproval rating is at minus 8%. Not good…but 4% better than June. This is what candidates face as in-person voting starts tomorrow morning.

→ 8 CommentsTags: 2014 Election · House · Senate

Hidden errors, overconfident pollsters?

November 3rd, 2014, 10:05am by Sam Wang

I am thinking about how to get the most accurate last-minute snapshots of races, and how to turn that into a scorecard for you (and me) to use on Election Night. I’m also thinking about Brier scores as a means of evaluating the various prognosticators, including me.

In the meantime, here’s your morning reading: an excellent analysis by David Rothschild, Sharad Goel, and Houshmand Shirani-Mehr on the problems endemic to current polling practice. They analyze 2012 in great detail, and identify errors that go well beyond sampling error. Because these errors are unlikely to have been fully corrected this year, they think there’s a good chance that Democrats will outperform poll aggregates. In other words, all poll aggregators, including PEC, might carry a hidden bias. My own view is that based on historical data, errors have gone in either direction by several percentage points across the board. Go read their article!

→ 41 CommentsTags: 2014 Election · Senate