For a scientist, a moment of truth for a hypothesis is the experiment. In this case the “experiment” is the election. Here’s a quick run-through comparing our predictions with outcomes so far. Performance is extremely good. As was the case in 2004 and 2006, the results are consistent with the idea that relatively pure polling data can predict outcomes accurately. That’s n=3 elections.
The Electoral College
Outcome: Obama 365 EV, McCain 173. The map (NE 2 not shown):
FiveThirtyEight: 348.5 EV. Error: 18.5 EV.
Electoral-vote.com: 353 EV. Error: 12 EV.
Our last-day Median EV Estimator for Obama: 352 EV. Error: 13 EV.
Our prediction: Obama 364 EV, McCain 174. Error:1 EV.
Variance-minimized polling snapshot (post hoc): Obama 364 EV, McCain 174. Error: 1 EV.
Closest: Princeton Election Consortium.
Individual state wins
FiveThirtyEight: 50 out of 51 correct, Indiana missed.
Electoral-vote.com averages: 49 correct, 1 incorrect (Missouri), 1 tie (Indiana).
Our prediction: 50 correct, Indiana missed.
Closest: Tie between the Princeton Election Consortium and FiveThirtyEight.
State margin estimates
Correlation of PEC state poll margins with actual margins, +0.975. FiveThirtyEight absolute error: 3.9 +/- 4.9 %, median 2.5%.
PEC absolute error: 4.1 +/- 3.8 % (mean+/-SD), median 3%.
PEC’s margin was closer in 21 out of 51 races (95% CI, 19 to 32 races).
Closest: Tie between the Princeton Election Consortium and FiveThirtyEight (FiveThirtyEight ahead by a statistically nonsignificant margin).
Net Bradley effect plus cell-phone bias
Outcome: Obama +0.7 +/- 0.8% from Meta-Analysis, 0% from popular vote.
Our prediction: maximum of Obama +1%.
FiveThirtyEight: Obama +2.8%.
The closer estimate: Princeton Election Consortium.
More after the jump.
Popular Vote Share
Outcome: Obama 52.7%, McCain 45.9%, third-party 1.3%.
Our prediction: Obama 53.0%, McCain 46.0%, third-party 1.0%.
FiveThirtyEight: Obama 52.3%.
Other comparisons are here, along with lively discussion.
The closer prediction: Princeton Election Consortium.
The Senate has unresolved issues:
In Alaska, convicted felon Ted Stevens (R-i) appears to be ahead of the last pre-election polls, which showed clear Begich leads but were also volatile (a 22% lead, then only an 8% lead). About 20 times as many early voting ballots as the current margin are yet to be counted. So Begich is likely to pull this one out. Even if Stevens wins, he still faces probable expulsion from the Senate. This would be followed by another election to replace him. Paging Senator Palin? Update: Begich won.
In Georgia, Chambliss (R-i) did not clear 50% and faces a Dec. 2 runoff against Martin (D). The odds favor Chambliss. He received 49.9% the first time and now does not have to run with Obama at the top of the ticket. Update: Chambliss won.
In Minnesota, the final count shows Coleman (R-i) ahead of Franken (D) by 0.02% 0.01% (fewer than 500 400 300 206 votes) . State law mandates a recount. Unsurprisingly, Coleman doesn’t want one. Update, 1/5/2009: looks like Franken by 225 votes, a margin of 0.01%.
And of course we don’t know who will take the place of Obama and Biden, though they will be Democrats.
I estimate that the Democrats have an 85% chance of getting to 58, and a 40% a 45% chance of getting to 59. But it will be some time before we find out. Update: 59 it is.
Finally…A note on methods. The Electoral College and state win predictions were done using state polls only. The cell-phone adjustment was done using Pew Center data. The Senate and House predictions were done using Congressional polls only. The popular vote share prediction was done using national polls only. Turnout was estimated using the work of Curtis Gans and data from InTrade and the Census Bureau. In no case was any demographic or pollster-specific information used. Overall, the results show that a high degree of accuracy is possible without complex model-building.