(Update: The comments are excellent. Thank you all. I’ve been a bit of an empty chair because of travel. I gave my own thoughts a few times, most recently Sept. 1, 4:36pm.)
Based on evidence from 2004-2010, I take the position that polls aggregated using robust, median-based statistics lead to the most accurate outcome for Presidential, Senate, and House races. Furthermore, the most appropriate measurement is whatever is closest to the final mechanism, e.g. state margins for the Electoral College. For confirmation, see the results documented in the left sidebar.
Today I ask for help in cataloguing ways in which these assumptions might be wrong in 2012. I will list a few just to get you started.
2004: Kerry voters in the woodwork. In 2004, when I originated the Meta-analysis, I assumed that polls did not reflect the true state of play between Kerry and Bush. At the time, it was discussed by Mark Blumenthal (Pollster.com) and many others that undecided voters typically broke against the incumbent. I also thought that Democratic energy was high, and that turnout might be underestimated.
These possibilities found their way into the Meta-analysis using a variable called, appropriately, bias. If crowds of pollsters leaned towards the Democrats, then bias > 0; towards Republicans, bias <0. As it turns out, bias=0.0%. Although using the bias in 2004 gave me a wrong result, this Wisdom-of-Crowds-of-Pollsters is invaluable in getting a snapshot of the 2012 race.
The bias variable has turned out to have many other uses. Now it powers many features here: the Meta-margin (equivalent to a national margin, but calculated from state polls), The Power Of Your Vote (tells you where your efforts are most effective), and the Obama+2% and Romney+2% features.
Rasmussen, (dis)honest broker? In 2012, I have heard various speculations about what’s wrong with polls. A common theme is that there is something wrong with polls. Republicans seem to think Rasmussen is the only unbiased pollster in town. I think everyone agrees that the value of bias for him is typically 4% relative to other pollsters. As stated above, I’m pretty sure he’s the one who is off.
The Trouble With Nate? Matt McIrvin wrote me regarding a speculation that because Nate Silver made an error in predicting the 2010 House elections, pollsters are on average pro-Democratic by several points. As I showed yesterday, this error was well within the election-to-election variation of linkage between popular vote and House seats. Also, as I wrote in 2008, voter enthusiasm in nonswing states seems to be hard for pollsters to measure. So I think the case is weak.
However, there may be other evidence. This year’s race is close enough that if bias>0, it might have significant effects for the Presidency, the Senate, and the House. If you can cite reasons for why this might be true, I’d be interested in knowing. I’ll help you – here are two popular right-leaning aggregators, Race42012 and ElectionProjection.com.
The comment thread is now open.