Princeton Election Consortium

Innovations in democracy since 2004

Outcome: Biden 306 EV (D+1.2% from toss-up), Senate 50 D (D+1.0%)
Nov 3 polls: Biden 342 EV (D+5.3%), Senate 50-55 D (D+3.9%), House control D+4.6%
Moneyball states: President AZ NE-2 NV, Senate MT ME AK, Legislatures KS TX NC

National polls and state polls: same methods, different predictive accuracy

November 2nd, 2012, 11:00am by Sam Wang

(original version released on temporary site; comment thread)

Yesterday I wrote a long piece on why national polls paint a different picture from state polls, despite the fact that they use very similar methods. Here’s the ten-cent version – the central argument.

Imagine for a moment that national and state polls use exactly the same methods (not exactly true, but close enough). Historically, pollsters as a group do well. But they aren’t perfect. In 2000-2008, national-poll medians missed the final outcome by 0.3%, 1.4%, and 2.5%, despite the fact that perfect methods would have missed by 0.6% on average. So there’s a large systematic error. How would this affect one’s snapshot view of the national and state race?

This year, the national race is close. A systematic error of 1-2% would make it hard to accurately determine who was in the lead nationally. But state races are usually less close. Even Ohio, a critical swing state, has a median of Obama +3.0 +/- 0.5%, a lead that would not be altered by that systematic error. Indeed, at the moment only two states are within range of flipping in such a way: Virginia and Florida. The likely outcome of the other 49 races would still be determined correctly. Across the country, the Presidency is decided by winner-take-all elections. Therefore our Meta-Analysis of State Polls is likely to come closer tothe correct result than national polls. In coming days I’ll combine the two to come up with a final prediction of the popular vote margin.

I should say that for similar reasons, the U.S. system of electing a President is more fraud-proof than a simple popular vote. Even if there were voting error in one state, the effects would be contained there, like flooding on a compartmented ship. Without the Electoral College, every time there was a close national race we’d have the Florida 2000 dispute (Bush v. Gore) in every precinct in the country. Blech.

Tags: 2012 Election · President

6 Comments so far ↓

  • Bruce Wayne

    How can Obama be ahead by 1% in FL, when he is down by ~1% in the Pollster average?

    • geo

      Because the polling aggregate is based on the median (the middle) value, not the average

  • Bruce Wayne

    Likewise for NC. Pollster’s average has Romney ahead by ~2%, but you have him ahead by only 0.5%. Why the discrepancy?

    • Mike

      Dr. Wang doesn’t use averages. He uses medians. That gets rid of the outliers.

  • Michael Weissberger

    Bruce, see this link for a description of the method:

    But the piece you’re missing is that Sam uses median-based statistics to discard outliers, while using a narrower time-window to restrict evidence to relatively recent polls.

    The answer to your specific question is that he doesn’t use the pollster average – he uses the median of the last 3 polls, or of all of the polls that have a median date within 7 days of the most recent poll.

  • Steven J. Wangsness

    Pollster makes certain “adjustments” to the raw polling data; Nate Silver, too. Plus, different aggregators may exclude some polls from less well-known firms.

    Good point about avoiding a nationwide, precint-wide recount — but that would be highly improbable, seems to me.

Leave a Comment