Improvements are in progress. Go to the interactive map and right-click a state. You will see options: access to original polling data, results from 2004, and the effects of adding support to your favorite candidate. More features will come.
And now for today’s question. How much has cell phone usage affected the reliability of polls? The answer may surprise you: Depending on what pollsters do about it, not much at all. Obama’s support may be understated by as little as 1%.
The question of whether polls have systematic errors is a continuing one. In the recent polling news is a Pew Center study that hits hard on the question of cell phone users. According to the survey, failing to survey people who have cell phones but no landline leads to a net underestimate of Obama’s support relative to McCain. According to a previous Pew/AP survey, cell-onlys comprised nearly 13% of households at the end of 2006. Cell-onlys prefer Obama over McCain by 18-19% (compared with an even split in the landline sample). Uncorrected, this leads to an error of about 0.13*0.185 = 2.4% in the Obama-McCain margin. Clearly this is significant, which is the Pew Center’s conclusion.
Pollsters have two strategies for dealing with this problem.
Strategy 1: Weighting.
National pollsters usually compensate for undersampled populations by weighting their results by age, race, marital status, and other factors. For example, landline registered voters aged 18-29 prefer Obama over McCain by 13% (52% to 39%; n=250). Since they are underrepresented in surveys, pollsters count their responses more heavily.
But there is a rub: cell-only registered voters don’t have the same sentiments as their landliner counterparts. Matched by age, they prefer Obama by a far larger margin of 35% (62% to 27%; n=146). That’s a discrepancy of 22%, which throws the weighting approach into question. What’s different about cell-onlys?
The Pew/AP survey is illuminating. It shows that of people aged 18-25, landliners are far more likely to live with parents than to rent (50% vs. 29%) than cell-onlys (19% vs. 57%). Cell-onlys are more likely to be in school, use Facebook/MySpace/email, and drink alcohol. Finally, cell-onlys are less likely to be married than landliners (8% vs. 15%).
This last factor helps pollsters. One of the greatest predictors of Obama/McCain support is marital status. Marrieds tend to favor McCain, and nonmarrieds favor Obama, for both men and women. The swing is huge, around 30%. Marital status is a much bigger correlate than gender! On this factor alone, pollsters who weight by marital status might get a correction of 30% * (15%-8%) = 2.1%. That would reduce the discrepancy to 20%. Still bad, but it’s an improvement.
Summary: With weighting, pollsters could conceivably get the systematic error arising from missed cell-onlys down to 1-2%. (The Pew survey itself puts the post-weighting systematic error at 2%.) It’s possible to imagine getting the error down even further with clever weighting on other demographic variables.
But why not try the more direct approach…
Strategy 2: Call cell phones.
This is an obvious solution that’s been avoided because autodialing technology to call cell phones is illegal, necessitating manual dialing, which costs more. But more pollsters are starting to do it, as reported by Mark Blumenthal.
Based on examination of organizations that include cell phones, Nate Silver estimates the cell-phone discrepancy as being rather large, 2.8%. His estimate has a standard deviation of 2.6% and an SEM of 1.0%. So I would take his estimate as being unreliable. Nonetheless, calling cell phones is a feature of polling is likely to improve matters, and is here to stay.
Conclusions (for now)
National polls that don’t call cell phones may understate the Obama-McCain margin by 1-2%. Some are on their way to overcoming this discrepancy by calling cell phones. Conservatively, I estimate that the overall error in national surveys is about 1%. It would be a mistake to assume anything larger. Obama supporters, the number of ponies here is small.
A big unanswered question is whether state polls perform weighting. Unfortunately, this is what we really need to know in order to gauge the effect of cell-onlys on the Meta-Analysis. Perhaps someone can enlighten me on this point?