I’ll continue my consideration of factors that may systematically bias polls. Yesterday it was the effect of cell phone users, which I estimated could very slightly undersample Obama supporters. Today let’s consider latent racism.
A hot topic among polling nerds is the “Bradley effect,” which occurs when a non-white (usually black) candidate falls short of opinion polls on Election Day when he/she runs against a white candidate. For this reason it has been suggested that support for Obama might be overstated – a hidden bonus for John McCain. Now comes a large-scale empirical study (in preprint form) by Harvard political scientist Dan Hopkins. He finds that since the mid-1990s, the Bradley effect has disappeared. His paper is a must-read.
Many people believe that opinion polls overstate support for the black candidate, for instance due to latent racial bias that respondents are unaware of or will not report explicitly. The topic was mentioned in a recent article on racial bias by AP’s Ron Fournier, which did not contain relevant polling data but still sparked discussion.
The Bradley effect gets its name from former Los Angeles mayor Tom Bradley, a black man who lost the 1982 governor’s race despite the fact that in opinion polls taken before the election he led George Deukmejian, a white man. Sometimes it is also called the Wilder effect, after Gov. Doug Wilder of Virginia, who had a comfortable lead in led by nearly 10 percentage points in his 1990 campaign, but only won by a whisker.
Until now, the empirical evidence for the Bradley effect rested on individual cases. Such cases might suffer from biased assimilation, our tendency to more readily accept examples that favor our position. A counterexample (see pages 487 and 490 of this PDF) is the 1989 mayoral primary race in New York City between David Dinkins and incumbent Ed Koch. There, Dinkins, a black man, led Koch, a white man, by 0-5 percentage points in the closing weeks, only to win by 8 percentage points. Other counterexamples are available as well. Because of the mixed evidence, the Bradley/Wilder effect has been controversial. Gary Langer, director of polling for ABC news, has referred to the Bradley effect as “a theory in search of data.”
Now Dan Hopkins has gathered some highly relevant information. In a recent paper he analyzes polling data and election outcomes for 133 gubernatorial and Senate races from 1989 to 2006. One result can be seen in this graph.
Polls did show a significant Bradley/Wilder effect through the early 1990s, which includes the period when Bradley and Wilder were running for office. However, Hopkins notes that the effect then went away in races from 1996 onward. To quote the study: “Before 1996, the median gap for black candidates was 3.1 percentage points, while for subsequent years it was -0.3 percentage points.”
Hopkins doesn’t know why this is – the data are after all correlative. He speculates on possible reasons, such as de-emphasis on race identity and tension. But something has changed to remove the discrepancy between polls and voting. It could even be methodological: Pollsters could be training interviewers differently. Automated polling by robot interviewers could remove bias. We don’t know.
The paper contains other interesting conclusions – for instance, there is also no evidence for a gender effect (the “Whitman effect”). One variable that does affect poll accuracy is that support for front-runners is often overstated. This effect averaged 1.9 percentage points, and could account for some of what happened to Bradley and Wilder. This year, such an effect would favor John McCain.
All this is not to say that racism is dead, or that people don’t use race to decide who to vote for. These phenomena still exist. However, Hopkins’s study does suggest that when it comes to opinion poll accuracy, black candidates do not suffer a hidden disadvantage compared to white candidates. I’ll look forward to seeing the final article.
Acknowledgement: Thanks to reader William for the article!