Princeton Election Consortium

Innovations in democracy since 2004

Outcome: Biden 306 EV (D+1.2% from toss-up), Senate 50 D (D+1.0%)
Nov 3 polls: Biden 342 EV (D+5.3%), Senate 50-55 D (D+3.9%), House control D+4.6%
Moneyball states: President AZ NE-2 NV, Senate MT ME AK, Legislatures KS TX NC

The disappearing Bradley effect

September 27th, 2008, 1:03am by Sam Wang

I’ll continue my consideration of factors that may systematically bias polls. Yesterday it was the effect of cell phone users, which I estimated could very slightly undersample Obama supporters. Today let’s consider latent racism.

A hot topic among polling nerds is the “Bradley effect,” which occurs when a non-white (usually black) candidate falls short of opinion polls on Election Day when he/she runs against a white candidate. For this reason it has been suggested that support for Obama might be overstated – a hidden bonus for John McCain. Now comes a large-scale empirical study (in preprint form) by Harvard political scientist Dan Hopkins. He finds that since the mid-1990s, the Bradley effect has disappeared. His paper is a must-read.

Many people believe that opinion polls overstate support for the black candidate, for instance due to latent racial bias that respondents are unaware of or will not report explicitly. The topic was mentioned in a recent article on racial bias by AP’s Ron Fournier, which did not contain relevant polling data but still sparked discussion.

The Bradley effect gets its name from former Los Angeles mayor Tom Bradley, a black man who lost the 1982 governor’s race despite the fact that in opinion polls taken before the election he led George Deukmejian, a white man. Sometimes it is also called the Wilder effect, after Gov. Doug Wilder of Virginia, who had a comfortable lead in led by nearly 10 percentage points in his 1990 campaign, but only won by a whisker.

Until now, the empirical evidence for the Bradley effect rested on individual cases. Such cases might suffer from biased assimilation, our tendency to more readily accept examples that favor our position. A counterexample (see pages 487 and 490 of this PDF) is the 1989 mayoral primary race in New York City between David Dinkins and incumbent Ed Koch. There, Dinkins, a black man, led Koch, a white man, by 0-5 percentage points in the closing weeks, only to win by 8 percentage points. Other counterexamples are available as well. Because of the mixed evidence, the Bradley/Wilder effect has been controversial. Gary Langer, director of polling for ABC news, has referred to the Bradley effect as “a theory in search of data.”

Now Dan Hopkins has gathered some highly relevant information. In a recent paper he analyzes polling data and election outcomes for 133 gubernatorial and Senate races from 1989 to 2006. One result can be seen in this graph.

Hopkins analysis of the disappearing Bradley/Wilder effect

Hopkins analysis of the disappearing Bradley/Wilder effect

Polls did show a significant Bradley/Wilder effect through the early 1990s, which includes the period when Bradley and Wilder were running for office. However, Hopkins notes that the effect then went away in races from 1996 onward. To quote the study: “Before 1996, the median gap for black candidates was 3.1 percentage points, while for subsequent years it was -0.3 percentage points.”

Hopkins doesn’t know why this is – the data are after all correlative. He speculates on possible reasons, such as de-emphasis on race identity and tension. But something has changed to remove the discrepancy between polls and voting. It could even be methodological: Pollsters could be training interviewers differently. Automated polling by robot interviewers could remove bias. We don’t know.

The paper contains other interesting conclusions – for instance, there is also no evidence for a gender effect (the “Whitman effect”). One variable that does affect poll accuracy is that support for front-runners is often overstated. This effect averaged 1.9 percentage points, and could account for some of what happened to Bradley and Wilder. This year, such an effect would favor John McCain.

All this is not to say that racism is dead, or that people don’t use race to decide who to vote for. These phenomena still exist. However, Hopkins’s study does suggest that when it comes to opinion poll accuracy, black candidates do not suffer a hidden disadvantage compared to white candidates. I’ll look forward to seeing the final article.

Acknowledgement: Thanks to reader William for the article!

Tags: 2008 Election

11 Comments so far ↓

  • mark

    How about any evidence of the reverse maybe white voters who tell everyone including pollsters that they are voting for the white candidate, but secretly voted for the black candidate?

  • Majorman

    This is an excellent and informative posting. It seems the pundits like to discuss the Bradley/Wilder effect to try to drive up racial controversy. I posted about the cell-only voters here:

  • Toni

    Thank you for this important report. I’ve suspected – hoped for – this.

  • Sam Wang

    mark, what a voter does after answering a survey is not guaranteed. If there are such people, they are not more numerous on average than the converse group.

  • maria

    The polls show that African-Americans, especially women, overwhelming support Obama. In fact, I saw one indicating 99% of African Americans would vote for Obama. As a mental health provider, patients disclose information to me that they would not publicly. Surprisingly, I have already come across a number of African Americans telling me, in a whisper in my office, that they would not vote for Obama. The fact that these patients whispered this disclosure even when my office was obviously private, might suggest peer pressure on African-Americans to say the right thing in public, i.e., vote for Obama, or suffer some social consequence.

  • Craig Z.

    I’m generally not one to worry a lot about grand conspiracies, but I must confess that my first thought after Palin joined the Republican ticket was that it could embolden cheaters to attempt to steal the election. If the outcome is significantly different from the polls they can now point to both race and gender to “explain” it. Depending on where it occurred that might be enough to limit investigations and quell major protests.

  • Preston

    I think this is interesting yet not conclusive. The premise of this analysis is that the Bradley effect would be equally prevalent in each part of the country. I understand that that there are not likely enough data points to look at this at a finer grain. Yet I would like to see this broken down by region and also the frequency of African American candidates. It’s possible that the first black candidate suffers from the Bradley effect while subsequent candidates benefit from the trail blazer.

    In the case of Obama such a (unproven) theory would suggest polling is accurate in IL, NY, and CA be less so in other states.

  • Mark

    Dr. Wang, since you’ve read the paper (I’m assuming), I wonder if you could comment on something. These data on their face seem to imply that in the early 90s, some racist people felt social reluctance to admit to a pollster that they weren’t going to vote for a black candidate, then got into the polling booth and wouldn’t go near the candidate — but that in recent years this “shyness” with the pollster has presumably disappeared. But the finding raises questions:

    1. In the past few years, automated polls that don’t involve a human interviewer have come into wider use. Were these polls excluded from the study, to keep things “apples and apples”? If not, it simply could be that polls like the Rasmussen are making it look like people are no longer reticent to be honest with pollsters, when in fact it’s because they’re dealing with a machine and can tell the poll what they really think.

    2. Did he control for rate of inclusion of African American respondents in the later polls vs the early ones? It seems like such a “reverse Bradley effect” (simulating a higher A-A turnout) could simply mask a reticence to be honest that is still there on the part of white voters.

    3. Fifteen years seems like an awfully short time in which for a social tendency like this to disappear. Assuming #1 and #2 above do not explain the finding, did he look at any available measures of societal racial attitudes over time as a covariate? I’d be curious to know whether he thinks that (a) people have gotten less racist or (b) people have gotten bolder/less embarrassed to admit it out loud (maybe because of talk radio validating their viewpoint). Conventional wisdom would say that it’s (a) but it’d be interesting to see what the hard data say.

    4. Was there any trend over time for the races being more or less northern vs southern? I ask because some of the discussion of primary polling this year seems to show a regional bias in terms of possible “Bradley effect”.

    Thanks for the post and for your site.

  • Larry

    Methinks maria’s comments require investigation: i.e., is she somehow influencing her patients’ whispered declarations?

  • Bob Haskell

    I’m with Mark, above. I’d like to hear statistics about the opposite effect – white people saying they won’t vote for a black candidate who end up doing so.

    Maybe, just maybe, some people who’ve said they won’t vote for Obama because we just “aren’t ready” for a black president – or whatever other self-deceptive reason they dredge up to justify their unwillingness to vote for him – just might, once they’re alone, vote for Obama, as the only sane choice, overcoming their racism in the private moment of choice.

    The “reverse Tom Bradley effect”… Given the drastic consequences and importance of this year’s choice, it just might go that way.

  • Patrick

    I don’t think several of the other comments reflect an understanding of the Bradley Effect; this is not a theory of peer-pressure amongst black people, who feel they must show racial unity publicly but then vote for the white candidate.

    The theory of the Bradley Effect is that racist whites hesitate to tell a pollster that they won’t vote for a black candidate. This is empirically supported by the fact that the Bradley effect is much stronger in areas that have a larger white demographic. The Bradley effect is minimal to non-existent in majority-minority districts.

    It’s role historically is generally accepted, perhaps wrongly, however its applicability today, and especially to Obama specifically, is certainly in question. I feel it probably doesn’t apply to Obama for many of the same reasons cited in this article.

    I would also caution Maria with the same advice as Larry: hearsay certainly doesn’t constitute poll data. It might be especially true in your case because as a mental health worker people probably like you and generally seek your approval, and will be more inclined to say agreeable things (or even be legitimately persuaded to change their votes) if they detect that this will be received well. But even if these people are voting McCain, the sample and method don’t constitute scientifically controlled polling data.