The debates are crucial – for both sides

October 3, 2012 by Sam Wang

Zooming around the web is a suggestion that Presidential debates do not matter in determining election outcomes. A 2008 Gallup study claims “no direct correlation between the winner of each debate and the winner of the presidency.” Let’s ask two questions. First, is post-debate change measurable with a sensitive enough instrument? Second, is the change limited to the Presidential race, or can it spread downticket? I believe that Republicans and Democrats still have much to gain or lose from tomorrow’s debate. (Update: I see David Gergen and I agree.)

A figure from a new book by Erikson and Wlezien, The Timeline of Presidential Elections, appears to make the same point as Gallup:

As great as this graph is (I am finding their book to be quite a valuable resource), the claim above misses the mark in two ways.

First, let us test whether the debates have any effect on opinion using the 2004 and 2008 campaigns, cases in which the Meta-Analysis was available to provide precise information. As most PEC readers know, the Meta-Analysis is a more sensitive readout of national opinion than other poll aggregators. The bottom line: the first debate can drive opinion by the equivalent of several percentage points of popular vote margin.

The more clear-cut case is 2004. This was a remarkably close-fought race between John Kerry and President George W. Bush. The lead switched several times during the general campaign season. In the home stretch, Bush pulled it out – barely. You can see the history here:

Median EV estimator from 2004 race

It is fairly clear here that Debate #1 was followed by a jump in Kerry’s fortunes of almost 30 EV. Indeed, at the time Kerry was generally thought to have bested Bush. The next two debates had little further effect – though the initial jump did persist to the finish.

From a cognitive standpoint, it makes sense that the first debate would make a difference. Generally speaking, contrasts between two objects or persons are easier when they are side by side. The usual format of campaigns allows access to only one candidate at a time, making them hard to compare directly. For example, who is quicker on his feet? Heck, who is taller?

In 2008, the first McCain-Obama debate was followed by a giant move towards Obama:

Here there is a complication: just before the first debate, Lehman Brothers collapsed, setting off an economic meltdown. This appears to have sent the race spiraling toward the Democrats. Whatever the cause, by Gallup’s definition, it is not obvious that events from mid-September onward mattered for the Presidential outcome, since Obama was already leading.

If you compare this with the “08” data point in the Erikson/Wlezien graph, you will see that the effect is large, but they concealed this by plotting total vote share, as opposed to before/after change. Their data did not allow a finer-grained measurement.

You can get a feeling for how large these effects are by examining the Popular Vote Meta-Margin. The approximate conversion is 15-20 EV/% Meta-Margin. A clear victory in tonight’s debate might swing opinion by up to 1-2%.

That would not change this trajectory:

In this light, my modification to Gallup/Erikson/Wlezien is that the effects of the debates are usually smaller than the margin between the candidates, and therefore unlikely to flip an outcome. This year, even if Romney does extremely well, he will still probably make up only 1-2% of the >5% margin between him and President Obama. That is not enough. This is comparable to other events that Andrew Ferguson and I have marked on these charts: the Bain attacks (25 EV), the Ryan and Palin VP nominations (20-35 EV), and the conventions (0-30 EV).

However, all of this misses a point that may be quite critical this year. The Presidential outcome is only one measure of the debates’ effectiveness. As I have shown, downticket Senate races this year have tracked Presidential preference closely. The same is likely to be true of House races. These linkages are driven in part by partisan voter intensity. If one side’s intensity fades or surges, it will affect races at all levels on the ticket. The Obama-Romney debate may well influence the shape of the Congress that the President will face in January.

21 Comments

Tapen Sinha says:

IMHO, you should add 47% comment in the last picture. Focus groups are clearly showing that it had a big bite. Of course, there are other opinions around
http://www.tnr.com/blog/107829/47-was-bad-romney-ryan-has-been-deadly
Tapen

David Mace says:

I am hearing from my daughter in NC that the 47% had a big effect. Teaching colleagues with retired military Rish voter husbands have expressed being deeply offended.

Michael Worley says:

Sam,
Off-topic, but looking at the polls today and yesterday, I’m not sure why the Meta-margin is still rising. Are a lot of old polls dropping out, meaning the Meta-margin is relying more on the polling from the end of last week as we’ve had relatively few polls so far this week?

Froggy says:

The Meta-Margin only drops polls if new polls come out. The movement over the last few days has mainly been a number swing states (Colorado, Nevada, New Hampshire, Iowa) moving from about Obama +2 to Obama leading by four or more percent, along with North Carolina dropping back into a tie. Dropping old polls in Ohio (when new numbers were released on Sunday) also moved that state from about Obama +4 to Obama +8.
With many swing states now very blue (see Ohio, Wisconsin, Nevada, and New Hampshire), it would take a big shift toward Romney to make this race even.

Joel says:

The biggest outlier looks like Ford in ’76, picking up 10 (?!) points on Carter after the debates. I wasn’t alive then, so I’m wondering if there’s some sort of mitigating factor at play.

George says:

Joel – I’m wondering about that too – being tuned into that election a great deal, the conventional wisdom was that Carter picked up points on Ford’s Poland gaffe.

Sam Wang says:

The event was Watergate. Richard Nixon resigned in 1974 after a massive scandal that brought the political life of the nation to a standstill. Vice-President Ford had been the replacement for Spiro Agnew, who resigned after his own scandal in 1973. For many, Ford was an unknown quantity. It is easy to imagine that a Ford-Carter debate would weigh disproportionately on voters’ minds.

Todd Horowitz says:

@Sam: Watergate could not be the event that triggered Ford’s boost in the ’76 debate.
Ford may have been an unknown quantity when he became President, but by the debate he had been President for some time. furthermore, he was a more known quantity than Jimmy Carter, who was a dark horse going into the Democratic primaries. Remember that Ford had to fight for the nomination, so it’s not like Carter had the spotlight to himself prior to the conventions.
This leaves open the question of why Ford gained from the debate.

Sam Wang says:

The trajectory of that campaign was far more variable than a typical re-election campaign. Ford’s numbers moved over a total range of nearly 20 points during the general election season. He had also never stood in a national election before. Empirically, Ford did not behave like a typical incumbent. It was a strange, strange campaign year.

MarkS says:

An issue is that comparable moves exist when there is apparently no corresponding event (eg. in April o4 and May 04). I’m not sure how to do it, but it would be interesting to study the statistical significance of the correlation between median-EV moves and a pre-selected list of events. Just eyeballing the graphs is not very convincing evidence of such a correlation, given the human tendency to see patterns in everything.

Than Hedman says:

Prof. Wang,
Any chance we could see a “median senate seats with confidence intervals” graph over time?
Cheers

Olav Grinde says:

The last Missouri poll has Romney with a mere 3% advantage over Obama. Do you deem this a swing state?

William Ockham says:

I am not sure that there is evidence of casuality here. If we accept the idea that the debate caused the change in the polls, what is the mechanism? I’m not aware of any studies that show that significant numbers of people change their minds based on debates. I understand how conventions could boost a presidential candidate’s standing in the polls. Conventions function largely as advertising, a chance to get your message out without opposition or rebuttal. But the evidence there suggests that convention bounces are often transient.
Debates seem to me to have less of a chance of making an impact, given that there is no way for a major party candidate to really blow a debate. All the just-so stories that are told (“Ford freed Poland”, “There you go again”, etc.) see a bit forced to me. Are partisan leanings that shallow?
There are always more than one thing going on in a campaign at any one time. How do we tease out how much of a shift was due to a debate and how much due to something else?

Amitabh Lath says:

Sam, I am a litle uncomfortable with assigning causes due to time-correlations.
In 2004, there seems to be a delay of one week (?) before Kerry’s EV score moves up due to Debate #1.
In 2008, there was an upswing underway before Debate #1. The first derivative of the EV score does not change, although I suppose one could argue that the debate did contribute.
Does anyone ask voters directly “did the debates change your mind about who to vote for?”

Sam Wang says:

Agreed about causes v. correlations. The best one can do with these data is identify when a change occurred. I still maintain the likely causality.
To really nail this, I would bring in the argument that news organizations seem to be able to handle about one major story per week. We’d have to ask what other stories there were. A second factor is audience – debates are watched. Channel 79 (or whatever), less so.
In 2004, note that the 1-week delay is not out of line given the delays of polling.

Amitabh Lath says:

Yes, maybe the effect of debates is to renew the news coverage, now with a win/lose narrative. So even those who didn’t watch the debates can get influenced.
I’m glad you understand my wariness of explanatory narratives. If the numbers had gone the other way, one could make a storyline explaining that as easily. It’s tough to fight human nature’s need for a explanation.

xian says:

Aren’t debates also a head-to-head competition of frames in the Lakoff sense? I’ve always felt on some level that in your gut you often feel like you know who is going to win very early based on more of a sense of which narrative is the most compelling.

Matt McIrvin says:

Since most of the livebloggers I’ve been reading seem to think Romney is winning this debate by a mile (I’m staying away from it myself to preserve my sanity), I suppose we may get another data point to test the hypothesis. How many points of Meta-Margin is a week of “Obama beleaguered” stories worth?

Steven S says:

Is there anything notable about the general observation that only 2 of the 10 data points show a post-debate improvement for the democrat? Are republicans generally better debaters? Or are the positions above and below the line just noise?
Those two cases – 2004 and 2008 – happen to be the ones for which PEC has data, with 2008 showing the bigger bump on the Erikson/Wlezien plot and in the meta-EV. Hmmm, the two candidates followed by PEC do well in debates…a causal effect? Now there’s a narrative!

pechmerle says:

Xian, wholeheartedly agree with you that the clash of Lakoff frames is what most viewers get from the debates.
I’ve been disappointed over the past couple of years with Obama’s poor effort to dominate the framing, vs. showing how well he understand the “facts.” But as in 2008, he’s done well so far this year in setting the frame.
That leaves the depressing thought that if (when) he wins, he’ll go back to being poorer at setting the frame in governing than he is in campaigning.

OwlOfMinerva says:

Very poor debate by Obama, solid if unspectacular by Romney. Looks like Obama dropped 6% or so on Intrade.

Leave a Reply

Your email address will not be published. Required fields are marked *