It’s hard to tell because of the ongoing post-debate surge for Obama. When events come so dense in time, it’s unclear what is really game-changing. For instance, on Sept. 24th, John McCain “suspended” his campaign. This has been dinged by Barron’s as being a critically bad event. On the 26th the first presidential debate occurred. All throughout was the ongoing economic crisis. During this period tracking polls showed a strong surge for Obama. Could there be said to be a single triggering event?

Recall my deconvolution of Gallup Daily’s 3-day moving average, in which I use the principle of variance minimization to extract single-day results (MATLAB code available here). Around the time of the first debate it gives

Assuming these numbers are correct, a 9-point swing or larger is significant (P<0.05, two-tailed). The 11-point swing from 9/24 to 9/25, which was sustained on 9/26, suggests that McCain’s suspension had an unintended bad effect for him, one that was maintained by the Friday debate. Or perhaps we should think of the two events as being a single event in the minds of voters.

There’s no visible bounce at all after the VP debate:

…which suggests that if the VP debate did move opinion, it’s not detectable this way. There’s a +11% event on 10/1 for no clear reason, which might be noise. The original average tells us the same thing, as does the EV estimator.

Of course, polls are not the only measure of a debate. Debates are also among the few chances we get to hear about opposing policies and plans, side by side. Also, as I wrote before, Palin’s performance may have rehabilitated her image with GOP base voters, many of whom liked what they saw. Regardless of whether they would be better off taking their party in a different direction, if McCain loses Palin’s a contender for the 2012 nomination .

Update: Here’s another local event, happening next Tuesday: a panel on the reliability of state polls featuring both pollsters and academics, including Andrew Gelman, author of Red State, Blue State, Rich State, Poor State. I hear the panel will be webcast. If so I’ll post a link.

In 2004, the median Electoral Vote estimator got noticeably spiky in October. The following graph uses the last-3-polls rule alone:

in large part because individual polls passed through the average quickly.

At the very end of the campaign we made a last-minute switch to a last-three-polls or last-7-days (counting back from the date of the most recent poll), whichever gives more data for a state. This performed extremely well on Election Eve 2004, nailing the result. We’ll start using the new rule tomorrow morning. The rest of the procedure – from calculating the median and SEM onward – will stay the same.

To reflect the posting schedule of our data partner, Pollster.com. the EV estimator will now be updated four times a day at 8:00am, noon, 5:00pm, and 8:00pm. We will stop doing the midnight update since Eric Dienstfrey and Mark Blumenthal haven’t been putting up data at night. (Good for them – they need their rest.)

Look at the current-polling probability map. There are two surprises: Pennsylvania (21 EV) is tied and Minnesota (10 EV) is very close. This is the closest that either state has been in the post-primary season. Before the conventions, these states appeared to be headed for Obama wins. Now the picture is less clear.

Perhaps this is part of the same bump that McCain received in frontier states such as Idaho, Montana, and the Dakotas. Another possibility is that one or more of the particular polling organizations – Big Ten, Fox/Rasmussen, the Star-Tribune – have some bias. As polls become more frequent we will adopt a rule that incorporates more than the last three, which is what we are doing now.

In any event, current state polls were mostly taken before last week’s financial events, so look for further changes this week.

You may have noticed that the Meta-Analysis is showing a fairly different result from FiveThirtyEight. The basic reason is that FiveThirtyEight attempts a prediction, whereas the Meta-Analysis is a pure snapshot of state polls. Major additional steps by FiveThirtyEight are (a) a correction to state polls to get them caught up, assuming they track national polls in a certain way; and (b) a projection to November 4th. There are other differences as well, but these two contribute a large amount to the difference at any one moment.

Update: The median-of-three approach rejects single outliers but does not work well when there are two outliers in the same direction. Not to point fingers, but the Big Ten polls seem somewhat fishy, and might add outliers in many of the ten (of course) eight states they cover. A reasonable solution would build on using more than three polls; for instance, all polls completed in the last seven days. I did this during the home stretch in 2004 with good results. Another attractive possibility is age-dependent weighting, but it’s not clear how to combine that with median-based statistics. I’m open to suggestions.

The Zogby Interactive approach potentially gets around biases that may occur in landline-based surveys, the polling industry standard. For example, millions now have cell phones but no landlines. On the down side, Zogby Interactive is still working out the kinks in this method. For example, they reported some distinctly odd results during primary season. For now I’m dropping them.

At some point in the near future we will probably recalculate the history without their data. For now, I’ll leave it alone.

Silver’s move is wise. There’s no a priori reason to “discount” a political event. After all, those shifts in opinion are real measurements. We also have no idea how long-lived they are – only some shifts are lasting, partly because later events can dispel or cement an impression. I’m glad to see him take a step in the right direction. Both in 2004 and this year, I have found good insights from readers. The one caveat is that one has to integrate advice carefully. There is the danger, as the referee, of being worked.

I have the converse problem, whether to include the nationally-measured bounce in a state poll-based snapshot. This has its own problems: it’s a trade-off between maximum precision (using state polls only) and improved timeliness (putting national shifts into the mix). Because events are moving quickly, I included an adjusted EV estimate in my previous post. But I’m still a general proponent of leaving unnecessary spices out of the stew. In other words, let’s see what the next wave of state polls brings – after Palin and McCain’s acceptance speeches this week.

In the meantime, let’s return to an topic that is obscure, but loved by some hardcore polling enthusiasts: co-variation.

A substantial fraction of my mail continues to concern the subject of correlated swings between states, and how that correlation may affect the snapshot. I addressed this topic before, giving an argument for why it does not matter for a snapshot. However, some people remain unconvinced. So let me take another bite at the question – with data.

The central assumption of the Meta-Analysis is that on average, recent polls sample a population equivalent to participants in a real election held today. Election Eve polls perform quite well (see here and here) in predicting Election Day outcomes. In the face of this fact, how should we think about the idea that outcomes among states are coupled?

At a basic level, this information is already contained in the primary polling data, which give margins in all states. Consider the following four states:

TX — CO — IA — NJ

where TX is the most reliably Republican, NJ is the most reliably Democratic, and the others are somewhere in between. If states are constant relative to one another – in other words, if fluctuation is uniform nationwide – McCain could win {TX}, {TX, CO}, {TX, CO, IA}, or {TX, CO, IA, NJ}, but no other combinations. Reflecting this, the order of polling margins is usually TX < CO < IA < NJ (or the other way around, depending on your political preference).

This tendency toward rank-ordering is related to why the probability distribution in the right sidebar is spiky. At any given moment, only a few states are in play. Recently, Obama campaign manager David Plouffe cited his focus on 18 states. Most of these states are represented in the jerseyvotes (voter power) calculation in the right sidebar. The number in substantial doubt on Election Day will certainly be fewer.

Co-variation is said to come up in two cases. Let us consider them.

Case 1: Polls reflect hypothetical voting, but a state is won by the candidate who is not leading in polls.

One intuitive view says that an unexpected outcome is evidence that polls are biased, and other states are likely to be off in the same direction. However, the current state probabilities indicate if we guessed the winner using a probability>50% criterion, on average we would get 3 of them wrong. This reflects the fact that the race appears to be near-tied in six states. Surprises are to be expected. Indeed, they are the reason for the existence of the Meta-Analysis.

Case 2: Sentiment differs between the polled sample and the voting sample.

This can happen for various reasons. Voter sentiment can change over long periods, especially when the election is months away. Some voting population may be undersampled, such as voters who have cell phones but no landlines.

The tendency for opinion in different states to shift (or differ) from polls in the same direction and by similar but nonidentical amounts has been addressed in detail by Nate Silver here and here. Instead of discussing the subject of how these shifts may vary from state to state, today I want to bring up a more fundamental modeling question: How much does adding covariation affect the statistical properties of a current-poll snapshot or a projection? This leads to the action item: In a snapshot or future projection, to what degree of detail is it necessary to consider covariation in detail?

Let’s look at the modeling question with a simple, worst-case calculation using today’s data.

To simulate covariation, let’s assume that true voting differs from the polls in every state by the same amount, S. This assumption is the simplest – and most extreme – version of the covariation idea. Modeling this case puts an upper bound on the size of the effect.

However, we don’t know what S is. So let it vary. For instance, if the election were today, it might vary over a range of -1% to +1%. In November the range might be wider. In all cases I will distribute it randomly around zero. It can also have a bias, but that’s not today’s subject.

In the Meta-Analysis, varying S is easily done by setting biaspct (which is S) to a range of values, calculating the probability distribution for each value, and averaging the distributions. Here is what we get:

Adding covariation to the Meta-Analysis

(Click the image to see details)

In all three cases, the distribution is spiky, as implied by the rank-ordering principle. As expected, the peaks are in exactly the same locations in all three distributions.

The statistics of the distribution are (all values given in Obama EV):

Meta-Analysis with no covariation:
Median: 298. Mode: 305.
68% confidence interval: [280, 312], 32 EV wide.
95% confidence interval: [267, 328], 61 EV wide.
Probability >=270 EV: 96%.

With +/-1% covariation:
Median: 298. Mode: 305.
68% confidence interval: [279, 314], 35 EV wide.
95% confidence interval: [264, 332], 68 EV wide.
Probability >=270 EV: 94%.

With +/-2% covariation:
Median: 298. Mode: 305.
68% confidence interval: [275, 319], 44 EV wide.
95% confidence interval: [259, 341], 82 EV wide.
Probability >=270 EV: 90%.

In all three cases, the median and mode are the same. With a covariation of +/-1%, the confidence intervals get 3-7 EV wider. With a covariation of +/-2%, the confidence intervals get 12-21 EV wider.

Covariation of +/-1% is an amount of variation one might expect on Election Eve – the snapshot approach. In this case covariation does not affect the basic estimate, but it does increase the uncertainty by a modest amount.

Covariation of +/-2% (or more) represents a case in which national voter sentiment goes someplace – that is, a prediction. The more the covariation, the wider the spread. The decreased certainty of the outcome is reflected in the tails of the distribution and the decreased probability of getting to 270 EV. If you understand the Popular Meta-Margin, you will not be surprised by this. Co-variation of +/-2% creates some scenarios in which opinion moves by almost the amount of the Meta-Margin and the trailing candidate has a chance. The Meta-Margin has the advantage of telling you explicitly what this quantity needs to be.

(By the way, I think “win probabilities” are intrinsically inaccurate for an event two months off, mainly because of the difficulties of figuring out the right assumptions. For this reason I don’t report it, and I think the differences above are of no consequence.)

The bottom line of these models is this: For a snapshot, covariation doesn’t matter. For a long-term prediction, changes in national sentiment need to be modeled in some way that should include partial or total covariation.

Finally: Since the biggest source of uncertainty in modeling long-term changes is knowing how large the national changes will be, an adequate assumption is total covariation. This is a simple but effective approach to long-term prediction.