Princeton Election Consortium

Innovations in democracy since 2004

Outcome: Biden 306 EV (D+1.2% from toss-up), Senate 50 D (D+1.0%)
Nov 3 polls: Biden 342 EV (D+5.3%), Senate 50-55 D (D+3.9%), House control D+4.6%
Moneyball states: President AZ NE-2 NV, Senate MT ME AK, Legislatures KS TX NC

When game-changing events collide

September 7th, 2008, 9:01am by Sam Wang

( readers, welcome. Try the Camembert. Zut alors!)

Nate Silver has an interesting post on cracking a tracking poll. The goal is to take Gallup’s 3-day rolling averages and extract the 1-day data. The last two weeks are worth analyzing this way because so many potentially game-changing events have occurred. However, Silver’s results appeared to have odd fluctuations, which made me wonder if he has it quite right.

In a strict sense it can’t be done with certainty. But a probabilistic approach is possible. Here is my approach, which successfully identifies simulated game-changing events and, in the case of real data, gives a clear answer. First, my tentative conclusion: Based on Gallup data, the biggest recent one-day bounces came after Hillary Clinton’s speech and after Sarah Palin’s speech.

The question is how to take N three-day rolling averages and infer N+2 one-day polls. This is a system of N equations for N+2 unknowns, which cannot be solved algebraically. Therefore we need more information or assumptions. The approach I have taken is to find a solution that minimizes the overall variance in the one-day polls. This is equivalent to assuming that on days when no major events happened, public opinion shifted minimally, if at all. Let’s see where this leads us.

I took Gallup Daily Tracker data going back to August 16th:

(click here for original Gallup page)

The quantities varied were the first two one-day polls contributing to the earliest three-day average. Knowing these two numbers is sufficient to determine all the other one-day polls. They were iterated to minimize the overall variance in each candidate’s support using this MATLAB script.

The result is (weeks start on Monday):

Date Obama %, McCain % (leading candidate and margin)

August 21 Obama 44, McCain 45 (McCain +1)
August 22 Obama 44, McCain 45 (McCain +1)
August 23 Obama 47, McCain 45 (Obama +2)
August 24 Obama 44, McCain 45 (McCain +1)

Democratic convention week:
August 25 Obama 47, McCain 42 (Obama +5)
August 26 Obama 44, McCain 45 (McCain +1)
August 27 Obama 53, McCain 39 (Obama +14) post-Hillary speech
August 28 Obama 50, McCain 39 (Obama +11) post-Bill Clinton/Joe Biden speeches
August 29 Obama 44, McCain 45 (McCain +1) post-Obama speech / Palin announcement
August 30 Obama 50, McCain 42 (Obama +8)
August 31 Obama 53, McCain 42 (Obama +11)

Republican convention week:
September 1 Obama 47, McCain 42 (Obama +5)
September 2 Obama 47, McCain 45 (Obama +2)
September 3 Obama 53, McCain 39 (Obama +14)
September 4 Obama 44, McCain 48 (McCain +4) post-Palin speech
September 5 Obama 44, McCain 48 (McCain +4) post-McCain speech
September 6 Obama 47, McCain 48 (McCain +1)

For most days, statistical sampling error appears to be sufficient to account for the variation in the margin. The sampling-based expected standard deviation of the margin is 3.2%; for the day-on-day change in margin, 4.5%. The cases where the day-on-day change is more than 2 standard deviations (95% confidence that a change occurred) are indicated in bold. The other changes are not statistically significant and therefore not clearly interpretable.

The two largest shifts in the Gallup data are a 15-point swing after Hillary Clinton’s convention speech, and and 18-point swing after Sarah Palin’s speech. If true (the Gallup organization and Frank Newport know for sure), these are amazing swings. But why? That’s a matter of speculation. Hillary Clinton’s speech was for many people the first indication that the talk of a split Democratic party was overblown. Sarah Palin’s performance ran counter to much of the preceding negative buzz about her family and qualifications and McCain’s ability to vet a candidate.

Needless to say, this is a lot to hang on one polling organization’s data – and on a particular method for deconvolution, however well-justified. The changes themselves may not be lasting. In both cases the responses seem to drift back toward some longer-term baseline. In the case of Palin, we don’t know yet where things will end up.

So check back in after the weekend polls are released.

Postscript: A commenter suggested the possibility of using Gallup weekly averages to constrain the problem further. However, as I pointed out to him, the roundoff in these values means that they cannot be used in a straightforward manner, i.e. to make a system of N equations. I have tried the constraint that proposed solutions must agree with the weekly averages. This gives bounces that are a few points smaller, but they are the same ones as identified above.

Tags: 2008 Election

8 Comments so far ↓

  • Kenneth W. Regan

    18 points!?! The real question is how much of this comes down to the effectiveness of snarky put-downs, even false ones? GOP strategy since Atwater has seemed to me truly predicated on bucking up people’s self-esteem by giving them others to belittle, and by demagoguing strength. How much research has been done on this? Well, they put so much money and effort into the ugliness, so they must have done the market research!

    As for me, I found the delivery of Palin’s speech excellent, but the content and tone abhorrent—see e.g.

  • James David Walley

    I would note that this contrasts fairly heavily with Nate Silver’s simultaneous analysis over at 538,com, which concludes that McCain won Friday by 2.6 and Saturday by 6.3. His post on the subject is probably worth examining.

  • Ben Ross

    I think (this is intuition – I haven’t tried to work out the math) that if you start from a single starting point, errors are going to accumulate as you go forward in time and you are going to estimate day-to-day swings that are bigger than they really are.

    I think that the better way to do this would be to use something resembling a smoothed window. Do the calculation iteratively with each day as a starting point, and then average them.

    By the way, the tracking polls would give better results if, instead of using a three-day average, they used a tailed window. They might weight most recent day 100%, previous day 80%, second previous day 40%, next previous day 20%, and day before that 10%. (In continuous time, the math calls for a Gaussian curve, but in practice I’d think a very rough approximation would do fine.) What they’re doing now is applying a time window that is a square wave. The Fourier transform of a square wave has visible fluctuations, which you can see in the polling data.

  • Sam Wang

    Ben, your intuition is not right because the method minimizes overall variance and therefore gives equal emphasis to late and early errors. Runs of the procedure on simulated data give good recovery of multiple bounces of about 6 points or larger. Smaller than that and we start getting into sampling error.

    I agree that a tailed window would be better.

    James David Walley, I have the same information he does. My code finds the true global optimum. He says that he simulates many trial solutions. If these trial solutions are selected at random, which is implicit, this is a haphazard method that can approach but never surpass the method I am using.

  • James David Walley

    I suspect we should get some idea of your accuracy versus Nate’s pretty soon. Since you have McCain losing relative strength (although still holding the lead) over the last three of days, while Nate has him gaining during that time, we’ll see if the next couple of results show McCain’s bounce fading or increasing. I agree that it won’t be a definitive solution, since the polls might have moved in the opposite direction today or may tomorrow, but it might make the picture a bit clearer.

  • Sam Wang

    For those who can’t read MATLAB, my procedure is:

    1) Vary the first one-day poll from 15% below the first three-day average to 15% above it.

    2) Set the second and third one-day poll to make an average that equals the first three-day average.

    3) Calculate all other one-day polls, which are at this point completely constrained.

    4) Find the value of the first one-day poll that minimizes overall variance in one-day polls.

    5) Now hold that first one-day poll constant and vary the second one-day poll. All other polls are constrained by these numbers.

    6) Again minimize the variance with respect to the second one-day poll.

    Each poll is varied in steps of 0.2%. The result is not affected by using larger steps.

  • Nicholas J. Alcock

    Interestingly, the updated data is for national polls. Critically, if the national polls are accurate then state polls will mirror this trend. As yet, no state polls are available.

    Also, is the PEC using RV and LV as identical polling or is the PEC using standard polling criteria? If, PEC is discriminating between RV and LV polling can you please inform us how?

  • Sam Wang

    The criterion is like’s: we use the polling organization’s best attempt to identify likely voters. If registered voter (RV) and likely voter (LV) results are both given, then LV results are used.

    In general, state polls track national polls. State polls usually give a very accurate EV estimate. Before the conventions, it’s because there isn’t that much movement. In a few weeks, there will be many state polls available.

    However, right now’s an unusual time: state polling came to a halt, and there’s a lot of national polling. So conditions call for an adjustment to the state-poll-only result.

    I expect to resume reporting purely state-poll-based results in a week or two. Until then I will be making these national poll-based adjustments.

Leave a Comment