Princeton Election Consortium

A first draft of electoral history. Since 2004

Meta-Margins for control: House D+2.8% Senate R+4.8% Find key elections near you!

Anatomy of a bounce

October 21st, 2012, 8:00am by Sam Wang


In the wake of his improved debate performance, President Obama’s recovery is now apparent. It is most clear when viewed in terms of the Meta-Analysis of state polls. Over the last four days, the Popular Vote Meta-Margin – the amount of swing it would take to create an electoral near-tie – has moved by over 1.0%. Today, the President’s effective lead, using Electoral College mechanisms, is Obama +1.8%. A rapid move like this can continue for a few days as polls catch up with the nation’s mental state.

But why do national polls continue to look so close, and in about half of cases good for Mitt Romney? To answer that, let’s take a look back at the effects of all three debates so far.

This is a graph showing the average national margin during a 4-week period when 50 national polls were taken. (If you wonder why poll aggregation is needed, that fact alone answers it.)

The graph is easier to interpret than most aggregates (compare with RCP and Pollster). To calculate it, I spread out each poll’s data across days when the pollster made contact. For example, the ARG Oct. 11-14 poll is split across four days. To capture the Debate #1 change, I left out a few polls where contacts were equally spread before and after October 4. The gray zone is the 1-sigma confidence interval.

The post-debate-1 correction was a nearly five-point swing. It was complete within one day. This means that the post-debate-day media meltdown could not have caused the swing – though it certainly helped cement perceptions.

What caused this crash? Considering the polarization of voters this year (only a small fraction are persuadable), it seems likely to be caused by a change in morale on both sides: hope among Romney’s supporters and despair among Obama supporters, and a consequent change in whether they meet the criteria for being a “likely voter.” Imagine that an Obama supporter was 60% likely to vote before Debate #1, and then 57% afterward – and vice versa for a Romney supporter. That could fully account for the change.

There could also be some fraction of voters whose minds were changed by suddenly-moderate-Mitt and stumbling-Barack.

Since that time, there’s been a reversal in the direction of change. On October 5, Romney had a narrow lead, about 1.0%. Today, President Obama is back in a razor-thin lead at the national level – about 0.5-1.0%.

Because the race is so close, individual polls will inevitably be all over the place. And news organizations love the outlier data points, like the Gallup poll showing Romney +6%. I find this to be a particularly unattractive trait in media coverage. It was what led me to start the poll meta-analysis in 2004.

But here is something interesting. National polls do not match the state polls – and it is state races that determine the outcome, via the Electoral College. In the Meta-Analysis that Andrew Ferguson and I report on this website, Obama has been ahead all along.

You can see this especially clearly in the Popular Vote Meta-Margin, which I have defined as the amount of voter swing needed to create a tossup as defined by the Electoral College. It is an extremely sensitive measure, precise to within 0.2-0.5%.

Here is what it looks like. I shifted it one day to the left to match the national-poll data. The general pattern is clear: viewed through polls that focus directly on electoral mechanisms, Obama performs 1-2 points better than in national polls.

The likeliest cause of this discrepancy is that in states where it matters most such as Ohio, Nevada, and Colorado, the two candidates are genuinely overperforming/underperforming. Also, as I noted in 2008, winners in non-swing states often outperform polls. So there is some question about the accuracy of likely-voter screens when a local result is very unequal. Which leads us back to state polls as a better measure of the race.

In this graph the Meta-margin appears to be a lagging indicator. However, it’s not an apples-to-apples comparison, since I am showing the calculation as it unfolded, day by day. It can move quickly if more state polls are reported – as has occurred since Debate #2.

I will be interested to see if on Election Day, the national vote and the electoral vote count still show this discrepancy. If both are accurate, President Obama’s re-elect probability is about 90% – but his probability of winning the popular vote is lower, about 70%.

Update: for hobbyists, the MATLAB source code is here.

Tags: 2012 Election · President

231 Comments so far ↓

  • NC Obama Guy

    Have you all seen RAND? It is worrying me greatly… Any comments?

    • Sam Wang

      Not really. The only quantity I track there is shifts-between-candidates, which is interesting to watch. It’s been steadily in a Romneyward direction, but in the gray zone, i.e. small. The election-winner graph is probably no better than any other poll.

    • Olav Grinde

      It will be really interesting to examine the swing in state-level polls over the weekend. I’m holding my breath.

      Dr Wang, one question: Is there information on when the SuperPACs are showing anti-Obama ads in the swing states? I read reports that there was especially heavy saturation around the time of, and immediately following, the debates. If true, I wonder to what extent this might have enhanced the perceived Romney-swing after Debate No. 1, and dampened the Obama-swing after Debate No. 2. With other factors, such as “candidate performance”, being the only ones that are credited…

      The Rand Poll shows almost a 5% disparity in likelihood to vote. If that reflects the general population, it is cause for concern, especially with regards to swing states.

      https://mmicdata.rand.org/alp/?page=election#intention-to-vote

      At this point of the campaign, it seems there is a huge battle to control perception. In fact, I get the impression that we’re seeing a psy-ops campaign rather than a political campaign.

    • Kathy Davis

      Good morning, NC Obama Guy. Please read Nate Silver’s 10/19 NYT article, “Should Obama Concede Florida.” Nate discusses this RAND number.
      Thanks, Mr. Wang. This site keeps me sane; I love seeing those deep blue states and can only hope the lighter blue ones join them.

    • Ram

      In Rand poll, the vote shifts have been towards Romney since Sept 25. It became net negative on Oct 1 and has remained so till today.

    • Ms. Jay Sheckley

      Olav, re ads, this was on twitter yesterday, and who can say the funding isnt from a PAC?

      [from Kenneth P. Vogel @kenvogel ]
      Interesting timing … @MormonOrg – which is barred from politics – buying “I’m A Mormon” ads on major websites weeks before Election Day.

    • wufwugy

      Perhaps RAND is beginning to show a drop off in respondents who have already voted. If that’s the case, it could explain why Obama is dropping and Romney is rising

    • Olav Grinde

      Ms Sheckley, that is interesting about the ad purchase of Mormon.org

  • Michael DiPasquale

    Nice presentation of complex stuff. Thanks.

  • George

    You need to stop “explaining.”. Every time you do, the margin goes opposite. :-). Now back under 2 again – 1.8 something.

  • Reason

    Dr Wang, interesting. But it looks like O went down in the MM again. Should we be concerned? He was over 2% and now he is down 9 EV’s in 2 days. Just trying to grasp how you measure the data. Thanks.

  • Ms. Jay Sheckley

    I’d love to see a graph for Ohio marking the events of the Youngstown soup kitchen debacle. It’s been a hot issue. Curious how local polling reaction contrasts or aligns with both sides’ local news site comments/vitriol.

    Again, thank you for providing this morning’s numbers

  • NC Obama Guy

    Thanks Sam,
    You are always very reassuring. Thanks. I certainly hope your model is correct. :-)

  • bflobillw

    Maybe Professor Wang or someone else can help me (and my friends) here. We find it frustrating that “major” 3rd party candidates like Gary Johnson and Virgil Goode are not included in these polls.

    Has any pollster projected their impact on the battleground states?

    Thank you!

    • Froggy

      bflobillw, PPP has had some polls which surveyed both the two-way race, and the race with additional candidates added.

      VA (10/4-10/7):
      Obama 50, Romney 47
      Obama 48, Romney 44, Johnson 4, Goode 1, Stein 0

      CO (10/16-10/18):
      Obama 50, Romney 47
      Obama 49, Romney 44, Johnson 4

    • Misha

      @bflobillw

      While third party candidates have been known to affect the outcome of a race, only one from this election cycle has a significant enough presence to make any meaningful/measurable impact – Gary Johnson. On top of that, historical data show that in tight races, third party votes decrease as novelty votes return to the top two.

      That aside, only PPP regularly poll Johnson, and more often than not, it is limited to swing states (Co and NM, in particular).

    • Olav Grinde

      In a tight race, is it not likely that Virgil Goode may have a significant impact on the outcome in North Carolina?

    • Ms. Jay Sheckley

      repost to proper sub-thread:
      If you do report on outliers [esp johnson, goode, stein] tell us also your theory if any re rumors of those committed Ron Paul delegates, and the likelihood of EV impact. For some reason, Paul delegates bring to mind those intermittent news stories we used to hear of lost lone Japanese soldiers, hiding for decades in Pacific foliage, surviving on nuts and berries, who were finally, gently captured, shredded uniform, ancient rifle and all to be pictured gaunt, bearded, wild-eyed, unaware that World War II had ended years and years before, still fighting for the Emperor.

    • Froggy

      Olav, it looks like Goode is on the ballot in NC only as a write-in candidate, which means no possibility of a significant impact there.

    • Misha

      Actually @Olav Grinde, Goode would’ve fared better in a more open race. As it is now, his core base demographic constitutes conservatives and libertarian-leaning conservatives, who will be more motivated to vote for Romney.

      The Romney campaign are so confident of victory there (internal polling?), their NC team left the state and are heading for Ohio. It seems Romney is not at all concerned with Goode.

    • Froggy

      @Misha, the latest word is that the Romney campaign is NOT pulling out of NC at all — it was just a head fake to confuse the media and the Obama campaign. http://www.huffingtonpost.com/2012/10/20/romney-north-carolina-campaign-victory-offices_n_1993263.html?ref=topbar

    • Misha

      @Froggy, thanks.
      It seems Mitt was pulling a reverse rope-a-dope on the Dems.

  • Ram

    @NC Obama Guy..

    In RAND polling, Obama was at his peak with 8% lead (50.45 % to 42.45%) on Sept 26. Today, he is up by 1.6% (48.07% to 46.47%). Thus, he has lost just 2.3% but Romney is up by 4%. I assume it is a normal tightening of an horse race. If the remaining undecided (5.5%) break evenly on election day (As Sam has shown in previous elections), he should be fine.

    The race has changed since first debate. It is likely to remain close and probably driven only by fundamentals which point to 1.0-1.5% national margin at best to a bare electoral college win at worst. If Obama had performed as well, in first debate, as he did in the second, it probably would have been a 2.5-3.0 % national margin for Obama.

  • Martin

    Why doesn’t Mr. Wang explain why his MM margin has steadily gone down in the last day and a half??? He seems to say one thing in his write up while his Meta Margin says exactly the opposite! What’s the deal Mr. Wang???

    • wheelers cat

      pardon, its Dr. Wang.

    • Ohio Voter

      One thing to keep in mind with the poll by Gravis showing a tie in Ohio. It’s consistently shown a GOP “House” effect, in fact, the last poll done in Ohio by Gravis was Romney +1 while Obama was pulling down +5 in other polls. Nate Silver goes so far as to call a tie an effective Obama +3 in Gravis.

      The PPP poll is a little off, but again, it’s actually shown less of a Democratic lean than usual. The most recent poll was only 532 LVs, which is about 40% smaller than the other polls PPP has conducted in Ohio.

    • Slightly Skeptical

      If I understand the aggregatation correctly, and barring a new Ohio poll coming today (my impression is not many polls get released on Sundays), Ohio should drop to O+1 since the O+5 PPP poll is over a week old. So I think we can expect the meta-margin to go even lower, at least for one day.

    • Some Body

      Slightly Skeptical – the new PPP poll should already have displaced the old. If I remember correctly, only the most recent poll by a firm would count.

    • Slightly Skeptical

      I thought so too until a few days ago, but someone here pointed out that was wrong. The FAQ says “For the current snapshot, the rule for a given state is to use the last 3 polls, or 1 week’s worth of polls, whichever is greater. […] At present, the same pollster can be used more than once for a given state.” Also, if you look at the Ohio polls in the table (below the graph, too me a while to scroll down and realize it was there!) at Pollster, you’ll see that O+2 can only be the median if the old PPP poll is included.

    • bsk

      Skeptical: I had the same question a few days ago. It seems that “last week” refers to the seven days prior to the most recent poll end-date, not today’s date. The most recent PPP poll was in the field until the 20th, meaning polls with end-dates up to the 13th should still be counted (i.e., including the next PPP poll).

      If that guess is correct, the old PPP poll should drop as soon as any new Ohio polls are released, unless their end-dates are the 20th or earlier.

    • Slightly Skeptical

      I see. I guess that makes some kind of sense: this way the median doesn’t change unless there is new polling in that state. Thanks for the pointer.

  • Ms. Jay Sheckley

    If you do report on outliers [esp johnson, goode, stein] tell us also your theory if any re rumors of those committed Ron Paul delegates, and the likelihood of EV impact. For some reason, Paul delegates bring to mind those intermittent news stories we used to hear of lost lone Japanese soldiers, hiding for decades in Pacific foliage, surviving on nuts and berries, who were finally, gently captured, shredded uniform, ancient rifle and all to be pictured gaunt, bearded, wild-eyed, unaware that World War II had ended years and years before, still fighting for the Emperor.

  • Alfred G. Cuzan

    Sam, are the actual data points, by date, depicted on the figure on the EV estimator available on your site? If not, do you plan to post the data in, say, an excel sheet?

  • Ms. Jay Sheckley

    Martin, Sam- Could we just be looking at jitter?

  • Olav Grinde

    Ms Jay Sheckley, yesterday you asked some questions regarding the impact of various voter suppression and cheating methods.

    It seems there is good reason to be especially concerned for the integrity of the voting and vote counting in Ohio: Tagg Romney and major Romney donors have invested heavily in…guess what…voting machines!

    http://www.opednews.com/articles/Does-the-Romney-Family-Now-by-Bob-Fitrakis-121018-232.html

    http://www.clevelandleader.com/node/19137

    http://www.allvoices.com/contributed-news/13221476-romney-family-buys-voting-machines-through-bain-capital-investment

    http://www.dailykos.com/story/2012/10/20/1147643/-Again-Back-to-Ohio

    The American Presidential Elections have attracted the attention of international monitors. That is surely a reversal of what we are used to, which is concern for the integrity of elections in emerging democracies.

    http://thehill.com/homenews/campaign/263141-international-monitors-at-polling-places-draw-criticism-from-voter-fraud-group

    And what is the history in Ohio? Well, take a look at this (from the top link):

    “In 2004, in the dead of election night, an electronic swing of more than 300,000 votes switched Ohio from the John Kerry column to George W. Bush, giving him a second term. A virtual statistical impossibility, the 6-plus% shift occurred between 12:20 and 2am election night as votes were being tallied by a GOP-controlled information technology firm on servers in a basement in Chattanooga, Tennessee. In defiance of a federal injunction, 56 of Ohio’s 88 counties destroyed all election records, making a recount impossible. Ohio’s governor and secretary of state in 2004 were both Republicans, as are the governors and secretaries of state in nine key swing states this year.”

    • Froggy

      Look, I can’t say for sure what did or did not happen in Ohio in 2004, but I will point out that Bush winning the state 51-49 is exactly what you’d have predicted from taking the median of the polls closest to election day (http://www.realclearpolitics.com/Presidential_04/oh_polls.html).

    • Mark in VA

      Olav, indeed this is cause for serious concern. Combine this with the arrest of a GOP operative destroying thousands of registration forms in VA, and you only get a whiff of the efforts going on from the GOP to ensure this election is either won, or stolen.

    • BillSct

      Ohio uses several different systems including Direct Read Electronic (DRE). All systems have a Voter Verifiable Paper Audit Trail (VVPAT).

      http://www.sos.state.oh.us/SOS/Upload/elections/newsletter/Issue%202_October.pdf

      Scroll down to page 6 to see the sidebar.

      In Connecticut we use paper ballots and electronic counters. It seems to take longer to count the vote then with the old voting machines.

  • JAM

    Why is meta-margin going down if there has been an Obama bounce?

    • Martin

      Exactly!

    • Ohio Voter

      While there is a bounce, there have still been a few more favorable polls for Romney in the last day. Two new polls in Ohio have a tie and Obama +1%, bucking the trend of Obama having some +3-5% in other recent polls, I assume that’s dragging down the MM some.

    • wheelers cat

      Obama got a bounce IN HIS BASE.
      But Romney also got a bounce from his base based on asymmetrical enthusiasm response.

  • wheelers cat

    The consensus between Blumenthal and Silver is that the polls are going crazy. But there is a simple explanation. Enthusiasm is asymmetrical between red and blue phenotypes.
    That is why the debates have had asymmetrical effects on the respective bases.
    Enthusiasm is much more critical to “conservatives”. This is observable in RANDs intent to vote graph.
    https://mmicdata.rand.org/alp/?page=election#intention-to-vote
    Debate2 actually increased “conservative” intent to vote, even though Obama was widely perceived to have won. This is because conservatives/republicans have committed to Romney at this date. Insofar as all the polls seem to be polling the subset of voters with enough enthusiasm to accept a poll, this could be biasing the aggregate as well as the individual polls.
    The variable to watch (imho) is the female vote.
    There are more XX than XY in the electorate, and they also vote at a higher percentage than males.
    If Obama gets the same turnout as he did in 2008 Romney needs 63-65% of the white female vote as well as 63-65% the male vote (dependent on if non-hispanic cauc is 72 or 72% of the electorate). This is because the demographic timer has already started going off.

    • wheelers cat

      pardon, 70 or 72%.

    • Amitabh Lath

      Wheeler’s cat: did you see today’s (Sunday) article in the NY times about google search terms correlating with voting patterns?

      Some interesting findings about voter intent.

      Sorry no link, I’m reading the actual paper.

      PS: Is wheeler’s cat refer to gravitation and general relativity like Schrodinger’s cat was for quantum mechanics?

    • wheelers cat

      Amitabh cher, its both.
      un double entendre.
      did you miss my response to you and ms. jay?
      here it is.

      “no Amitabh cher, you are exactly right.
      Ms. Jay, legend has it that Einstein and Wheeler lived in the same neighborhood, and Dr. John Wheeler’s cat was passionately fond of Albert Einstein’s house.
      So Wheeler’s cat could be found either at Wheeler’s house or Einstein’s house.
      Thus the aphorism, Wheeler’s cat…much luckier than Schrodinger’s cat.
      And hopefully lucky for the reconciliation of loop gravity.”
      http://election.princeton.edu/2012/10/19/welcome-npr-listeners/#comment-18144

    • wheelers cat

      merci mille fois for the link Amitabh.
      that is exactly what i mean about exploiting new media.
      http://campaignstops.blogs.nytimes.com/2012/10/20/googles-crystal-ball/

  • BillSct

    Reason & Martin – Look at the history of the MM, it is inherently spikey.

    The outcome in the electoral college is governed by whether each of the fifty states goes R or D. Most of the States are stable one way or the other. Thus the movement in the MM is mainly affected by the eleven states listed in the right hand margin and most of those states have been stable too. So a few outlier polls in the closest of the close states will make the MM twitch back and forth. And this can happen over the course of a day.

    At one point (and this may still be the case) the final graph for the day only recorded the final result for the day. So you could see a downward or upward tick mid-day that would not even be there at the end of the day.

    The graph is inherently noisy so focus on the general trend of the line, not the soikes.

  • bflobillw

    Thanks, Froggy.

    I actually thought Goode was polling higher than 1% in VA. This suggests he will not be a difference maker.

    However, Johnson’s number that you report (in VA and CO) very much suggest that he could.

  • paul

    I was curious if the 95% confidence interval for the senate midterm was posted in late October 2010? I found the archive that stated the 85% interval, but could find the 95% interval. Thanks.

  • Elizabeth Duvert

    Wheelers cat: what is “GaJo”?

  • JAM

    What about the new NBC poll just out? Tied in LV…O up with RV? So much for O’s momentum???Getting nervous here…HELP

    • Sam Wang

      The whole point of poll aggregation is to stop looking at individual polls. Good luck.

    • AggieMan

      Dont worry about that NBC national Poll.

      1) National Polls are meaningless
      2) Their last poll was the day before the first debate. We already knew Obama lost about 3 points or so from pre-debate time, and that’s the exact number they just showed.

      If ALL of NBC’s polling comes true, Obama is president again!

  • Reason

    So is Ohio now a tossup?

  • Lee Hostettler

    What % of Ohio voters have already voted?? and are those numbers taken into consideration when all these numbers are tossed around…..I believe I have heard the close to 70% of those who have already voted , voted for Obama….and if true what % of those yet to vote would have to vote for Romney for it to flip his way??/ Thanks Dr .Wang for you wisdom….

  • E L

    Dr Wang:
    Each night just before i go to bed I swear I will not look at the national polls the next day. Each morning I take the same oath. Then… about an hour later, I peek at them and start to think, OMG. Do I need an intervention?

    • L. Murray

      You are not alone in this.

    • Ms. Jay Sheckley

      As with a declaratins, I risk being either obvious or wrong. Ahem. But, way I see it, PEC _is_ the national poll, EV-oriented, and with a breakdown.

    • Ms. Jay Sheckley

      Yuck, sorry. Should read:
      As with _all declarations_ , I risk being either obvious or wrong.
      (When I talk with my physicist kid, these seem the only possible results. When I said so, he laughed but agreed. Oy. Here, some people agree with each other! r answer with more info. Much thanks to those who have responded to my comments. Really appreciate it.)

  • Reason

    Chris, thanks. That does not look like that much of a tie, but still too close for comfort. O needs to pull a Mortal Kombat and finish R tomorrow. Hopefully, they will somehow drag the improving economy into the debate tomorrow. As really, our economy does affect foreign policy. Your thoughts on that, Dr. Wang? Also, this site is great time killer. As I am at home recovering.

  • Alan Cobo-Lewis

    Sam

    Great analysis. Can you add confidence bands to the meta-margin?

    Alan

    • Sam Wang

      Not conveniently. I think we’ve been over this. The issue is systematics (pollster variability) and sampling error combined. One must make an assumption about whether pollsters are OK overall. I guess there is some kind of resampling of MM data that might be helpful. It’s a bit of a project…

  • JAM

    Does anyone know if the LV stats include first time registrants? Seemingly, many of the LV filters include voting history which wouldn’t account for first time voters. Any insights?

    • Lee Hostettler

      LVstats do not include first time voters…

    • Froggy

      Here’s Gallup’s likely voter model: http://www.gallup.com/poll/111268/how-gallups-likely-voter-models-work.aspx. As you can see, there are special rules in it for those 18-21, to take into account that they cannot have voted in the last presidential election. So under this scheme at least some first time registrants can be considered likely voters.

    • Ralph

      Some of their questions would put me on the ‘not likely to vote’ list. I haven’t missed a presidential election since 1964…but I don’t have a clue where my polling place is here. My wife does and I guess we’ll find our way there when she gets back Wednesday. We both want to vote against Walsh. She wants to vote against Romney and I want to vote for Obama…not that our vote would matter either way in Illinois.

    • securecare

      Their screen doesn’t apply to Oregon or Washington since we do all mail voting, no poll locations any more.

  • JAM

    Then given the Obama campaigns success in registering new voters, that gives me renewed hope…right?

    • Some Body

      Until you think of another factor going in the opposite direction. Assuming you live in the US, there are better things you can put your time and energy into, in the context of this election campaign, than just feeling neurotic about it.

  • Howie Weiner

    I think the President is in a very solid position. Once again I repeat that there is no way to ascertain in real time how many early votes have been cast or the composition of those early votes. That will be known after the election. The best way to surmise that total is through the aggregate reporting of the polls themselves. They are the only scientific method of doing so. At this point thanks to Dr. Wang we have a precise account of the standings in the state polls with an accurate probability. Barring some jarring unforeseen event, the President has a 90% chance of winning re-election as of today.
    As Kevin Bacon said in “A Few Good Men”, “Those are the facts, and the facts are indisputable.”
    Of course he lost the case (lol) and thus the President can lose the election, but getting upset or excited about one poll does not change the state of the election.
    For Obama supporters all we can do is work as hard as we can over the next two weeks and watch and wait. We will be very pleased with the results.

  • Olav Grinde

    Dr Wang, has the impact of the Presidential Debate been rather limited, especially compared to the first, or are we still waiting for state-level polls that show the effect?

  • Ken Dogson

    Dr. Wang,

    In a fist fight, who wins? You or Nate Silver?

    Or should I rephrase: in an aggregate of all the opinions of who would win, what % of the time would you win?

    • wheelers cat

      lol, its a sword fight not a fist fight.
      “When you are wrestling for possession of a sword, the man with the handle always wins.”
      ― Neal Stephenson, Snow Crash

    • Ms. Jay Sheckley

      I think Silver gets too excited and reaches for the blade not the handle. As heard on the NPR show, poor Nate is overinterested in himself. So when polls need weighting, our neurologist has the advantage.
      Plainly, Mr Wang wins any contest, but is to tasteful to respond.
      Check out PEC’s history and watch this site! :D

  • Anbruch

    Given that the MM has been a lagging rather than leading indicator, I’m wondering how you will make a final assessment about its general reliability. Would sort of result would lead you to question its efficacy? If Romney ends up winning the race by say 2% and a narrow EV, would that lead you to question the predictive value of the summer estimate or even your current prediction?

    If poll aggregation is fast becoming the standard for horse race coverage (sort of like a stock index fund), I do think aggregators like yourself need to think carefully about the data going in. You are presuming that the variation in the data is randomly distributed, but if one side of the political spectrum has nearly infinite resources and if actual voting behavior is affected by the reporting of favorable and unfavorable poll results, it seems inevitable that the side with infinite resources will invest in the production of favorable (i.e., non-randomly distributed) polling results.

    • Olav Grinde

      “…it seems inevitable that the side with infinite resources will invest in the production of favorable (i.e., non-randomly distributed) polling results.”

      The real question is whether, and to what degree, this has already happened.

      Another issue that I believe merits examination is the considerable shift that has happened in media ownership — especially news media ownership.

      Today it seems that media budgets and editorial budgets to a far lesser degree allow for in-depth or investigative reporting. Instead, there is an over-emphasis on the “he said, she said” style of reporting, devoid of true analysis or fact finding, based on the delusional premise that this results in “fair and balanced” reporting. Regrettably, it does nothing of the sort.

    • wheelers cat

      exactly, Anbruch!
      who is to say the underlying structure of carbon-based reality is gaussian?
      That is why I prefer pattern-recognition to DSP lately.

    • Anbruch

      I agree that polling is already being generated only for the political effect, but I do not think the target this year was primarily aggregators but merely to drive news cycles. The effect on the aggregators has only been incidental. But I predict next election the targeting of aggregators will be systematic and the distorting effect will therefore be planned, timed, and more pronounced.

    • MAT

      The question then becomes, if timing and quantity of polling is performed to influence media narrative AND the aggregators (who are gaining more and more influence) are also targeted – then it would appear to me that both sides would start to engage –

      For example, Ras polls on the red side could easily be counterbalanced by PPP, which is already under contract to the Daily Kos on the blue side. At some point, you just wind up with a polling arms race, and as long as it stays roughly even, then Dr. Wang’s methods should continue to work.

      It would be interesting to see a study ultimately done on the effect of the rash of small sample size ‘red’ polls that dumped immediately after the first debate on their effect of media coverage and on the polling aggregators (which also drives media coverage)

    • MAT

      Since Wheelers Cat appears to be a fellow Niel Stephenson fan, I’ll mention that it appears that we are entering the phase of polling bogons and artificial inanity.

      (from Anathem, if other Stephenson fans are straining to get the reference)

    • MAT

      The obvious end result of such an polling arms race (at least IMHO) results in state by state tracking polls conducted every day, vs the hit and miss nature of state polling we have today.

      Then, we can all go back to arguing polling methodology ;-)

    • wheelers cat

      @MAT
      lol!
      Would you say PEC is a non-‘sline website?
      I sure would.
      The commenters here are avouts I think.
      And there is no bulshytt allowed.
      ;)

    • A New Jersey Farmer

      @MAT

      PPP has been polling a little more red this cycle. Perhaps that’s why the lean seems to be more towards Romney on the national polls, and a few state polls.

      As I said before, do not make any life decisions before Tuesday based on these data. I believe we’ll have a clearer idea by then which way the election is moving.

    • theDAWG

      Using the Meta-Margin is something similar to the Cartesian Concents since it’s one step removed from any actual interaction with the real world.

  • Olav Grinde

    * and editorial priorities

  • 538 Refugee

    On one hand I’d like to see the site adopt a ‘real’ forum format because it is painful to try and follow new posts by going back through and looking at post times.

    On the other hand I spend too much time here anyhow.

    But (ran out of hands) there is some good information in the posts….. sigh…

    • Anbruch

      It’s only been a problem the past couple of weeks as the number of comments on each thread has risen above 50. At this point, it’s probably not worth fixing for this cycle, but something to consider for the next election cycle.

  • A Reader

    Color me skeptical.

    The range of results from Dr. Wang’s model show a clearly skewed distribution of results (no bell curve here). If we looked at these modeled results in other measurement contexts, we would be suspicious that this asymmetrical distribution suggested evidence of someone’s “finger on the scales” …. as in fact there was when Dr. Wang predicted a Kerry win in 2004. The outcome-influenced comments of Dr. Wang also undermine objective credibility (odds of Obama win increase as his MM advantage decrease?).

    I suspect this model is substantially understating the possibility of an Obama defeat. It will be interesting to see if there is an acknowledgement of systemic predictative error, as there was when the model significantly underestimated the 2010 House Republican pickup margin.

    • wheelers cat

      Are you familiar at all with Nassim Nicholas Taleb?
      There is an emergent group of the statistically l33t that believe in the asymmetry of carbon-based systems.
      Please try to keep up.
      ;)

    • ChrisD

      “…evidence of someone’s ‘finger on the scales’ …. as in fact there was when Dr. Wang predicted a Kerry win in 2004.”

      Um, looks to me like Dr. Wang predicted a Bush win in 2004.

      http://election.princeton.edu/history-of-the-2004-race/

    • Sam Wang

      Go to the original site to see my error. Polls-alone fix the error.

      No finger on the scale again, ever. It was a strongly administered lesson.

    • Eric Fisher

      I have repeatedly tried, in private, to explain to Prof. Wang that his calculation of the meta-margin is in error. In my own simulations using his code, I was able to verify that it worked for simple cases where the met-margin was between -0.5% and +1.0%. It gives wrong answers otherwise. I am very surprised that a person with Prof. Wang’s professional credentials would continue to pontificate about a computer program with such a simple bug in it. The last three polls in OH have a median of +1% for Obama. Thus a 0.5% shift in voter sentiment in Ohio alone would win the election for Romney. I think this is the actual “meta-margin” now, and I admired Pro. Wang’s honesty in letting me see his code. He also had a very good idea to think of the Meta-Margin, and he has an attractive website.

    • Sam Wang

      Probably user error in implementing the code. The Meta-Margin behaves fine over all ranges that we have tested it.

    • ChrisD

      My apologies, A Reader and Dr. Wang.

    • Billy

      What finger is there on the scales? The median-based approach is non-partisan and completely computer generated. Nate Silver weighs the pollsters but that doesn’t happen here. Also, this site includes *all* polls and treats them equally.

    • theDAWG

      A Reader, the code is freely available.
      http://election.princeton.edu/methods/
      Grab it and run it yourself. No monkey business here.

  • Steven J. Wangsness

    Oh, man. The MM is under 2 again, after just breaking the +2 barrier the other day. Meanwhile, Obama is up to + 0.2 on RCP, up 0.1 from yesterday. Then over at 538, nothing’s changed for two days.

    I can’t take it anymore. This election needs to be over. Now.

    • Ms. Jay Sheckley

      Steven, relax. nothing new is going on. In electoral votes, the President has been on top _the whole campaign, ok?

      Don’t read the other things, just come here, maybe just read the top predictive line and forget the M-M. Fluctuation within a day or two is likely largely just poll jitter -maybe new info from disagreeing sources- but we’re seeing a steady upward correction which began before the VP debate.

      If you want to, you can take time off from poll watching and I can write you and let you know if the predicted outcome changes :D

      Something you can do to relieve stress is the email equivalent of door-knocking. Write to the people on your email list and ask them to promise to vote. Don’t disturb the ones you’re pretty sure oppose your candidate.

      Just come here. Relax.

      Or, conversely, be sure and listen to every crazy poll, especially Gallup! By all means, watch the debate! I’ll be doing that, god nose why. The debates, the cocksure pundits who follow and the over-emphasis on kennelsfull of undecideds is for me the very trefecta of freak out.

      No, no. Instead, play with the maps here, why not? Theyre great. Just come here, read the electoral prediction, not the MM.
      Gonna be ok.

      Sam, please please any corrections or additions.

  • Dean

    Wow, the national polls released today via RCP show quite a disparity. Gallup has Romney up by 7, and IBD/TIPP has Obama up by 6. I see also that the RCP average of Ohio shows a tightening race.

    The RCP Virginia average shows a tie, but the state is red in the “No Toss Up” category (bias?). If Virginia goes to Romney and Ohio is tightening, then Obama stands a very good chance of losing this election. If Romney gets Ohio and every other state in which he currently leads, it’s over for Obama. Why are the models still so favorable toward Obama in this scenario?

    • Slightly Skeptical

      The short answer is that Romney needs _both_ Ohio and Virginia, whereas Obama just needs one of them. The long answer would go into probabilities, standard deviations, and correlations.

      Still, the election is two weeks away, so if Romney gets makes gains he could easily win (with Ohio). But the way it is today it’d be hard to imagine him winning unless there is a serious systematic error in all the Ohio polling.

      Comparing Sam and Nate’s numbers and methods, I think the poll aggregation part is where they agree most and is soundest, whereas the predictions disagree more and seem like more of a stretch. So I’d say to just keep your eye on the metamargin as the election gets closer.

    • Brian

      For some odd reason, RCP isn’t including 2 Virginia polls from PPP that were released on 10/16 and 10/19 that show the VA race as O+1 and O+2.

    • xian

      RCP is definitely selective about when it adds and drops polls. Is it not owned by Steve Forbes?

    • Ms. Jay Sheckley

      Slightly Skeptical writes: “The short answer is that Romney needs _both_ Ohio and Virginia, whereas Obama just needs one of them. ”

      Well said!

      Dean, you write: “If Romney gets Ohio and every other state in which he currently leads, it’s over for Obama.”

      Every _other_ state? Mitt’s not leading in Ohio, Obama leads by 1%. That could reverse, sure, but so could the two states Mitt leads in by the same shallow margin.

      You write: “Why are the models still so favorable toward Obama in this scenario?”

      Because Obama is clearly winning.

      Gallup will always make headlines- it’s exciting rather than reasonable.

      I’m no PEC spokesman, but the purpose of this poll aggregate seems to be to turn polls into a reasonable view of what is likely to happen at the electoral college.

      Just come here.

  • Ram

    I can’t make believe today’s IBD/TIPP poll showing O: +6. This is for Oct 15-20 period. Probably an outlier like Gallup’s recent Llikely Voter polling.

  • Iseeurfuture

    Dr. Wang,
    I’ve featured your story today over at Daily Kos.
    http://www.dailykos.com/story/2012/10/21/1147898/-Anatomy-of-a-bounce

    I hope it is ok :)

  • Stuart

    Anyone worried about the election should be making calls using obama’s dashboard.

    And we all should be worried at this point!

    https://dashboard.barackobama.com/

  • skmind

    Somehow this does not feel right. I am not questioning the good doctor’s (or Nate Silver’s) analysis, just a feeling that this is overly rosy, given how much the race appears to have tightened in Ohio, and New Hampshire since the VEEP debate, and since Obama’s win in the second debate.

    I am afraid that the effect of Rove-led SuperPAC monies are being underestimated.

    At this point I fear that Biden’s arrest of the free fall, and a bounce even, has been undone by Obama’s debate performance good as it was.

    • wheelers cat

      /sigh.
      I think we can all agree that the poll shops are only sampling the subset of the electorate that will consent to to take a poll. So if the stratification adjustment is wrong, like Gallup’s assuredly is because they have % white at 79%, then the polls are wrong.

    • Reason

      So Obama winning the second debate means he made things worse? Really? And most polls are not showing that.

    • Anbruch

      So is Gallup not making a demographic correction or are they estimating the white turnout at 79%?

    • Howie Weiner

      The effect of the super pac money and everything else will show up in the polls. When all is said and done we only have the polls as scientific evidence. All the rest is “feeling,” “supposition,” our own biases. If the polls are inaccurate or biased or anything else, then the forecast, predictions, and state of the race will be off. We will know the day after Election Day how accurate the polls were. And that will be based on the closest poll to Election Day itself. As somebody else once said, “all the rest is commentary.” Right now the polls show a very narrow Obama victory. The only question is how precise they are. We shall see. Meanwhile let’s all hope the President gives as strong a performance tomorrow night as he did at Hofstra.

    • wheelers cat

      @Reason.
      no he didnt make things worse.

      @Anbruch
      from pollster.

      But beyond just the impact of Gallup’s likely voter screen, there is another factor that may explain why Gallup is once again a big outlier. Despite some recent changes in its sampling procedures designed to make its initial sample of adults more representative of the U.S. adult population, Gallup’s likely voter sample appears to be substantially under-representing non-white voters. Although Gallup does not report the racial composition of its likely voter sample (or any of its other samples), based on the results presented in their October 16 report on the standing of the presidential candidates among whites and non-whites, one can use interpolation to estimate the racial composition of the likely voter sample. The results show that about 80 percent of Gallup’s likely voter sample consisted of non-Hispanic whites while about 20 percent consisted of non-whites.

      Gallup’s estimate that only 20 percent of this year’s likely voters are non-white is far lower than the 26 percent non-white share of voters found in the 2008 exit poll or even the 23 percent share found in the 2004 exit poll. It is actually very close to the 19 percent share found in the 2000 exit poll. So according to the Gallup tracking poll, the racial composition of the 2012 electorate will be similar to that of the 2000 electorate despite the dramatic increase in the nonwhite share of the voting age population that has occurred in the past 12 years.

    • Olav Grinde

      “I am afraid that the effect of Rove-led SuperPAC monies are being underestimated.”

      SkMind, as far as I know there is a very close correlation between the amount of money spent and victory in an election.
      This is one reason why I’m very worried about the intensification of TV commercials by Romney and anti-Obama SuperPACs during this final stretch.
      The other reason I’m worried, is that the Romney-Ryan Campaign — and the candidates themselves — have made it very clear that their message will not be dictated by fact-checkers.

      I don’t have a television, so I’m missing out on that — but however much people are fed up or might believe they are immune, many will be influenced.

  • wheelers cat

    And IPOF, here is a conservative dimbo putting all her eggs in the gallup basket.
    http://www.nydailynews.com/blogs/the_rumble/2012/10/obamas-struggles-with-white-voters-racism-has-nothing-to-do-with-it

  • Doubtful

    @skmind…

    I have cut two checks for O campaign. But I am a little doubtful of both Wang and Nate’s models. If R holds the states (NC, FL, VA & NH) where he currently leads or even, O has no margin of error. The point is if OH/WI/NH moved to R by 3-5%, there is NO reason to believe, he can’t move another couple of points with all the money that is available to him.

    • theDAWG

      Some reassurance for you, Doubtful:

      It’s not clear Romney leads in VA for a 5 person race, like what’s going to appear on the actual ballots.

      It might be too late for Romney in Ohio once you consider the early voting.

      The polls do not take into account what I call the “Palmyra Effect”: Are evangelicals really going to turn the lever for an LDS candidates? Given that evangelicals are ~40% of the registered GOP base, even a 5% desertion rate could be a gamechanger.

      These are not excuses for slacking off, though.

  • skmind

    @Reason

    “So Obama winning the second debate means he made things worse? Really? And most polls are not showing that.”

    No, I realize I did not state that quite correctly. Obama did better in the second debate, but I am not convinced that he did way better than Romney, and any bounce that he had *feels* like it is gone already, if it was indeed *his* bounce and not caused by Biden’s under-rated performance.

    @Howie Note that I did not challenge Dr. Wang’s or Nate Silver analysis, all I said is that it does not *feel* as rosy for Obama as those projections.

    Ohio is way tighter than it looked post Biden’s performance. In the last few days the women voter’s edge that had closed after Debate #1 and had opened up again has closed yet again and rather quickly for supporters of Obama.

    The SuperPAC money I think is the main reason for that, and while you are correct that it is reflected in the polls, the huge ad buys of late are being ignored I think and the ‘debate performance’ given too much weight as poll-movers.

    • Howie Weiner

      @skmind—I agree with you. Romney buried Gingrich and Santorium under an avalanche of campaign money. He is a smart businessman and has Rove an expert political operative at his side. They reserved a considerable sum for these last two weeks and are unleashing the torrent now. All I said was that the effect of it will show up in the polls. Obama’s bounce after the 2nd debate is small but significant (he has the lead). His campaign is not without significant resources either and Axelrod and Plouffe are no slouches at this game. They can counter particularly in Ohio and Wisconsin. If they win both, the game is over for Romney. Iowa and Nevada will go for the President, I don’t care how much money Romney spends in those states. And that’s all we need.

    • wheelers cat

      so you cudlips are all going to just masticate whatever gallup and rasmussen are feeding you?
      maybe the statistics nerd core is just my fantasy.
      maybe there is no asymmetry after all.

    • Corrupted

      Ad buys are not really getting ignored actually. I’ve read several articles of late that talk about them. While Romney is spending more, he’s actually showing fewer ads than Obama because Obama bought earlier and isn’t relying as heavily on SuperPacs (who have to pay inflated prices).

  • wheelers cat

    Not only has Nate recently learned the importance of nonparametric statistics, but he is now cognizant of the gender gap.
    http://fivethirtyeight.blogs.nytimes.com/2012/10/21/gender-gap-near-historic-highs/#postComment

  • Ms. Jay Sheckley

    These charts show I was right to avoid all election news but Sam’s. Those days were hard enough! What I saw on the State-by-State unpleasantly messed with my sense of reality. Maybe it should have said “currently” above where state were called “safe”. Maybe there should be a darker color yet….
    And yet, PEC spared me the popular margin charts seen here above. I do love info, but that wasn’t relevant to the outcome… I had no idea I didnt know!
    Very grateful.
    PS re early voting, I suppose a lot of you can deal with http://elections.gmu.edu/early_vote_2012.html
    So much Ohio data there! Too much for me. I’d love to hear what others make of it. With the GOP claiming early voting “is unAmerican”, (though Ryan just told Buckeyes to use it), and with Ohio voting taking up to 12 hours without early voting, wouldn’t a lot of voters be angry with the GOP for trying to cause em all that grief?
    To me it smacks of bad intent and if I were there I’d want to vote that party out. I picture swarms of resentful soccer moms.
    Sam? Anyone? Has there been punditry or far better yet polls on this? On this, I’m even interested in personal opinion

  • ML

    Dr. Wang (or any of the other wonderful regulars here),

    I would love to see your thoughts on this piece by Nate Cohn over at The New Republic:

    http://www.tnr.com/blog/electionate/108927/the-five-things-you-need-know-about-obamas-lead-in-ohio-polls

    I thought the most interesting points were the ones about the seemingly-significant differences between Ohio surveys that include cell phones vs those that do not, and the point that most of the companies that gave Pres. Obama his best #s in Ohio before the debate have not gone back and re-polled since then.

  • Reason

    FYI for all the other stalkers of this site: PPP new poll out for Iowa. O back on top, 49/48. Leads heavily with women, young voters and early voting.

    https://twitter.com/ppppolls

    • Michael Worley

      Well, that is nominally good, but the MM will be ironically lower because the last PPP poll will stay in. so if IA goes to +1, we could see the MM drop, and a OH poll that is +1 or less could put the MM back at +1.

    • Arbitol

      Must’ve caused abetter IA poll to roll off though, judging from the MM.

    • Michael Worley

      “good”= pro-Obama, which isn’t my actual position.

    • Froggy

      The drop in the MM at 8pm was caused by a new Ohio poll by Angus Reid, a tie that dropped OH to O+1. It’s an online poll of people who sign up at a website to take surveys — hardly a random sample. But here at PEC, all surveys count the same.

      I’m not sure how the model treats the new IA PPP O+1. Since the polling dates overlap with the earlier IA PPP R+1, I think the earlier one gets kicked out, which would move IA from O+1.5 to O+2. If both say in, IA drops from O+1.5 to O+1. Not a big change either way.

    • Michael Worley

      Re: IA why would a R+1 to a O+1 move the median from O+1.5 to O+2?

    • ChrisD

      Michael: Click on the IA row in “The Power of Your Vote” table in the right-hand column to see Pollster’s underlying polls. From latest to earliest, they are:

      O+1 PPP 10/18-19 (NEW)
      R+1 PPP 10/17-19
      O+8
      O+3
      tie

      Replacing the old PPP poll with the new PPP poll changes the median of four polls in the last week from AVERAGE(O+3, tie) = O+1.5 to AVERAGE(O+3, O+1) = O+2. Odd as it seems, that’s the way it works.

  • Renee

    For all the worry warts who can’t stop looking at the polls…go out and be productive

    NBC/WSJ: Obama leads by 5 among registered voters. But pollsters say Republicans more likely to vote. Romney only wins if Dems don’t show up

    • L. Murray

      We do have to make sure we close the so-called enthusiasm gap. Oddly it seems to me that the more that people believe their candidate can win the more likely they are to vote; it’s like they want to be part of the victory.

    • Lisa

      Because it”s TRUE!

    • Howie Weiner

      @L. Murry. “Everyone loves a winner, when you cry you cry alone.” An old sentiment but oh so true. This is how waves of enthusiasm are built. This election has an amazing amount of money flooding into its process, a war is being fought on many fronts, why would we think that polling firms are any different. As we can see a positive poll or especially a series of positive polls is a piece of propaganda that can be utilized to influence potential voters. I’m sure this has been well thought out and now some polling firms are essentially tools of the campaign. I’m certain if we could look at the funding of polling companies, this would become clear. My background is in leukemia clinical trials. If these trials were run like some polling companies, the results would certainly be suspect. However, having said that, the polls are in a sense our laboratory they are the place where the roiling cauldron of the electorate can be most closely examined. Dr. Wang has certainly demonstrated that even with this interference in the scientific process of polling the end result actually reflects the reality of the process. The race is tightening around the data point of its climax. The President is in the drivers seat for the moment, let’s hope it continues.

    • Lisa

      It will, if we catch a disease called enthusiasm,belief and ACTION!

    • wheelers cat

      Forgive me Lisa but I disagree. Conservatives have an organic basis to be motivated via enthusiasm. Its baked into their phenotype.
      Liberals otoh have an organic bias to be motivated by logic and reason. That is why it should be possible to just explain that if liberals turnout, we will win.
      While there is apparently a near perfect correlation in organic conservatives (R^2=1) between enthusiasm and likelihood of voting, im unsure if there is that same perfect correlation in liberals.
      Since polls are only sampling the [probably enthused] subset of the electorate willing to take a poll and guestimating the non-responders, we wont know until the election is over if my hypoth is correct.
      ;)

    • Ms. Jay Sheckley

      L Murray writes: “seems to me that the more that people believe their candidate can win the more likely they are to vote”.

      That’s the consensus. Though my aggregate impression is that because disappointed GOP voters are spoken of angry and disappointed DEMs are more despondent, the disappointed GOP voter’s mood provides more energy to get the polls, which increases voter likelihood.

      “It’s like they want to be part of the victory.”
      Yup. and some of us dont want the blame of defeat. VOTE! And make sure the people you know will do so regardless.

  • Renee

    @EJDionne
    NBC/WSJ: Likely voters Obama, Romney tied at 47. Registered voters Obama 49-44. Shows how much organization will matter

    http://firstread.nbcnews.com/_news/2012/10/21/14599504-nbcwsj-poll-presidential-contest-now-tied?lite

  • Olav Grinde

    I’ll have significantly more peace of mind when President Obama flips some of these states: New Hampshire, Florida, Virginia and/or North Carolina.

    Must confess that I do not feel at all comfortable with Obama having to wind Ohio.

Leave a Comment