Princeton Election Consortium

Ipsos/Reuters rips off the Band-Aid

—

by

Sam Wang

In today’s 50-state release of data by Ipsos/Reuters, some have commented on the fact that some individual state results are not convincing. I take a different view: having so many data points at once is a gift. This helps the state poll snapshot reach its equilibrium more quickly. Even if individual-state subsamples have middling accuracy, as long as the overall bias is small the data set moves the Meta-Margin towards a value that incorporates recent opinion. I don’t know if it’s at equilibrium now. I do note that the national poll median is currently Clinton +1.5%, close to the Meta-Margin.

For us, the large error bars for the Ipsos polls are acceptable because we aggregate them into a single snapshot, which effectively reduces the overall error. For example, if each poll error is +/-10% and 20 states are competitive enough for the polls to be useful, then the aggregated error is +/-10/sqrt(20), or +/-2.2%. Think of each poll as being a mystery body part; what PEC provides is the head cheese.

Comments

88 responses to “Ipsos/Reuters rips off the Band-Aid”

  1. A New Jersey Farmer Avatar
    A New Jersey Farmer

    And the data also shows that despite almost two brutal weeks, Hillary is still ahead. Cake anyone? It’s baked pretty well.

    Reply
    1. BF Avatar
      BF

      Yes, but they are both now in the same boat in terms of paths to 270. At one point, Hillary had a dozen ways to clinch, while Trump had only one narrow path. Now, starting for this point, Trump only had to outperform in PA or WI or MI or CO to win. He was trying to pull to an inside straight, whereas now he’s got multiple hands in play.
      Hopefully this is a low point for Hillary, but if we go into election night with the current situation, it could easily get dicey for all sorts of idiosyncratic reasons.

      Reply
      1. DebbieR Avatar
        DebbieR

        In response to BF at 9:03
        That is a sobering thought.
        Clinton still has the advantage regarding the path to 270 I believe. However, no denying this is currently too close for comfort.
        If we were to go into election night with the current polling situation, I think I would check-out for the sake of my health and wellbeing, only checking back in when its all over.

        Reply
        1. Brian Avatar
          Brian

          Do not tempt fate.

          Reply
  2. Mark F. Avatar
    Mark F.

    A 71% to 81% win probability should have Clinton supporters concerned. At least a little.

    Reply
    1. Seth Gordon Avatar
      Seth Gordon

      An 81% win probability is still ten or fifteen points lower than I’d like it to be. I mean, if someone gave you five Skittles, and you expected one of them to be poisoned…

      Reply
      1. Kevin King Avatar
        Kevin King

        LOL, in my opinion you win this comment thread, Seth! 🙂

        Reply
  3. Phoenix Woman Avatar
    Phoenix Woman

    By the way, Monmouth, for the second straight poll, has Hillary up in Florida:
    http://www.dailykos.com/stories/2016/9/20/1572314/-GOOD-NEWS-Hillary-up-by-5-in-Florida

    Reply
  4. Alex Avatar
    Alex

    Things are basically converging to 2012 except that HRC is over-performing Romney in most red states and DJT over-performing Obama in most blue ones.

    Reply
  5. calvinhobbesliker Avatar
    calvinhobbesliker

    How far back are polls included? I though it was just 1 week, but your Florida median seems to include the CNN poll that was in the field 8 days ago at the latest.

    Reply
  6. George Avatar
    George

    Sam, your statement about reaching “equilibrium more quickly” assumes that those data points bear enough connection to reality so as to reach a “real” equilibrium as opposed to a disconnected from reality equilibrium. I posit that the data points are too flawed to reasonably do what you argue they are doing. The flaws as I see it are at least two-fold: First, the data is “smeared” over an extended period of time – in some cases up to three weeks. Second, the total numbers (sometimes as low as <100) are so small as to have 95% confidence bands of close to +/- 10%. And to compound that, those low numbers are spread out over that extended period. If in fact our politics was like a true random number generator, then I’d think you’d have to be suspicious of the kind of variation that we have actually seen. And since I think even you acknowledged that things seemed to be “happening” – that takes us out of the random field and into a field where there are in fact forces acting on the data points (polled voters). And if that is true then the small numbers and long time frames again mess up the clarity of what we are trying to see. Since science is about falsifiability- let me make a prediction: That as these Ipsos numbers move down and out of the data chain the meta-margin will move back to or past the 2.8 it was at before the Ipsos dump. If my prediction is correct, then that means that some combination of a truly moving target AND the “GIGO” principal has been at work. If the meta-margin does not move – and stays at the 1.7 level +/- a few tenths, then your hypothesis of reaching “equilibrium faster will be more correct. Let the experiment run. 🙂

    Reply
    1. Amitabh Lath Avatar
      Amitabh Lath

      So it’s bad data and it’s skewing the results, but statistically insignificant so not affecting the results at all?
      I see your bigger point but I do not believe there is any way to weed out “bad data” without introducing unacceptable bias.

      Reply
      1. George Avatar
        George

        Amitabh, the assumption of the meta-margin is that there is in fact a real difference between the two candidates. If the data is so bad that it is essentially random (and if you look at some of the state oddities, that is not a bad argument), then what you have done is dumped a large quantity of “0.0 meta-margin” data into a pool of “2.8 meta-margin data,” thus diluting it. It doesn’t have to favor Trump, it just has to be so unreliable (“bad”) that it nets out to 0.0 or something close to that.

        Reply
        1. Amitabh Lath Avatar
          Amitabh Lath

          Maybe the 0.0 MM data is correct. Maybe everyone else is wrong, and this data is bringing us closer to truth.
          What metric are you using to decide this particular data is of low quality? That it does not agree with previous data? That it integrates over a larger time window than other pollsters?
          I would be afraid to pass any judgement on a dataset once I’ve looked at its resultant value, because I would almost certainly be cutting on my political prejudices.

          Reply
          1. George Avatar
            George

            Amitabh, I could cut and paste I suppose, but I think Electoral-Vote.com summarized it nicely in their explanation of why they dropped the Ipsos data – as they noted, it has too much variation from all of the other polls averaged together. So, either their internet polling is better than all established polling methods, or they still have lots of random error. As I noted earlier, meta-margin movement over the next week or so (or however long it takes to digest and eliminate the Ipsos influence) will tell the tale. I think the problem is that they are going to be doing these massive dumps every week or so, which may continue to whip-saw things.

            Reply
            1. Amitabh Lath Avatar
              Amitabh Lath

              The lowest point in my undergrad life was having to redo a lab because I dropped a couple of data points simply because they varied too much from all the others averaged together.
              The guy who scolded me for playing fast and loose later won the 1990 Nobel Prize in Physics.
              The only way to get rid of “bad” data is to accumulate more data to make it irrelevant. Any post-facto selection criteria, however justified it may seem, is going to be biased.
              Would the Ipsos Reuters data be examined as carefully and be ripe for discarding if it had said Clinton +5? I think not.

              Reply
              1. Jeremiah Avatar
                Jeremiah

                @Amitabh I don’t think they should get rid of the points because they are “bad.” I think they should get rid of them because the authors themselves have low confidence in the numbers and the samples are so small. Both factors mean that PEC is weighting the Ipsos results more heavily than those from other polls/pollsters.
                Another factor is the results won’t get “averaged” (medianed out?) out by other results unless there was a lot of poor quality polls like this one put into the mix.

                Reply
                1. Amitabh Lath Avatar
                  Amitabh Lath

                  Jeremiah, I do not understand your assertion. The authors of the data are just as likely (perhaps more) to be biased. They look at their data, see the potential hit to their credibility, and start fudging. Unless they come out with a statement of actual error (something egregious like “this data is wrong, we were actually calling Canada…”) we simply cannot discard data once we have examined it.
                  Also, what I meant by averaged out (you are correct, in PEC it would be medianed out, if that is a word) is that only more data will determine if the Ipsos/Reuters data is actually an outlier, or maybe an early indication of a race that is close to being tied. Since we do not know, throwing out the data would be wrong. Always, always wrong (barring any indication of data acquisition malfunctions).

                  Reply
                  1. Jeremiah Avatar
                    Jeremiah

                    Unless the authors explicitly tell us what their confidence level means we will not know if their data can be trusted. I am assuming that a low confidence means that because of the small sample size and/or demographics sampled they cannot represent the population with any confidence.
                    I think then that they are not being as careful as other polling organizations with this data release. Also, it is internet only and I strongly suspect that it is very difficult to get a truly random and/or representative sample doing that.

  7. Rex Avatar
    Rex

    *Gulps heavily*
    As much as I enjoy refreshing my browser 47 times a day on this blog, I will be much happier when this election is over(I hope).

    Reply
  8. A Avatar
    A

    Sam,
    I just went back to the 2012 archives. Looks like in October of 2012 after the 2nd or 3rd debates, there had been a lot of fluctuation, with the meta margin at +4 for Obama going as much as to +1 for Romney.
    Your SD was set in the 2% range so there seems to have been less volatility with the prediction range, where you had Obama with about a 90% confidence to win, and it was fairly stable.
    But if I’m tracking it remotely correctly, then it seems to me that the movement we’ve seen after the post convention bounce wore off in 2016, is that Clinton’s lead is fairly stable and probably not that much different from Obama’s.
    Thus, we can likely see some big movement around the debates, but after that “bounce” wears off, we’ll probably come back to the original state, which would be Clinton at +2 or +3 ish…
    It’s difficult not to get caught up in all of the movement that comes from these big events, but it feels like whenever the events wear off we revert back to the mean, and back to the polarization of which you’ve spoken time and again.

    Reply
  9. Ken Schulz Avatar
    Ken Schulz

    Noise-chasing. Odd that they color Vermont red, New Mexico pink, and Rhode Island and Alaska gray, while the table shows embarrassed dashes; presumably indicating insufficient data. Anybody willing to bet that Trump could win any of the first three, or that Clinton has a chance in AK?

    Reply
    1. Matt McIrvin Avatar
      Matt McIrvin

      It looks as if most of these surveys have sampling sizes in the 500-700 range, which means their 95% confidence margin of error just from pure sampling noise is around 4-5%; some would be larger. And there are 50 polls here, so we’d expect about 2 or 3 of them to be outside even that range by pure chance. Given that, it’s not surprising that there’s some weirdness in the results even if we discount systematic error.

      Reply
  10. Mark J Avatar
    Mark J

    Enjoy the blog, Mr. Wang. The article by Nate Cohn today on how five different pollsters (including his group) came up with a 5-point spread using the same raw data was one of the most important ones I’ve ever read on polling. The fact that it confirmed my suspicions didn’t hurt.
    All data is biased and the analysis of it as well. Early polling in the cycle is liberated from any kind of real check except for herding. A lot of Trump’s recent momentum is really just the polls herding more as we near Election Day.
    Trump will win. The signals are everywhere — voter registration shifts, primary totals, crowd sizes, media panic, the zeitgeist, certain subtle data points, early ballots — but the polls are slow to catch up.
    Hope that doesn’t trigger anyone.

    Reply
    1. Bob Smith Avatar
      Bob Smith

      You’re basing your Trump victory prediction on rally sizes, the zeitgeist, and subtle (but unnamed) data points? Not exactly scientific. Indeed, hardly more scientific than “I’ve got a feeling deep down somewhere but I can’t explain it.”
      The other “signals” don’t really prove anything. To the extent that they even qualify as signals, they were there a month ago when there was virtually no chance of Trump winning.

      Reply
      1. Matt McIrvin Avatar
        Matt McIrvin

        This kind of analysis was why so many people were convinced that Mitt Romney was headed for a win on Election Day 2012, including Romney and his own campaign team.
        Trump can definitely pull this out. But we haven’t even gotten to the debates yet.

        Reply
        1. Kevin King Avatar
          Kevin King

          It did trigger me. To laughter. Not so much the prediction itself but the reasons why and the trolling tone. Yeah, Trump could win, but it’s unlikely. If he does it’s because a significant part of the Democratic coalition votes for minor parties or doesn’t vote. Absent that, Clinton wins.

          Reply
          1. Chillax Avatar
            Chillax

            If “all data is biased and the analysis of it as well”, then why should we trust your “data” (if we generously consider your “signals” to actually be data) or your analysis?

            Reply
            1. Marcia Avatar
              Marcia

              As soon as someone mentions rally sizes as a signal, I know they aren’t being objective. If the ability to get a bunch of people into a concert hall were a defining factor, Beyonce would be our next president.

              Reply
              1. Matt McIrvin Avatar
                Matt McIrvin

                I won’t even say a Trump win is “unlikely” at this point; we’re back in Russian-roulette territory at best. But watching the zeitgeist is a really bad way to call these things. It was what got Kerry partisans convinced they were winning in ’04 as well.

                Reply
                1. Nancy Avatar
                  Nancy

                  Reminds me of a lot of the arguments people started making at this point last time … for Romney

                  Reply
                  1. Sam Wang Avatar
                    Sam Wang

                    Ro-mentum is back!!!

                  2. Seth Gordon Avatar
                    Seth Gordon

                    In all seriousness, it would be interesting to see, along with the graph of EV/PV estimates and their confidence range, a “momentum” metric: something that measures the likelihood that poll movement in the last week or month can be attributed to something other than statistical noise.

                  3. DK Avatar
                    DK

                    Weren’t crowd size and the zeitgeist among the many reasons Bernie was destined to win the primary over Hillary? *snicker*
                    The latest poll of NC out today shows Hillary +2, a virtual tie. Hillary has 31 offices all over NC to get out the vote, where early voting has already started. Trump has zero offices open in NC.
                    Zeitgeist ain’t gonna help Trump avoid his coming loss.

  11. Josh Avatar
    Josh

    I’ve been playing around with the Ipsos results a bit, and I’m a little confused about what they’re actually saying. The article noting the tight state of the race today links to their “States of the Nation” polling, which seems to be making predictions according to Ipsos’ model of voter turnout. And their model makes some interesting assumptions.
    They predict 68% turnout of white voters, which would be just a hair above 2008 and 2012. Makes sense. What I don’t understand is why their model predicts just 54% of African Americans are going to vote after turnouts of 56.8, 60, 64.7 and 66.2 percent between 2000 and 2012. I get that Obama’s no longer running but that would be a decline of over 20%.
    Ipsos also predicts 33% turnout from Hispanic voters, which would be 12-15% lower than any election since 1996.
    I apologize if I am badly misreading Census data or seriously misunderstanding Ipsos’ methodology. I am just thoroughly confused about what appear to be significant predicted drops in minority participation. Their methodology section references self-reported likelihood to vote as an included piece of their calculations…but even so, this would be a heck of a drop.

    Reply
    1. Ken Schulz Avatar
      Ken Schulz

      It looks as though polling firms have told themselves a story about 2008 and 2012, that Obama, the first African-American major-party candidate/President, attracted a larger turnout of minority voters, who vote heavily Democratic, and got a higher percentage of their votes as well. With two white candidates, they expect turnout to drop among voters of color.
      Remember Nate Cohn’s article about the demographics of the 2012 electorate? Based on the Current Polulation Survey and voter files, there were more older, white, working-class voters than was previously thought based on exit polls. At least two alternate stories are compatible with the new demographic data: Democratic turnout among older, white, working-class people was higher than Republican because 1) they were motivated by Obama’s unique qualities, or 2) the vaunted Obama data-analytic/new media effort turned out the Democrats at a higher rate. In research language, the technology-driven campaign factor is ‘confounded’ with the candidate-of-color factor, making it impossible to distinguish how much each factor contributed to the total Obama vote. It seems quite possible that even some portion of the increase in minority-voter turnout was due to the Dems’ GOTV tech, not just identification with Obama.
      This year should be interesting – Clinton is building on the Obama tech/social media base, but not even she claims to have the political or oratorical skills of Obama.
      I’m betting on the tech.

      Reply
      1. Frank Palmer Avatar
        Frank Palmer

        Black participation will be down from the Obama elections, that is clear. Down from Kerry’s election? That makes no sense. It will be up from ’04.
        1) Once you have voted, you’re more likely to vote the next time Consider Kerry’s figure as the floor, and Black participation will sag towards it.
        2) Clinton has strong positives in the Black community. She won the the primary among Black voters decisively running against a guy who had been arrested in a civil-rights sit in in the ’60s.
        3) Obama is pushing her. That is not as good as running himself, but it is better than anything Kerry had.
        Latino participation rates will be higher than recent history. How much higher is far from clear. But to assume that it will be at a recent-historic low is delusional.

        Reply
  12. Kari Q Avatar
    Kari Q

    I do not believe these polls at all. There are too many results that make no sense and that undermines my acceptance of those that seem plausible. However, there is no reasonable argument for excluding them.

    Reply
  13. TJ Avatar
    TJ

    I see that http://www.electoral-vote.com/index.html removed Ipsos from the math and gave an explanation for doing so.

    Reply
  14. AAF Avatar
    AAF

    What do you think of electoral-vote.com’s reasoning for removing the Ipsos polls from their model’s database?
    Basically, he says that as compared to the averages generated by the non-Ipsos polls, Ipsos produces 21 States that are outliers and 9 States that are big outliers by 10% or more. 40% of one’s results shouldn’t be outliers. He also has some criticism of their methodology that could explain why they get bad data, but I think that very simple statistical point was very interesting. Is his analysis valid?
    http://www.electoral-vote.com/evp2016/Pres/Maps/Sep21.html#item-11

    Reply
    1. Sam Wang Avatar
      Sam Wang

      If I recall correctly, they use a “most-recent-single-poll” rule for each state. Therefore noisy individual polls affect their calculation in a way that PEC escapes. Thus their quality control process requires close inspection of the frequency of outliers.

      Reply
      1. Matt McIrvin Avatar
        Matt McIrvin

        It’s most-recent-single-poll unless there are multiple polls there in the past week, in which case he takes the mean. So if it’s a place like Vermont where polling has been very sparse, one weird poll can have a large effect for a long time.

        Reply
  15. Phoenix Woman Avatar
    Phoenix Woman

    Debbie R: If you’re nervous, go find a downticke t race in a swing state and donate/phonebank, as Dr. Wang suggested a few days ago.

    Reply
  16. Mark F. Avatar
    Mark F.

    Trump could win with a 1.6% Clinton popular vote lead, as the distribution of his voters probably gives him an small EC advantage. In contrast, Democrats had an EC advantage from 2004-2012. Remember that Kerry came rather close to winning despite a big popular vote loss.

    Reply
    1. Josh Avatar
      Josh

      I don’t think the numbers bear either of your observations out.
      First, Clinton had a lead of roughly 1% last week and she still led in the Meta-Margin (the electoral college was up for grabs). If the electoral college was basically a tossup with Clinton +1, it’s hard to see how Trump has an advantage at Clinton +1.6.
      Second, saying Trump *could* win if the polls were Clinton +1.6 doesn’t mean much, because even if Trump had a .001 chance of winning you’d technically be correct but your observation would, in a practical sense, be meaningless.
      Third, a lead of 1.6 a month before the election is very different than a lead of 1.6 on the night before the election. If Clinton led by this margin on November 7, it’s very, very unlikely Trump would win.
      Fourth, all of this also ignores what you’re really insinuating, which is that there’s an asymmetry that favors Trump; that if the opposite were true–if Trump had a 1.6 lead in the polls–Clinton couldn’t win. It’s hard to know if this is true or not, but I suspect it is not.
      Finally, Kerry only lost the popular vote in 2004 by about 2.5%, which I’d hardly call a big loss.

      Reply
      1. Mark F. Avatar
        Mark F.

        Josh:
        I think the GOP may have a general EC advantage, perhaps very small. I do not think Trump has an advantage at Clinton +1.6%, but he may at Clinton + .5%. There is evidence that Clinton will have more “surplus votes” in her states than Trump will have in his. But, of course, Clinton still might have a smallish chance at Trump +1.6%, more than “nearly impossible.”
        Trump’s chances of winning with the polls at +1.6% Clinton the day before November 8 are still a lot more than .0001%.
        Kerry’s loss was not especially big, but it was surprising he ALMOST won with -2.5% in the popular vote.

        Reply
  17. Michael Avatar
    Michael

    Am I wrong in thinking Wisconsin is really the Alamo for Clinton? If she doesn’t hold it, it’s pretty much over, and if she does, same thing?

    Reply
    1. sk Avatar
      sk

      No, I think Pennsylvannia is.
      She can lose Nevada, Colorado, Florida, and one of Ohio, NC or Wisconsin, and she’ll make it.
      PA is the firewall.

      Reply
      1. Kevin King Avatar
        Kevin King

        I think it’s Colorado.

        Reply
        1. Josh Avatar
          Josh

          In a way, maybe?
          But I think this is an unhelpful analogy. States don’t vote in a vacuum; that’s why the Meta-Margin, which measures state polls, moves more or less in tandem with national polling.
          Perhaps a better way to think about it is: if Clinton wins the election she’ll have probably won Wisconsin; if she loses, she’ll probably have lost Wisconsin.

          Reply
  18. Howard Avatar
    Howard

    Evidently, some undecided voters moved to the Trump camp in the past few weeks.
    How do we know how many of the voters polled who pick a given candidate really are not still undecided?
    You mentioned how by late October we have a pretty decent picture of the electorate.
    What do our models and past elections show about identifying undecided voters?

    Reply
  19. Paul Quirk Avatar
    Paul Quirk

    A couple of days ago you said A) HRC’s percentage would shrink (state polls catching up); but B) it would probably rise again (regression to the mean for the election). It has now shrunk. But this is basically an admission that your forecasts omit important evidence: If all were included, change would be a random walk.

    Reply
  20. Kevin King Avatar
    Kevin King

    Looking at the Senate and House data, I have no problem at all thinking the meta margin is an accurate picture of current conditions, Ipsos or no. The one outlier is Obama’s approval. If the meta margin were 1.8 in favor of Clinton, but the Senate and House were strongly in favor of the Democrats, along with Obama’s approval, I might think the metamargin is underestimating Clinton.
    Having written that, I’m still noting that Trump hasn’t taken the lead one single time. There were points in both 2008 & 2012 where Obama was losing. I won’t “panic” until Trump actually takes a persistent lead.

    Reply
  21. Robert Del Medico Avatar
    Robert Del Medico

    mixed results in today’s national polls but the new NBC poll has to be a good sign – lead back up to 6, and it was taken after her return to the trail.
    think state polls have to be right behind. We are gonna be ok folks

    Reply
    1. Kevin King Avatar
      Kevin King

      Today I quickly looked at the medians of the national polls completed since the 14th; both head to head and with Johnson included 7 polls. Unless I am mistaken (since I looked on my phone) Clinton is ahead by 3% in the head to head polls by this measure and 2% including Johnson. These are slight upticks for Clinton.

      Reply
  22. Paul Avatar
    Paul

    “Think of each poll as being a mystery body part; what PEC provides is the head cheese.”
    Smelliest statistics metaphor ever.

    Reply
  23. MAT Avatar
    MAT

    Sam, any chance I could talk you into graphing the 2012 meta margin against 2016?

    Reply
  24. Billy Avatar
    Billy

    Hi Sam, great post as always. I love reading your blog posts that add a bit of sanity to the election.
    I recently did another analysis of my own and thought I would ask for your critique here. I performed jackknife statistics on state-level polling data since September 1st to generate estimates of mean and standard errors, both of which are used in a t-distribution to calculate probability of D/R control (which is followed by a simulation to get a distribution of EVs).
    Here’s some results: https://imgur.com/a/iIiAU
    From this, I think it’s pretty clear that Trump has probably run out of states to flip. In his path to victory, he needs to flip D states while keeping his own as R. The analysis shows that he has about a 16% chance of winning, which is consistent with your random drift model. In my analysis, I speculate that a lot of polling variation in this cycle comes not from actual changes in political opinion but rather from survey response bias. Therefore, getting a larger polling window may be more accurate.
    Nevertheless, I believe it is becoming increasingly evident that Trump has only one way to the nomination, and that is by flipping MI and NC.
    I am not a professional statistician, only a postdoc that does some data science in genomics. The analysis was implemented in R.
    Thoughts?

    Reply
    1. RoF Avatar
      RoF

      I am getting whiplash. 5pm updated had the margin up, now 3 hours later it took another nosedive. Oy

      Reply
      1. A Avatar
        A

        To RoF,
        Regarding the downgraded meta-margin, I’m guessing that might have been due to the state polls from Fox that had Trump up in Nevada and Ohio…?

        Reply
  25. Dean Avatar
    Dean

    The key states that Trump is winning or nearly winning today are Florida, North Carolina, Ohio and Nevada. A couple of recent polls show Clinton leading in Florida.
    If Trump holds onto the above states, he needs to win either Colorado, Michigan or Wisconsin to win the election. Any one of these states is a possible win for him, though perhaps difficult, as Clinton has been leading consistently in those states.
    It’s a different race today. Clinton’s consistent lead has apparently vanished or nearly vanished. If the polls reveal the true state of the race, there is plenty of reason to get nervous for Clinton supporters. It’s a wake-up call for everyone, especially complacent or unsure Democratic/Democratic-leaning voters.

    Reply
    1. ideo Avatar
      ideo

      The firewall should be Pennsylvania according a couple analysis I read

      Reply
      1. Mark F. Avatar
        Mark F.

        Yes, I agree . Note that if Trump wins an electoral vote in Maine, which seems likely, he can also win by adding New Hampshire to the above noted states. Clinton still has a small edge, however, and I would not put money on Trump right now.

        Reply
  26. RDT Avatar
    RDT

    Have I got it right that the meta-margin changes both when new polls come in, and when old polls age out?

    Reply
    1. Froggy Avatar
      Froggy

      Old polls age out only when new polls come in. If there are three or more polls in the last week, all of the polls in the last week are used, and this week is calculated back from polling date of the latest poll.

      Reply
  27. Violet Avatar
    Violet

    Hopefully not off topic, but what I find keeps me from freaking out when I see these numbers is to just go to the Clinton web site and just start making phone calls to the state that Dr. Wang’s or others’ numbers shows is the tipping point. Nevada is so close, and winning it makes winning the rest easy, so I just commit to making calls to at lest 10 Nevada voters in a day.
    I change it sometimes to NH or Ohio, but you can choose a state on their website. As Sam likely knows given his specialty, it’s much less stressful feeling like you can control something, rather than fretting about what we can’t control.

    Reply
    1. Rachel Findley Avatar
      Rachel Findley

      I’m on my way to a Hillary phone bank in California. I am not sure where we’ll be calling; probably Nevada.

      Reply
      1. Sam Wang Avatar
        Sam Wang

        That is a great response. Do it for downticket too!

        Reply
  28. Rachel Findley Avatar
    Rachel Findley

    So, “momentum” as it’s used in politics is roughly an antonym for “regression toward the mean.” But “momentum” sounds trendy, moving, strong. And “regression …” sounds incomprehensible, weak, and negative. Is there a nice alternative verbal expression for “regression toward the mean?” –Like “back to normal” ?
    Sure, we wish the citizens and commentators understood the technical language. But maybe at this point we should throw them a life saver, or a thesaurus, or something.

    Reply
  29. Robert Del Medico Avatar
    Robert Del Medico

    another Hillary 6 point lead in national poll showed up late tonight (AP). obviously won’t budge this or DailyKOS but I have to feel like some state polls are going to break next week.
    esp after the (admittedly Trump-house effected) LA Times/USC tracking poll, which polls the same people over a period of time, went as far as 7 points Trump and is now back to 2 a week later.
    I think a blowout of 80s proportions is out of the question at this point but I feel like 320 EV is where this one could wind up for Hillary.

    Reply
  30. Veronica Avatar
    Veronica

    Why is Hillary now at 289 EV? New polls are coming in that she’s up, yet here, I’m seeing that she’s going down. When will the new results be factored in? What’s going on here?

    Reply
    1. Joel W. Avatar
      Joel W.

      Remember that the EV and Meta-Margin are based on state polls only, and some of them (IA, FL, NV, OH) aren’t so good for Clinton. Others show a tightening in places like MI and WI. There are some good results for VA and NH, and the national polls are better than last week’s, but the national polls don’t count in the calculations.

      Reply
  31. AAF Avatar
    AAF

    Re: national margin correlation with meta margin –
    The meta margin isn’t really a snapshot of today’s state of the race. It’s a snapshot with a three or four week long shutter speed, where the button was pushed about six weeks ago and the shutter closed about a week and a half ago, (I’m guessing at those numbers. They’re probably higher in June and certainly will shrink over the next few weeks).
    National polling average is a snapshot of a different, but very closely related, subject, where the camera was clicked maybe 9 to 14 days ago with maybe a week long shutter speed, as the newest polls include calls from just a couple of days ago and the oldest ones in the average probably don’t have any calls older than two weeks ago.
    I think it should be possible to graph the national margin vs. the meta margin, and figure out how much time delay is built into the meta margin (time adjustment shrinking as we get closer to the election, due to higher volume of state polls).
    This would show us two things:
    (1) it would reveal the EV advantage that one or the other candidate has – for example, if the best correlation shows that the meta-margin is always 1 point lower than the three-weeks-ago national margin over the summer, and the two-weeks ago national margin more recently.
    (2) it would give the impatient among us a better crystal ball into the next few weeks. Instead of “just wait for the MM to catch up”, it would be “in about X days the meta-margin is likely to be today’s Huffpollster national average minus 1.2%” (or whatever the right numbers are).
    And, it would acknowledge that today’s meta margin is really just revealing the true, but unknowable at the time, meta margin of X weeks ago, and could give a good estimate of the true, but unknowable at the time, meta margin of one week ago.
    How complicated would that be? I wonder if superimposing the two graphs and time shifting one of them would make it very clear, but even that is beyond my excel skills and is probably not mathematically valid. I could do it with a transparency projector and two transparencies, but I’m not in high school AV club anymore and it’s not 1985, so I don’t have access one of those.
    OK, I wasn’t cool enough to be in AV club. But I could have asked someone who was.

    Reply
    1. Stu Avatar
      Stu

      To help with this analysis, just add columns that push the state poll data ahead in time by one to three weeks and see what the correlations between state and national polls look like at +1, +2, and +3 weeks. Those correlations should give the an estimate of the “lag” time involved. If two weeks is best, than there is a two week wait time for the national polls to be reflected in the state polling. My guess is roughly a 10-14 day lag between national and state polling, so the latest jump for Hillary is likely not reflected yet in the state polls. By the time it does, unfortunately the debate impacts will start to impact the national polling, but not much you can do about that.

      Reply
  32. Veronica Avatar
    Veronica

    I just found this article at The Washington Post’s website:
    https://www.washingtonpost.com/news/the-fix/wp/2016/09/23/trump-is-headed-for-a-win-says-professor-whos-predicted-30-years-of-presidential-outcomes-correctly/?tid=pm_pop_b#comments
    Sam, what do you think of this guy’s methodology? Is it true or is it bunk?

    Reply
    1. Amitabh Lath Avatar
      Amitabh Lath

      Veronica, here is the standard answer to the argument from precedent:
      https://xkcd.com/1122/
      Also, the Iran nuclear deal, and relations with Cuba? Pretty big foreign policy achievements don’t you think?

      Reply
      1. Joel W. Avatar
        Joel W.

        The article concludes:
        “So very, very narrowly, the keys point to a Trump victory. But I would say, more to the point, they point to a generic Republican victory, because I believe that given the unprecedented nature of the Trump candidacy and Trump himself, he could defy all odds and lose even though the verdict of history is in his favor. So this would also suggest, you know, the possibility this election could go either way.”
        That’s quite the fudge factor.

        Reply
        1. Amitabh Lath Avatar
          Amitabh Lath

          A lot of the “keys” are quite subjective. I would take the assertion that it postdicted elections from 1860-1980 with grain of salt. For instance, in the Dewey vs. Truman election, how would you score the “policy change” and “charisma” keys? Knowing how it actually turned out makes a huge difference.
          I suspect after a Clinton win he will “rethink” some of the keys, maybe the foreign policy or charisma keys. The “model” will remain valid and undefeated, it will be chalked up to user error.

          Reply
          1. Ed Wittens Cat Avatar
            Ed Wittens Cat

            Amitabh:
            here is one perspective on the Iran nuclear deal.
            it makes far more sense than anything else I have read.
            http://www.tabletmag.com/jewish-news-and-politics/213851/obamas-syria-policy-striptease

            Reply
            1. DaveM Avatar
              DaveM

              “…he could defy all odds and lose even though the verdict of history is in his favor.”
              I’d have to say that if Trump loses, the verdict of history is, by definition, NOT in his favor.

              Reply
              1. Sam Wang Avatar
                Sam Wang

                I find this “method” to be foolish. I like the XKCD take.

                Reply
                1. Thomas Avatar
                  Thomas

                  I agree with Amitabh that many his keys are highly subjective, matters of opinion, really. As noted, Iran and Cuba stand out as major foreign policy achievements, so that brings the number of keys lost to the Democrats down to four. Also, he explains that the Dems lose a key because, at the moment, polls show that Johnson may get more than 5% of the vote, and he says that punishes the party in power. But that seems to imply that nearly all support for Johnson come from putative Clinton voters. To be sure, it’s fairly clear that she’s losing some voters to the Libertarian, but it likewise appears that Trump is, too. According to the Huff Post/Pollster averages, when he’s included in the mix, Clinton loses 4 percentage points, but Trump loses 3. Earlier this year, Lichtman, in another interview, indicated that his keys forecast a Clinton win. I suspect a month or so from now, if polling data still indicate a Clinton victory, he’ll give another interview and will once again say his analysis of the keys portend a Clinton win.

                  Reply
                  1. RoF Avatar
                    RoF

                    He has been correct for the last 8 election cycles. In all circumstances, the polls were pretty clear who would win. Not exactly Nostradamus.

  33. Ruth Rothschild Avatar
    Ruth Rothschild

    How are 3rd party candidates (in this case Gary Johnston and Jill Stein) factored into the meta-analysis calculations? Or, are the calculations based on only the candidates from the two major parties? In the various polls I’ve seen, when they include the two current 3rd party candidates, the numbers shift toward or away from Clinton or Trump.

    Reply
    1. Sam Wang Avatar
      Sam Wang

      Generally, the Meta-Analysis takes each pollster’s “topline” number, i.e. whichever one they list first in their report. Sometimes that is the 2-way race, sometimes the 4-way race.
      Follow the link from “Johnson” to see SurveyMonkey data indicating which way Johnson and Stein voters will eventually vote.

      Reply
      1. Ruth Rothschild Avatar
        Ruth Rothschild

        Thanks, Sam. And yes, I did follow the link to the SurveyMonkey analysis and also read your article on not putting too much into tonight’s debate, which I’m not planning to watch. I found SurveyMonkey’s analysis interesting because I figured that with neither Clinton nor Trump being very well regarded, 3rd party candidates would play a larger role and possibly garner more votes than in past elections.

        Reply
  34. Anthony Avatar
    Anthony

    Hi Sam,
    Not sure if this is the right post to comment this with, but its definitely related: I noticed the electoral map just changed to 307 EV for HRC and the Bayesian jumped up to 87%… seems like a big jump so did something “just happen” that we’re not seeing?
    Or is your model “catching up”, possibly, with some other positive results for HRC?
    Thanks!
    -Anthony

    Reply
    1. Anthony Avatar
      Anthony

      Ok, updating my comment… Florida was what changed and its changing again?
      A noisy set of polls?

      Reply

Leave a Reply

Your email address will not be published. Required fields are marked *