Princeton Election Consortium

Innovations in democracy since 2004

Sep 27: Biden 353 EV (D+5.2% from toss-up), Senate 52 D, 48 R (D+3.7%), House control D+3.6%
Moneyball states: President IA NV AZ, Senate MT KS AK, Legislatures KS TX NC

Our Polling Trauma

July 8th, 2020, 1:42pm by Sam Wang


Did the 2016 election make us too gun-shy to trust polls? New in the Columbia Journalism Review, I tell what went wrong, polling successes in 2018 and 2019, and what it all means for 2020. I offer an opinion on how journalists could incorporate data into their coverage of this year’s race.

I also point out downticket races, such as U.S. Senate, state legislatures, and other local races as deserving attention. Polling and other information can draw attention and help mobilize citizen action. For example, we’re using that data to optimize donations (see the ActBlue and WinRed links at right).

Read on! (and here is a hyperlinked draft for factchecking purposes)

P.S. A few of you wrote in, confused on a particular point. In 2016, my final probability was 93 percent for Hillary Clinton, not the higher number produced by the automated calculation (the NYT tracked this).

Tags: 2020 Election

19 Comments so far ↓

  • A

    As much as I want to believe that we need to overcome our trauma, and our ability to predict the outcomes based on the polls will look more like 2012, 2018, 2019, etc, isn’t it possible that there is something fundamentally different about Trump-supporting conservatives that hinders our ability to fully count them? Should we be a little more cautious in our reading of the polls? (e.g. I’m having a hard time believing Biden is leading in TX)

    • Sam Wang

      Read the piece, please? The error in 2016 was 3 points, and 2018/2019 polls have been quite good.

      Biden’s lead is effectively 6 points in the Electoral College, the largest of any candidate in the last 5 Presidential elections. If you doubt that, then stop following polls.

      Finally, the focus here is on your personal activism. If you are not taking that exhortation seriously, it’s A-OK if you leave this site.

    • Jonathan Hutchinson

      I came back for the activism piece! (My Trauma lingers on) How is the voter power calculated? Pseudocode? or location in the code?

  • Jon Wiesman

    Sam, why does your model give NH to Trump?

    • Froggy

      After 2016 I swore I wouldn’t get sucked into following Sam’s site again, but all it took was one simple question to give up on that pledge.

      The latest Biden-Trump NH polls at 538:
      6/13-16 (St. Anselm College) Biden +7
      4/23-27 (St. Anselm College) Biden +8
      2/19-25 (UNH) Trump +2
      2/8-10 (AtlasIntel) Trump+2

      I assume that the model is throwing out the older of the two results from the same pollster, and is taking the median of the remaining three, to give an output of Trump+2.

      This is despite the fact that the two polls showing Trump leading are pre-Covid, pre-recession polls from when the dissension of the Democratic primary was still hot (and before Biden’s comeback).

      We need more poll results from NH.

    • Sam Wang

      I agree, New Hampshire needs more data. It sticks out.

      One could introduce a prior, but I think it is better to just accept that individual states will have this kind of issue. The whole point of the calculation is to avoid cherry-picking. Should we also reject polls that are surprisingly favorable for Biden in Texas, Arkansas, or wherever?

      Welcome back, Froggy!

    • Priest

      Poll released today by Univ. of New Hampshire has Biden +13.

    • Sam Wang

      At last, you guys will stop coming at me with New Hampshire comments… ;-)

  • Amitabh Lath

    That journalist “cheat sheet” mentions poll weighting but not turnout model. Maybe poll aggregation integrates over all sorts of turnout models (Rasmussen overcounts Rs, PPP overcounts
    Ds, everything works out).

    But in 2016 everyone under-represented non-college whites. They usually don’t turn out to vote, but they did for Trump. Oops.

    This is called systematic uncertainty and it’s a bear. There is no correct way to estimate it and that is as true in 2020 as it was in 2016. However pollsters are unlikely to make that same mistake.

    • MarkS

      Pollster Stanley Greenberg in The Atlantic on July 2: “So one reason to trust my polls more now than in 2016 is this change: Four years ago, those without a four-year degree made up 48 percent of my survey respondents; today they account for 60 percent. Whites without a college degree were 33 percent of my surveys; today they are 43 percent. That is a huge change—an elixir against being deceived again. The pain of Trump’s victory and disastrous presidency has concentrated the minds of campaign staff and the polling profession in ways that give me confidence that Biden’s lead in the polls is real.”

    • Amitabh Lath

      John Hammond:
      Don’t worry, I’m not making the same mistakes again.

      Dr. Ian Malcolm:
      No, you’re making all new ones.

      (Jurassic Park: Lost World)

  • Jack

    Sam

    Biggest bang for the buck with 10k to donate? I am looking for the biggest possible D victory national and state.

    I have been donating to Joe B plus top 10 Senate races, what am I missing?

    Love your work.

    • Sam Wang

      See the ActBlue thermometer at right for state legislatures. State-legislative races are a particularly good bargain – 10 years of effects on representation!

      We’ve done quantitative calculations and will reveal those soon. The answers are at the top: Kansas, Texas, and North Carolina.

    • Jack

      OK, I just sent 2500 to each. I’ll be looking for some more opportunities.

      Thanks for having this site, it’s priceless.

  • A New Jersey Farmer

    Sam:

    My apologies for being slow, but have you posted a link to the sites you use for the polling data? When you used pollster it was easy to see the polls you used. Is there a similar link here?

    Thank you.

    • Sam Wang

      It’s all FiveThirtyEight this year. Will link, sorry – I’ve been slow with the documentation this year. Too many balls in the air this summer!

  • Richard Wiener

    Sam,
    I very much enjoyed the draft of the article for the Columbia Journalism Review. I have always believed the most valuable aspect of PEC is the meta-margin. It gives a single number estimate of how much polls would need to be off for a race to be even. Of course any such reduction of data to a single number throws away lots of information, but is nonetheless a wonderful estimated encapsulation of the temporal changes in the race. Knowing that number, any reader can decide for herself how “certain” a race is. In 2016, I watched Clinton’s meta-margin shrink as the election approached and had more and more consternation. In fact, I recall that from when I went to bed on the eve of election day to the next morning, there was movement in the meta-margin toward Trump, as the last polling results came in. By that point the meta-margin indicated a very tight race with Clinton still a slight favorite but a trend toward Trump. Which was exactly correct. Indeed, as you point out, Silver’s model predicted 70-30 for Clinton and 70-30 is not good odds for Russian Roulette, which it turns out is an excellent metaphor for the 2016 election. Keep up the terrific statistical analysis. No other site like yours! Richard

  • Emigre

    Sam,
    do you have any data indicating when it is best – or too late – before the election to donate to campaigns: presidential, senatorial or house?

  • Deb O

    Is there any current data on those changing party affiliation in either direction in the last year?

    What about new gen z registrations by party?

Leave a Comment