Princeton Election Consortium

A first draft of electoral history. Since 2004

Some secrets are not dirty

October 10th, 2016, 9:48am by Sam Wang



Yesterday, Hillary Clinton’s PEC win probability hit 95%.

In last night’s debate, the 2005 candid video of Donald Trump saying what he does with women was still on everyone’s mind. In response, he brought up many topics beloved by Republican rank-and-file voters: Bill Clinton, Benghazi, emails…it was a veritable Greatest Hits of 1996-2016. The likely consequence of this scorched-earth strategy is that Republican leaders are trapped. All their base (R) belong to Trump. This will reverberate downticket.

This seems like a good time to reveal one of the Princeton Election Consortium’s own secrets. Thankfully, it does not involve an Access Hollywood video.

Here it is: poll-based Presidential prediction is not very hard.

I guess that is a pretty boring secret. Sorry.

It is an interesting irony that poll aggregation got popular in 2008, a year when there was not that much suspense in the Presidential race. That year, Barack Obama led John McCain for almost the entire campaign season, with the possible exception of the week after the Republican Convention, where Sarah Palin stole the show. That ended up with a 7-percentage-point popular win, and an electoral outcome of 365-173.

President Obama’s re-election in 2012 carried even less suspense: he never lost the lead to Romney. The closest he came was right after the first debate, though even then he was slightly ahead. The eventual outcome was a 4-percentage-point popular win and an electoral outcome of 332-206.

I have formed a sneaking suspicion that the runaway success of poll-based forecasting arises from these two victories. If this is correct, then quantitative prediction models at sites like The Upshot and FiveThirtyEight (and PEC) basically serve as prurient entertainment for progressives. Which is okay with me. Everyone needs an outlet. Republicans got theirs in 2010 and 2014.

I think it is a good thing that the other sites did not start in 2004. When many hobbyists (including electoral-vote.com, me, and many others) started doing poll aggregation, it was a tough year: John Kerry and President George W. Bush traded the lead several times, and it was a photo finish, coming down to Ohio. When it comes to probability, it is too easy to do a suboptimal job of extracting all the possible value out of polls. That would have led to a boring year of commentary: “it’s too close to call!” seems okay for a pundit to say, but is that what we really want from a data nerd?

This year, Hillary Clinton’s lead has been remarkably consistent, despite the emotional drama offered by commentators. At some level the drama is justified by the expected value, which is defined as the size of a payoff (or cost) multiplied by its probability. This year, the cost of a presidency as profoundly disruptive as Trump’s would be enormous. Even 5% of that would be notable.

Reader Damien asks wonders if “one candidate leading an ‘open seat’ presidential race from wire-to-wire is almost unprecedented.” The answer is that we haven’t seen anything like it in over 60 years. The comparisons are 1952 (Eisenhower v. Stevenson), 1960 (Kennedy v. Nixon), 1968 (Nixon v. Humphrey), 1988 (G.H.W. Bush v. Dukakis), 2000 (G.W. Bush v. Gore), and 2008 (Obama v. McCain). Of these, the only race where one candidate led consistently from start to finish was Dwight Eisenhower, who eventually won by 11 percentage points (electoral outcome, 442-89).

Statistically, the two most notable features of this year’s Presidential race have been its closeness and its stability. The stability arises from polarization, in which opinion has been nearly immovable. The Meta-Margin has averaged Clinton +3.6% with a standard deviation of 1.0%. Clinton’s lead is 3.5 standard deviations, which is really big, statistically speaking. It’s a core reason why the PEC win probability has been so stable.

If the Meta-Margin were to drift as high as Clinton +6.0%, it would still be within two standard deviations of the average. Then again, the last week has been pretty surprising. If that happens, we can drill into why, as well as talk about how it affects the Senate and House.

That is not to say that prediction is perfect. I certainly learned some strongly administered lessons in 2004 and 2014. But Presidential analysis seems to be a problem that is well under control, thanks to the abundance of data and analysis from hordes of nerds.

In coming weeks I will get into the remaining puzzles of this year, such as how our approach differs that of other sites, and what it would take for Democrats to re-take the House. Other commentary is likely to concern one of the following topics:

  • Downticket (where you should get involved!),
  • Senate/House prediction and analysis,
  • Partisan polarization, and
  • Making fun of pundits.

These are more in the domain of “data journalism,” which reader NFB points out is a big source of added value. Still, the horserace coverage is what initially draws people in.

Did I mention that you should get involved downticket? Explore the links in the High-Impact Races sidebar on the left.

Tags: 2004 Election · 2008 Election · 2012 Election · 2016 Election

115 Comments so far ↓

  • Roger

    Not related to this site but I found this very interesting – a sampling error by a single person in rolling tracking sample that amplifies the poll’s model drastically and causes that one person to weigh from a minimum of 30X the average person in the sample to as much as 300X:

    http://www.nytimes.com/2016/10/13/upshot/how-one-19-year-old-illinois-man-is-distorting-national-polling-averages.html

  • Jon Greenberg

    You hold up the most recent two elections as evidence for the superfluity of sites like 538, and suggest that some of us turn to them merely for prurient entertainment.

    However, a slightly longer look back into history reveals not only the volatile and down-to-the-wire 2004 race (where bad exit polls led to liberal overconfidence on election night), but also the once-in-a-century close 2000 race.

    Against THIS backdrop, it’s no wonder that liberals turn to wunderkind Nate Silver in 2008 to shield them against reliving the trauma of 2000 with its minor echo/aftershock in 2004. What Silver does may be easy for you stat guys — but for the public and the punditocracy it looks like rocket science (maybe that’s easy too?).

    We turn to Silver because those whom Calvin Trillin called the Sabbath Gasbags are so astonisingly bad at what they do.

  • Steve Scarborough

    Dr. Wang, you say: “Here it is: poll-based Presidential prediction is not very hard.

    I guess that is a pretty boring secret. Sorry.”

    I disagree. I note that Olympic divers often make it look easy. In my opinion, the intellect and hard work of you and your team is what make it “not very hard.” Keep up the great work,

    Be well,

    Steve

  • Trump+Democratic Congress?

    Professor Wang,

    Certain Donald Trump supporters are starting a movement in the pro-Trump community to withhold support from the Republican down-ballot or affirmatively vote for the Democratic Party candidate on the down-ballot. The goal is to flip the House and the Senate to the Democratic Party.

    So what I want to ask is what is the easiest way to identify the most vulnerable Republicans? We want to focus our efforts to inflict to maximum damage on the Republican Party, and that means flipping the House and Senate to the Democratic Party.

    If you can point me in the right direction, I can get the information to the right people in the Trump community, and it might very well have a real effect in favor of the Democrats.

    Also, if the House and Senate do flip, will there be a way statistically to distinguish to what extent the collapse in Republican support was due to Trump alienating people versus Trump supporters voting for Trump at the top but then rejecting the down-ballot?

  • AAF

    A wish for a statistical/data-mining analysis. I would like to know:

    Among men who have issued public condemnations of Trump’s comments on the infamous tape, what percentage of the Republicans, vs. what percentage of the Democrats, preface that condemnation with, “As a [relative of a female]“, as justification for why they are eligible to be offended by Trump’s comments.

    Is that something that could be done relatively easily, or is that kind of data collection too difficult to automate?

    • Ravilyn Sanders

      Coding part is not very difficult. Some linux box, wget or some other fetch method, some grep and regular expressions, may be a little web crawl. It would take a while to set up the infrastructure. But once done, it would relatively easy to search.

      But probably not worth it. Use boolean operators in google search, limit it to news, and add a time window and count it manually.

    • Violet

      I hate that “as the father of daughters” stuff. It perpetuates the knee-jerk reaction to only view women through their relation to men, as alleged protectors. Men should be outraged because women are their equals and fellow humans, not because they have a mom/daughter/ wife.

      It’s like saying only someone married to or having adopted someone of color can be outraged by racist behavior, when we should all be outraged. End of rant.

  • Chin Gian Hooi

    Right after the second debate, I went to bed hoping that your crunched numbers turn into electoral reality, but allow me to play devil’s advocate on the situations still lingering in my head.

    1. What about the surprise effect of Trump, as demonstrated by him winning the Republican primary without many people foreseeing it?

    2. How about voters who may not have been polled, especially the swaths of voters he has energized to come out of the non-voting camp? Besides, how exactly are these polls collected? It seems that people, at least those I work with, seldom get calls from pollsters during work hours, which I assume is the pollsters’ work hours too. And internet polls, all of which Trump claims he won, aren’t exactly scientific.

    3. How about voters who answer the polls one way and vote another? Some people may not be terribly proud to admit they are voting for Trump, even if it’s to an unknown pollster.

    4. Is it possible that another Brexit-esque situation would occur, and if not, why? eg. Are American presidential politics polling much more frequent and distributed than Brexit’s?

    I suppose the situations above deal with the limitations of poll-based presidential election prediction, so how certain can you be of these factors, or do you lump them into noise captured by the meta-margin / standard deviation?

    • alurin

      1. “The surprise effect of Trump” is not really a thing. I mean, sure, nobody took him seriously, but that’s not a magical superpower. And the Clinton campaign is definitely taking him seriously.

      2. – 3. During the primaries, polls accurately predicted Trump’s vote share. There is therefore no evidence for “voters he has energized to come out of the non-voting camp”, or people voting for trump but not admitting it to pollsters; there are also robo-polls. Pollsters actually have different work hours, precisely because they don’t want to call people at work.

      4. Yes, American presidential politics is subject to much more frequent polling than Brexit. Sam (and others) nailed the previous two elections using this method.

    • Sharon Machlis

      One of the key differences between polling Brexit and US presidential elections is that for US elections, there is historical data that helps with modeling voter turnout. There was no such data to help pollsters figure out who was actually going to turn out and vote on Brexit.

      Remember the state where polling was off the most for the Democratic primary this year? Michigan. It’s no coincidence that the 2008 Democratic primary in Michigan didn’t have good data – Obama wasn’t even on the ballot because the party was punishing Michigan for holding its primary too early.

      Measuring public opinion is only part of the polling battle. Pollsters have to also decide who is going to vote those opinions.

  • David D.

    I love the graph of the Clinton win probability over time — is there any possibility that this can be added to the site as a permanent, automatically updated feature?

    • Matt McIrvin

      I think the past history of this is a bit less meaningful than for the other graphs–the sudden jump up in mid-August was from a formula adjustment.

    • Sam Wang

      Probably everything is ok except for yesterday’s estimates, which I will fix later. If I recall correctly, until now there haven’t been many multiple contributions from a single pollster within a seven-day period.

  • Norbert Hirschhorn

    Are there enough never-voters among disaffected white persons (who may not show up in polls) who come out in “droves” (quoting Netanyahu) on November 8 to tip the balance in the key EC states, Florida, et al?

    • Kenny

      The same question can be asked about enthusiastic anti-Trump Latinos and others who have newly registered to vote against Trump. There actually seems to be more evidence of that — based on some of the stats I’ve seen for new registrations.

      But yeah, I don’t believe either of these types of voters would show up in the LV numbers.

    • Matt McIrvin

      My impression is that there hasn’t been a concerted effort to get these “missing white voters” registered.

    • Damien Smith

      IMO, the “missing white voters” are simply people who hate politics and don’t want to have anything to do with it. Even if you somehow forced these people to register you would have a heck of a time getting them to show up on Election Day.

    • Orlylicious

      They’d would have needed to register to vote and in many cases I’d bet that’s unlikely. Many deadlines have passed now.

    • Sam Wang

      Yes, but recall that the first debate in 2012, in which Obama did poorly, was on October 3rd. By October 10th, Obama was at his weakest in polls for the entire year. That HuffPollster comparison is smoothed and doesn’t show the events, and is therefore a bit misleading.

      That said, it is easy to imagine Trump getting stuck at 42-43%. It does seem to be where he has been all year. Then Clinton could get a >10% popular-vote win. Echoed downticket, it would probably be sufficient to give the House to the Democrats.

      At this point the House Republicans’ main job is to distance themselves from Trump; Democrats’ job is to tie them to Trump. That’s pretty much the whole ballgame.

    • Matt McIrvin

      At this point in 2012, Obama was into his scary slump after the first presidential debate. Biden was just about to debate Ryan. “Please proceed” and Hurricane Sandy were still in the future.

    • Davey

      This is a great election to test the resilience of gerrymandered districts. The point of gerrymandering is to spread voters out to create as many 53/47ish districts for your party as possible, so that you just win as many as you can.

      Therefore, we’d assume that if a horrible candidate made turnout drop significantly, suddenly being a Congressperson in a 53/47 district isn’t as awesome as being the one in the 85% district where all the opposition party voters are packed in.

      If Trump loses by double digits, we’ll have a rare opportunity to study how much this affects the various districts, and if the gerrymander backfires. It will be an interesting election night, for sure.

    • Ed Wittens Cat

      If most gerrymandered districts are 53/47 then this does seem like a situ where we could see measurable GOTV effect– this paper claims field office location and numbers and can increase county vote share up to 1%:
      http://joshuadarr.weebly.com/uploads/5/1/8/1/51819147/darr_and_levendusky_final_publication.pdf
      remember the epic failboat (huh) of ORCA in 2016 that left field operatives claiming they were “flying blind”– at a minimum local field offices are supposed to send canvassers to ppl that havent voted yet in the last weeks of the campaign– ORCA couldnt even do that much.
      So while Trump still holds the handle of the sword and has forced Pence and Priebus to kneel, Trump doesnt hold the purse strings.
      Priebus officially committed to working with Trump but that doesnt mean he will sink funds into a campaign that appears to be doomed.
      Also Clinton has nearly 3x the field offices that Trump has (489 to 178) and dominates in every battleground state.
      So it seems (to me) like Project Ivy might make a difference in the 2016 election environment.

    • Sam Wang

      Mostly not 53/47, more typically 60/40 or so. See Figure 4b of my Stanford Law Review article. However, these districts are typically more sensitive to swings than the nation at large. So they are sensitive to national changes in mood.

    • Steven

      If I recall earlier discussion at PEC, gerrymandering creates the risk of a tipping point – perhaps 60/40 – where the trick collapses? If you plotted the D minus R vote margin across a gerrymandered state like PA, versus the seats won by Ds, would you expect to see the seats lagging for typical margins, but then a sharp upturn once the margin gets big enough, so the seats won would shoot up and exceed the margin of popular votes?

  • Richard Vance

    Sam and all commenters and checkers.
    Thank you.

  • Rachel Findley

    I made a contribution to the Clinton campaign in honor of my mother, who was born in Pennsylvania and lives there now. My mother is an Eleanor Roosevelt Democrat. If she were able to understand that Hillary is leading the Democratic ticket, she would be thrilled. So now I discover that the close Senate race in Pennsylvania includes a Democratic woman candidate. I’ll send a donation tonight. And I’m now looking for a close House race in one of the many places she lived in her long life. I know she would want to give Hillary Clinton a supportive Congress so she can appoint a good Supreme Court justice and promote the well-being of all Americans.
    Doing this helps me deal with the sadness that my mom’s mind didn’t hold out long enough for her to celebrate the progress women have made in her lifetime.

    • Rachel Findley

      I neglected to state my gratitude for PEC and Sam Wang for helping me find the best downticket races for my mother’s honor.

    • Shawn Huckaby

      The way you are honoring your mom is beautiful. I’m sure she would approve!

    • Violet

      Sorry about your mom. My mother is a 93 year old Holocaust survivor who is thrilled she will get to vote for Clinton.

      The election has been traumatic for her because she saw Hitler come to power and no one took the threat seriously.

    • Michael Hahn

      @Violet

      My parents (90+) also grew up in Germany in the 20′s and 30′s, and they have exactly the same reaction to this election as does your mother. They are very frightened to see and hear the same thing again in their old age. And, yes, they ARE going to vote, and it will NOT be for Trump!!! (as will I, with strong conviction!!)

  • AA

    I felt Trump performed much better in the last one hour of second debate. Clinton still struggles with the emails, which seems like her achilles heel. The CNN post poll numbers where very interesting. Polled debated watchers handed a resounding victory to Clinton. It makes me think that voters have given up on Trump after last Friday leak, and there is very little he can do to regain traction.

    • Robert

      Well Trump did “well” with diminished expectations at hand. Trump scored points but has done little to suggest he’s actually fit for the job. Really Hillary could’ve done better but she did well enough I’ll bet this debate doesn’t alter things.

    • Kenny

      Robert’s right. Him doing “well” is a perception based on the fact that people thought he was going to have a worse meltdown than the 1st debate… and it started to look like that during the first half hour. But when he wasn’t an absolute mess he was still giving barely coherent answers to questions that weren’t asked.

      I guess he looked more like the aggressor getting some of his alt-right talking points out there, but he didn’t do well…

  • A

    Curious about folks’ thoughts on the early voting numbers. Here: http://www.huffingtonpost.com/michael-p-mcdonald/early-voting-a-month-to-g_b_12424562.html

    It’s promising to see the NC numbers looking promising for democrats but the numbers in Iowa are not great relative to 2012.

    And Florida seems to still be a mystery…

    I’m curious if there is more info out there about early voting and how the race appears to be shaping up so far…and whether anyone thinks there is anything to learn from what we do know.

  • TJHalva

    Poll based presidential prediction is not hard.

    You are absolutely correct. Its predicated on just having all the data, which isn’t a trivial task.

    I’ve been playing with really simple models. My favorite so far is just a linear, least squares regression in each state based on polling end date.

    http://howwillamericavote.com/?technique=poly&params=1

    That’s sort of a back door link, as I don’t have the UI setup to support changing the model yet.

    I really wish there would be more of an emphasis on polling methodology and quality, but everybody just focuses on the topline numbers.

  • A

    Big drop in Meta Margin, went from 4.0 to 3.3…wonder why. I thought the state polls had been going up consistently…

    • Sam Wang

      A big change like that is probably Florida or Ohio getting closer. You can probably figure it out by poring over The Power Of One Vote to see what the close states are.

      Also, I’m told that some older UPI polls didn’t get omitted even though there are fresher ones that just entered the calculation. Need to look into that.

    • Kevin King

      I noticed that as well. I looked at the sidebar, and Virginia stood out. It seems like it should still be Clinton +7 rather than +1, unless the UPI/CVOTER poll is counted twice. Thing is that the latest poll is an improvement to Clinton +1 from Trump +5. Am I wrong?

    • Kevin King

      Jinx with both Sam & Rob!!! :)

    • Kevin King

      Also, New Hampshire looks like it should be Clinton +5. UPI/CVOTER went from a tie to Clinton +5. There are two other polls showing Clinton +7 and a Clinton +2, as well. The second CVOTER poll seems to have knocked out the Clinton +7 poll, which leaves one Clinton +2 and the two CVOTER polls. :)

    • Kevin King

      And Arizona, too! CVOTER improved for Clinton from Trump +10 t0 Trump +6, and the median should have stayed Trump +2, with one other poll showing a tie, but the latest CVOTER poll knocked the Trump +2 out, leaving Trump +6 as the median.

      Amazing that as bad as the meta margin looks for Trump, and as bad as his chances are, they should actually be worse!!!!

    • Kevin King

      Obviously, I have too much time on my hands, or else I am procrastinating while needing to do other things but got sucked into a meta margin hole.

      Take your pick. :)

    • Paul Mainwood

      The CSV is showing Virginia sunk down to 60% prob of Dem win. That looks like a data error. Latest polls all showing Clinton around +7% unless there are new ones I don’t know about?

    • Matt McIrvin

      So UPI/CVOTER is starting to spam the state aggregation sufficiently that its fairly clear house effect is affecting the result. I always wondered if something like this might start happening.

    • George

      I agree with Matt, whether you consider it spamming or not – but the same thing happened when Reuters/Ipsos dumped a mega number of state polls, of uncertain quality into the mix. It tended to “flood the zone” so to speak, and until they washed out the MM was over-depressed. Unfortunately, we had the whole Trump melt-down associated with new state polls so we really didn’t get to see where we would have stabilized – but were headed back towards high 2s to low 3s, as I recall. Now we have the same thing again, with UPI again flooding the state poll zone again, even though their national number of +6 for Clinton is quite reasonable.

    • Erik Pescara

      As of the FAQ: ” At present, the same pollster can be used more than once for a given state.”

      So I would think this is intended. But I also remember that older polls from the same pollster should be omitted.
      Especially in the states that are less polls the UPI/CVoter poll has more weight than it should. And of course the new Reuters/Ipsos 50 State poll is still not added by HuffPo…

    • Sam Wang

      I apologize for the ambiguity. At different times, the rule has been different. It ought to be omit-older-polls-from-the-same-pollster, but it is more complex – see my post for Tuesday morning.

      We hope to get it fixed today…though this particular UPI issue may get washed out first, as more polls come in.

    • Kevin King

      Wow! That adjustment made a Yuge difference!

  • A New Jersey Farmer

    As Sam has said, this race was over before this past weekend and it looks like the Democrats will retake the Senate. Any other talk of Trump winning the election has corporate eyeball envy written all over it.

    • Ed Wittens Cat

      right now “Paul Ryan” is trending– and its full of threats and vituperation with rabid Trumpkins calling Ryan a “cuck” and a traitor and a globalist.
      What happens to these peoples rage when they realize (like Ryan apparently did) that Trump cant win?
      CNN’s Jake Tapper just screened results of the current battleground polls and there is no way for Trump to get to 270– Clinton is already at 272.

  • Dan

    I just want to say that I very, very much appreciated the “All their base (R) belong to Trump.”

    A++

  • DonC

    Calculating poll medians may be easy to do. However, PEC, by including insights — such as the Republican rules enabling Trump to get the Republican nomination, or the movement of voters who would ultimately vote for one party or the other being interpreted as “swings”, or how the electorate has become polarized — does more than do simple median calculations.

    Of course one could say these things are obvious, but the truth is that insights always appear obvious in retrospect.

  • Lorem

    This is a little off-topic, but I’m actually very curious about the effect of many Republican leaders renouncing their presidential candidate.

    At first glance, it seems to me like this is poor strategy – they are likely to lose some small part of Trump supporters, and I’m not sure they’ll really gain any votes (compared to the strategy of “say nothing and avoid the questions”). Perhaps many of them are actually expressing their personal views or trying to fix their long-term reputations, so several years from now they can say “I was against Trump”?

    Regardless, it seems like this renouncing may actually have a noticeable impact. In particular, if some of the more old-school socially conservative Republicans feel validated in their refusal to vote for Trump, and some Trump supporters feel validated in their refusal to vote for old-school Republican candidates, I could see the vote shares of both groups going down by 2-3%. Which, I feel, in the context of a race like this would be quite significant.

    • Ed Wittens Cat

      well…I was wrong.
      it looks like Ryan thinks he has the handle of the sword after all no matter what Pence does.
      Hafta think this means War.
      http://www.politico.com/story/2016/10/ryan-to-house-lawmakers-i-wont-defend-trump-229541
      apolos Dr Wang– i guess i need to have a t-shirt made that says Data Over Drama.
      ;)

    • Scott J. Tepper

      Down ballot Republicans are in a pickle. Trump has a hard core of what I’d off-the-top-of-my-head guesstimate as 15%-20% of the vote. Those Republican candidates who separate themselves from Trump will lose the votes of many of those people. They seem to be generally angry and vindictive based on how they reacted to Trump’s performance last night, and before then.

      Another unknown percentage of Republicans who would vote down ballot for their fellow Republicans dislike Trump and may punish GOP candidates who don’t separate themselves from Trump. This is especially true since the Billy Bush conversation was released.

      And those who try to walk down the middle of a sharp sword, like Paul Ryan, may be punished by both groups. (Ryan himself will still win his seat back. But he will never win a Profiles in Courage Award.)

      Down ballot Republicans also have another decision to make. How do they cooperate with a GOP GOTV drive if the GOP includes Trump in that drive.

      Bottom line is I think they’re screwed, and we’ll find out just how screwed after the vote. I look forward to a detailed dissection of the data in months to come. (I think there are too many variables to make any quantitative analysis right now. Especially how the GOP runs its GOTV operation to help down ballot candidates.)

  • Vatnos

    PEC tends to be erased in talks about polling aggregates. I’m not sure why. Perhaps because it’s pure statistics without the punditry that 538 and RCP engage in?

    I find it useful to look at RCP as an extreme scenario.

  • josh f

    nice arms-length analysis.

    as we await more trumped up desperation, here’s my for-fun contribution :

    clinton 2016: “data > drama”

    trump 2016: “polls are meaningless- rigged unless i’m winning- although wang beats silver has a nice ring.”

  • Some Body

    It also depends on what you’re looking for. RCP have a simple average of recent polls, without smoothing, special sauce, or what have you. PEC, of course also keeps it simple, but medians are a more controversial choice (at least for the lay audience), and the MM is again a more sophisticated “product”. Linking to RCP gives a feeling of being closer to the raw data, as it were. And for some purposes that’s useful.

  • BenD

    Great post, Sam. Been touting PEC for the whole cycle. Love the 90s reference too. All your base are belong to us. Let’s talk about Congress…

  • Runner

    The 95% Bayesian win probability for HRC is startling and quite interesting. It is, in itself, a topic for intelligent and lively discussion.

    It has been said that Bayesian inference is common sense reduced to computation. I think that may be accurate in this context. Common sense tells me that HRC will win.

    However, the reason this site excels over all others is because it seems to be quite unbiased. It relies upon a plethora of collected data, not personal biases or preferences of the site’s owner. The Bayesian inference utilized by Sam tells me, in effect, which hypothesis I should believe in, and how strongly, given all the accumulated data. At this point, the data says the cake is baked and will have HRC written on top of it.

    But, there are, in my mind, at least 4 primary factors which could theoretically call the accuracy of the 95% Bayesian win probability into question. They are:

    1. An unrepresentative sample of potential voters;

    2. Survey responses being a poor measure of voting intentions (eg., people saying they are undecided when they are not, or an overly large number of actual undecided voters);

    3. A significant shift in attitudes between now and Nov 8;

    4. Unpredicted patterns of voter turn-out.

    I do not see #1 to be applicable to any meaningful degree. The samples relied upon are voluminous and seemingly representative.

    As to # 2, I do not realistically see people saying they are undecided when they are truly not in this highly polarized era. And I find it hard to believe that there are really many undecided voters out there.

    But 3 and 4 are more interesting and theoretically relevant. As to 3, could there be a significant shift in attitudes between now and Nov. 8? Could such a significant shift affect the result? What, if anything, could cause such a shift?

    And I think 4 is the most important. That is, how likely is it that unpredicted patterns of voter turn-out will occur in this presidential election? And how likely is it—if at all—that unpredicted patterns of voter turn-out will call into question the 95% Bayesian win probability for HRC?

    We will know for sure on Nov 8th, but is it at all likely that any of the above factors will, so to speak, un-bake the cake?

    • Some Body

      Just to note that if (a man forbid) Trump wins, that does not necessarily contradict a 95% probability of him losing. Events with 5% or less chance of happening actually happen, 5% of the time…

    • Commentor

      You are essentially wondering whether the polls are “skewed.” Except for your #4, i.e., systematic error in likely voter models with most or all pollsters, all of the other four factors you raise are addressed in a statistical sense in Sam’s algorithm.

    • Bernd

      #3 is what keeps fueling the 5% chance of a Trump win. #4 seems unlikely, as pollster turnout models worked reasonably well in the primaries, and voter registration data also does not suggest that there’s something unsual going on.

  • e uprichard

    I donated to senatorial tickets using your analysis. Thank you so much.

  • Jim

    “All their base (R) belong to him.” I see what you did there. Genius. As a long-time reader I have to wonder if I’ve been missing such Easter eggs for years…

  • Brian

    The 1988 election was an “open seat” race as well. George HW Bush vs Michael Dukakis. I recall Dukakis had an early polling lead in the summer, and then the polls switched to HW Bush by Labor Day. HW Bush kept his steady lead until he won on Election Day. Thanks for a great web site Mr. Wang.

    • Sam Wang

      Thank you for this correction.

    • Frank

      Yes, but GHW Bush was Vice-President in 1988. This was truly open seat – no Prez or VP running again. 1952 was like that also (Eisenhower v. Stevenson, the Gov of Illinois at the time — Truman refused to run again).

  • Michael

    I’m still a bit stuck on why CNN, MSNBC, etc. don’t cite this website. Isn’t it really because it would leave them with nothing to say to fill the hours?

    • Sean

      The sort of analysis that Sam engages in isn’t suited to high TV ratings.

    • Chuck Lavazzi

      I think you just answered your own question. The 24/7 “always on” news cycle actively encourages the manufacture of “breaking news” and a focus on trivia.

    • Kevin King

      They could cover the issues and detailed analyses of the candidates proposals.

  • Ryan Casey

    Sam – I’m still intrigued by the widespread (but probably incorrect) intuition many Democrats have in which they fear that an election that seems “in the bag” for Hillary will cause Democratic voters to stay home, thus jeopardizing her election. Clearly David Plouffe does not share this fear. But it’s amazing how even leading Democratic strategists have this intuition, with no empirical evidence to support it. Seems to me it’s not unlike the mistaken “rational choice theory” traditional economists apply toward human purchasing decisions.

    So I think that is a great topic for future analysis and commentary. Thanks!

    • Some Body

      It’s a good thing they’re worried, though. Even a tiny chance of such concerns materializing far outweighs the costs of some people feeling a bit nervous for a while.

    • Daniel Barkalow

      I wonder if a lot of Democrats are concerned that saying this election’s in the bag for Clinton will cause lots of Democratic voters to stay home, causing them to lose down-ticket races. It looks to me that Clinton can win entirely on people who are actually excited to vote for her, rather than needing voters who are concerned about the outcome, but there are a lot of races where voters are choosing between two non-horrible people with very different politics, and the Democrats could really use the voters who show up for the presidential election to provide the standard bounce.

    • Commentor

      You would think that it would be equally plausible that an expected victory would spur enthusiasm. For example, the Alabama Crimson Tide doesn’t have trouble drawing fans to games which everyone knows they will win. As you suggest, we’ve had some elections with clear leaders so there should be some data to answer this question.

    • Steve

      I think that people are worried enough about what a Trump presidency would be like, that even with a Clinton victory highly likely, few of them are going to stay home. In fact, I wouldn’t be surprised if this election had one of the highest turnouts in a while.

    • Ryan Casey

      Daniel, your point about down-ballot races is well taken, but my point isn’t so much that “Hillary’s election is guaranteed, therefore I’m not worried that some voters may stay home,” but rather, “the intuition that voters who think an election is in the bag are more likely to stay home is not backed up by empirical evidence,” and may even be the opposite in terms of the net turnout effect vis-a-vis supporters for the “doomed” candidate. Some of this research is out there, but much of it is held close to the vest by astute data-mindful campaigns (i.e. Obama 2012 and Hillary 2016).

    • Sophia

      I have commented on this sort of question before but I’ll repeat it if it is OK. People show up for winners and tend to not show up for losers. That is why the Denver Broncos sell out every game and the Florida Buccaneers don’t. That is also why the Republican politicians are no longer supporting Trump. They know he’s a loser. It feels good going into the voting booth knowing that my choice will win.

  • NFB

    The biggest strength and raison d’être, as I’m sure you know, of data journalism is as a correction to single-poll-to-single-poll horse race journalism.

    Even in this, the third race of the data journalism era, news stories about “Trump up 2 in new poll!” are still abound in major news networks. Even otherwise quality news organisations such as politico will still engage in this behavior. The incentives are just to great, as poll-to-poll horse race reporting just creates a much more gripping narratives driving views than steady-as-she-goes data journalism averages.

    I agree that 538 and the Upshot have benefited greatly from a liberal fanbase wanting confirmation (or reassurance) of inevitable victory. But I do think you’re missing something here.

    The thing that truly brought me over to Nate Silver wasn’t his predictions in 2012, it was his 50-state series going in depth into the demographics of each state and what past and future shifts likely meant.

    I think in this article you’re running close to reducing data journalism to just predicting presidential election outcomes. This ignores the work Nate Cohn and 538 has done on demographics, and the work you yourself have done on partisanship and gerrymandering. That is also data journalism, and equally important to improving the quality of political journalism as a whole.

  • Bruce Sands

    Hi I enjoy your website. I started going to electoral-vote back in its early days.

    I don’t think Trump, at this point, cares about winning. I do think he cares about keeping the base. I believe he and Ailes are going to start a cable network, aimed at his base, shortly after the election.

    Thank you for your website and analysis.

    • Damien Smith

      I also think that Trump’s motivation from the very beginning was to build an anti-Hillary media empire. Without expecting to win I’m sure even he’s surprised he got this far. People have aptly compared the Trump campaign to that 1960s movie The Producers. Two guys write a truly awful play called “Springtime for Hitler” that’s designed to be the biggest flop in the history of Broadway. But when the play opens people love it and it’s a runaway hit.

  • Matt McIrvin

    Maybe I’m a weirdo, but 2004 was actually what got me hooked on these sites.

    And what convinced me they were useful was looking at how amazingly well they performed in the 2004 aftermath–even though PEC called it wrong! Prof. Wang’s error was the result of his not really trusting his own data, just like Nate Silver not really trusting his own data during the 2016 primary. And, perhaps unlike Silver, he learned the right lesson.

    2004 was such a photo finish that it was almost impossible to call. And yet, if you looked at the very last week of polling at PEC, Electoral Vote and RealClearPolitics, you’d have seen the map implied by polls alone converging to pretty much what we saw on Election Day. Even though leaked exit polls were saying wildly different things, and giving Democrats false hopes that led to conspiracy theories… the Election Eve polling was right. It was amazing, as depressing as the result was.

    • Rob in CT

      For me it started with 2008, but what cemented it was 2010, 2012 and 2014. The mid-terms are a little harder to call, but the aggregators still were mostly right. I knew full well that the Dems were going to catch a beating in 2010. I had time to prepare for that emotionally. Ditto 2014. Neither outcome was good from my PoV, but I didn’t watch in stunned horror either. I knew what was coming, basically.

    • Arthur Neelley

      Let’s not forget that Ohio (not to feed that beast too much more than has been done for the past couple of decades…) was very much contested and remained in question for years after Kerry lost there. There is evidence of vote machine tampering to “toss out” one vote per machine per precinct that may have then given the outcome to Bush. We are all pretty much certain of the shenanigans that went on in 2000 in FL that cost Gore that race by more underhanded tactics, and has been upheld by a consortium of newspapers that did a study later on. So, actually, Sam was probably correct after all, except that maybe he should have (in hindsight) thought to factor in the “underhanded shenanigans” factor, which has become so much a part of a certain element of the American electorate desperate to hold on to their power and privilege.

    • Matt McIrvin

      I think the election-eve polling strongly suggests that whatever happened in Ohio in 2004, it wasn’t enough to flip the result. I remember that Mark Blumenthal at Pollster came to that conclusion after a fairly detailed analysis.

      I suspect any vote-machine tampering was less important than the stuff that was visible just out in the open: Ken Blackwell was trying really, really hard to suppress the vote in Democratic-leaning areas, by rejecting registration forms for being printed on the wrong kind of paper, bringing down the number of polling places, etc. But I think Bush probably would have won in an even tinier squeaker if it had been fair and square.

  • Michael

    Sam, News pundits never mention your site, and rarely mention 538 and The Upshot. Do you think that is because their ratings depend on the appearance of a competitive race and that they have 24 hours each day to fill with something, or is to ask the question is to answer it?

    • Sam Wang

      Maybe it’s because we are hobbyists who work without payment.

      However, it does leave open why RealClearPolitics gets mentioned often. They are right-leaning and attract less vocal complaint. Also, they just report averages, which does not require a ton of explanation.

    • Matt McIrvin

      I’ve noticed that liberal-leaning essayists cite RCP pretty often–it’s a convenient rhetorical tactic, since RCP is trustworthy enough that it’s not that far off from reality, but it’s an obviously right-wing site that nobody can call liberally biased.

    • bks

      That’s how I use it, Matt. I will also give a foxnews.com URL for AP and Reuters stories.

    • Matt McIrvin

      Even the conservative RCP!

      It’s kind of the counterpart of all those center-right pundits who have an antediluvian reputation for liberalism that is sort of grandfathered in, and get cited all the time as “even the liberal X”. Only RCP seems to be sincerely conservative.

    • Brian

      Sam is the KRS-One of electoral statisticians. He is true to the game, doesn’t sell out and because of that, he gets less shine. And that is my hip hop stats reference of the day.

    • trlkly

      I’d say part of it is a lack of a catchy name. Or a top level URL. I don’t see anyone promoting sites that are X.Y.tla anymore. (Or recommending anything .edu, for that mattet) It’s definitely not easy by word of mouth–it’s three parts that are atypical.

      The other thing might be that posts often directly call out pundits, and thus they are less likely to link. RCP is just numbers.

      And I think that the smoothing makes it less exciting, so people check it out less often. And the less often you go there, the less often you think about the site, which means fewer recommendations.

      BTW, I got here from a recommendation in the Facebook comments on 538. And that only happened because someone was saying 538 was too much of a “horse race.” And, even then, they didn’t link it.

  • E L

    Thank you, Sam. I believe you. However, to me Trump is so scary that a 1% chance leaves me with a deep chill. We have never seen a candidate with no govern experience in his entire life run for the highest office in his/her first election.

    • Michael

      That’s not quite true. Neither Grant nor Eisenhower had governing experience prior to running for president. Perhaps the fact that they were both commanding generals, both Republicans, and men whose policies would be unrecognizable in today’s Republican Party mean something, though what it is I can’t tell.

    • BillSct

      Both Grant and Esisenhower lead young American men to victory. It should be no surprise that those young men, grown older, would elect them to lead the nation. Military experience is government experience.

    • Michael

      Bill: Agreed, but Grant and Eisenhower didn’t just lead young men, they led the entire nation — under the leadership of the two greatest wartime presidents — to victory over the greatest threats the nation ever has ever faced. Funny that Trump cites Patton and MacArthur, both relieved of command for not understanding who is responsible for policy in our system. He also tells a vile lie about Pershing, which seems to excite his followers.

    • Michael Ralston

      It’s not the fact that Trump has no experience in government that scares me about him – in fact, it makes him marginally less scary to me.

      Experience is good, but… if he was at all good at selecting advisors and assistants, he’d able to accomplish what he set out to do without too many missteps, even without it.

      It’s just that what he wants to do is implement fascism, and he’s totally incompetent at judging people. That’s what’s scary to me.

    • scdennis

      Wendell Willkie was nominated by Republicans in 1940 with only corporate management background. He had been a Democrat until 1939.

      Also, William Howard Taft had only held judicial or appointive positions before Teddy Roosevelt picked him to run as vice-president.

      So, not entirely unprecedented.

  • Scott J. Tepper

    I’ve gotten involved in the California 49th where Democratic bête noire Darrell Issa is looking very vulnerable. I suspect there are a lot opf those races around the country, propelled as much by demographic changes as by an increase in the meta margin. The Democratic challenger, Doug Applegate, is a progressive former Marine.

  • Jacob

    I think that even if the lesson learned is that we shouldn’t visit PEC/538/Upshot/etc. breathlessly for POTUS polling updates, it serves as a nice corrective in the discourse. It was hard to find a right-leaning pundit in 2012 who would admit that Romney seemed hopelessly behind as election day neared. One hopes that if they get embarrassed enough times, they will begin to look and see just what kind of data they are up against.

    While we’ve seen some unreal stuff from the Trump side regarding polling, they’ve always received rather forceful pushback on their crazy polling claims. Even so, the talk of polls from the Trump camp has gone down massively since the end of primary season and you will see tacit admissions left and right about the fact they are behind. Without the history of poll aggregation I fear the discourse would be reduced once again to skewed polls and more egregious cherry-picking than ever.

    All of that said, I for the life of me can’t figure out why RCP is the go-to reference for averaged polls on cable news.

    • Matt McIrvin

      I think Sam got it: since RCP is a conservative site, conservatives won’t complain about it being biased, and liberals generally regard it as not too bad as a purely factual aggregator, so they’re OK with it too.

      Though I could see Trumpists complaining about it being too old-school-conservative.

Leave a Comment