Princeton Election Consortium

A first draft of electoral history. Since 2004

Three Reasons To Ignore Debate “Expectations”

September 26th, 2016, 12:15am by Sam Wang


Polls are likely to move after the debate. It is the moment when voters get to make a direct, side-by-side comparison of the two candidates. This may also be the last time for any significant shift in the race.

Both before and after the debate, pundits will emit opinions about “expectations.” This commentary does not have predictive value. It would be better if they kept their focus on policy substance or factchecking.

Here are three reasons why you should basically ignore the onslaught of horserace punditry that is about to rain down.

Above is a mild example of what you can expect in the coming 24 hours. This particular statement is a bit circular: of course Trump will probably be perceived as “winning” if his numbers improve. However, there’s a bigger problem: the premise of “meeting expectations” itself carries no predictive value.

Trump could take the lead, but it would go against what we know so far. I would characterize the race as being very close, but not as uncertain as you might think. Why? The unappreciated story of 2016 is the amazing stability of public opinion. As measured by national polls, 2016 marks the most stable Presidential race in >60 years of modern polling. At the level of state-poll-based analysis, the stability is even greater. This basic fact should inform all analysis.

1. What commentators think about “exceeding expectations” is an anti-indicator. Via Will Jordan and CNN/Gallup, here is survey data on who was expected to do better in past debates.

Notice a pattern: all of these candidates ended up winning the popular vote. The explanation is simple: people can tell who the stronger candidate is, and expect that person to do well. As you can see, Hillary Clinton is narrowly expected to perform better tonight.

Now, think about this for a moment. What does it mean for a candidate to “exceed expectations”? It usually means that he/she was the weaker candidate to begin with. So if some commentator emits hot gas about either candidate doing better than expected, there is a good chance that candidate will be the loser in November.

2. If polls move after the debate, the reasons were baked in a long time ago. Since Phlegm-ghazi broke as a story on September 11th, national surveys have moved toward Trump slightly, then moved back toward Clinton. In polls that included September 12th, the day after Phlegm-gazi, the national margin was Clinton +1.5 ± 0.3% (n=8 polls, median ± estimated SEM). In polls that started on or after September 15th, the margin was Clinton +2.5 ± 0.7% (n=10). This is a pretty small move, but at a minimum it suggests that Clinton’s slight decline in national polls (and the state poll-based analysis that the Princeton Election Consortium provides) has stabilized:

Above, we have a starting point for anything that will happen in the coming weeks: a recent Meta-Margin of around Clinton +1.6%, with a median expectation of Clinton 294 EV, Trump 244 EV.

The overall stability of the 2016 race suggests that compared with today, polls are more likely to move back toward their past average than away from it. Think of public opinion as being like a rubber band: the race has some set point, like a thermostat. If opinion gets far from this natural equilibrium, it gets drawn back to that. This kind of dynamic is often called “regression to the mean.” It is a core assumption of the PEC model.

Given that tendency, I will make a prediction that may sound odd: I have no idea how Clinton and Trump will perform in Monday’s debate. But based on the regression-to-the-mean principle, I expect polls to move toward Clinton in the 1-2 weeks after the debate. Of course, if conditions move in the other direction, come on back and mock me!

3. Polarization has made it difficult for opinion to move much. The volatility in this year’s Presidential race is smaller than any race since the advent of modern polling. This is a symptom of severe political polarization, in which most Trump supporters would never consider supporting any Democrat, including Clinton; and vice versa.

Consequently, opinion is nearly immovable. Whatever happens in the coming weeks, the size of the the change would not have attracted notice in elections preceding 1996.

The groups that may choose up sides are self-described undecideds (4%), Gary Johnson supporters (8%), and Jill Stein supporters (3%). Undecideds and Johnson supporters are likely to split evenly between Clinton and Trump, while Stein supporters should break heavily toward Clinton. Tomorrow is a chance for them to get on the bus.

Tags: 2016 Election · President

93 Comments so far ↓

  • Daniel Rosenblatt

    I was wondering if there was any correlation between convention bounce and who “wins” the debate (in the sense of poll movement, not pundit evaluation). Basically, I wonder if both are simply based on how people react to candidates when they are actually looking at them and paying attention to them.

  • TDubs

    OK, I went to dinner, come back and the Metamargin rose +2.4 and the win probability is 87%. What happened? Ipsos?

  • Brian

    I’d kill to get Nate Silver and Sam on a stage to debate their models. I’d love to see the exact data behind Sam’s argument that the polling data is the most stable it’s been in 60-plus years, and Silver’s that essentially suggests the exact opposite. Both are smart, and make those statements as fact, but they can’t possibly both be right. Without the specific reasoning behind the argument, though, it’s hard for a layman to tell who’s wrong.

    • Sam Wang

      Okay…but my side is not rocket science. You can test it yourself. Do the following:

      1) get the data
      2) write down the Clinton-minus-Trump margin for each date at two-week intervals
      3) calculate the standard deviation of these margins.

      You will get a number that is well under 3%. That is low, period. To compare with past elections, use the Search box on this site, and look my posts on Wlezien/Erikson data and “sharpening the Presidential predictions.”

      Maybe he is counting pollster-to-pollster variability, which does not count as true variation in voter sentiment.

    • truedson

      I remember in 2012 when both were on NPR for a program about a month before the election …a congenial show if I am right. Very good too.

    • Amitabh Lath

      If you look at the History of the Meta Margin graph it’s quite clear. The MM had been around +4 for a long time. Then in mid-Aug it spiked up to above +6 and then plummeted to below +2 and is now just slightly higher than that.

      Math-wise, the long time (May-mid-Aug) being stable at +4 dominates the volatility metric. Gut-wise, the very recent spike up to 6 and dive to 2 makes it feel like things are volatile.

    • trlkly

      Sam Wang: I think Silver is just going by his own prediction model, which keeps changing. He sees the changes as volatility in voting rather than questioning his model.

      Given what you say, I would question his modeling. I do note that he uses national polling and demographics data to predict to fill in for State polls in between the gaps. Maybe that’s where the volatility is coming from.

  • AySz88

    Fleshing out something observed on Twitter:

    Sam noticed that YouGov’s tracking poll shows a really stable race on the two-way question. But someone’s reply pointed out that consecutive data points look too stable.

    It’s a weekly tracking poll, with advertised MOE is 3.9 points (variance around 4 points). If each week’s sample were independent, the difference between each week’s datapoint should have twice that variance (about 8 points), or a standard deviation of around 2.8 (mean deviation of 2.2). The graphs don’t show that sort of variance.

    My guess is that YouGov might not be grabbing a new random sample every week: it looks like they effectively draw a single sample, and the same (or similar) people take the survey over and over.

    This sounds fine to detect changes in preferences much more sensitively than drawing a new sample every week, but then each new edition doesn’t refine the estimate of the absolute proportion of support. (You can’t, say, average all the weeks and declare a MOE of one point or something.) It reminds me of “mean temperature” data in climate science, where the relative difference between two years, or difference between a year and a reference period, is a lot more certain than the absolute number.

    So this raises some questions for me: What does this do to the idea that this race is “more” stable than other years, and the idea that stability implies confidence? Is the apparent past instability just an artifact of those old polls drawing new (small) samples every ~month? And is that methodology creating a deleterious effect on the final projection that isn’t being accounted for?

    • Vince

      “My guess is that YouGov might not be grabbing a new random sample every week: it looks like they effectively draw a single sample, and the same (or similar) people take the survey over and over.”

      I think that’s exactly what they’re doing if I remember correctly. They’re not doing traditional polling and asking new people but tracking the thoughts of the same group of people over time.

  • Mary

    Just a fan, here! I listened to your podcast this morning. Although, much of it went right on over my head, a good deal of it was very informative. Thank you for keeping calm in the current wave of hysteria. Love your site. – Mary

    • Jay Sheckley

      Mary, I learn the most when I find something goes over my head. I look up words to learn more about what others might mean by them. Here on PEC, I also explore, study and read explanatory sidebar links. Then I listen/read again. And again.
      Even my pets like the Woocast Jazz.
      Maybe everything’s like a full immersion language course. Concepts accrete slowly around my mind, like layers expanding a pearl. Heck of an oyster we’re in this election. Bon appetit! -Ms Jay

  • Domingo Tavella

    Sam,
    Do you have any thoughts on the disparity between your win probabilities and the betting markets’?
    The betting markets have been consistently at about 60/40 for several weeks now, with minor ups and downs.

  • Steven Fondo

    NPR Story, Aug 2 2016.

    However, the four-day-long advertisements are remarkably good at setting the stage for November, as David Leonhardt wrote in the New York Times this year: “Since at least 1952, no presidential candidate has overcome a polling deficit immediately after the party conventions and gone on to win the popular vote.”

  • Steven Fondo

    What happened to the meme repeated ad nauseam in August by every polling analysis site that no presidential candidate in modern history has ever come back from a post-convention deficit (2 weeks post) to win an election? Just curious.

  • josh f

    sam again great article. your consistent quick-take style’s value is even more clear this year with 10x more being written about nothing.

    probably not enough data to conclude (and so many comments today couldnt’ see if already mentioned)…. but if only for fun couldn’t debate gap chart be a better indicator of margin than result?

  • A New Jersey Farmer

    Looks like Ipsos has done and pumped up the Meta-Margin with some interesting results. Michigan, Wisconsin, Ohio and North Carolina being the topline highlights. I know: it’s only one poll.

    Have they just reapplied a new band-aid and then ripped that one off?

    Happy debate watching all. By 9:45 it will look and sound like past ones. Unless Donald is losing badly.

  • Seth Gordon

    Hey, look, Clinton’s Meta-Margin is up to 2.4%—your post-debate prediction came true before the debate even started!

  • Davey

    Thank you, Dr. Wang. This article was precisely the sort of data I was curious about regarding the debates, and was much more informative than the circular discussions of the pundits.

    On a slightly different topic, many of us will be familiar with the USC poll, which differs in methodology from other polls. For one, they’re sticking with a specific group of respondents throughout the cycle. Setting aside partisanship or what this poll has produced thus far, I’d be interested in any insight on the mathematical implications of their methodology choices. Conventional polls run the risk of selecting a sample that is an outlier…wouldn’t using the same sample over and over run the risk of producing an endless string of outlier results?

  • Matt McIrvin

    I remember Kerry being generally perceived as having exceeded expectations in 2004. And his numbers did improve–just not quite enough to get him the win.

    • Damien

      It seems that pretty much every cycle the challenger has been thought to have “won” the first debate – and it always makes no difference in the end.

    • Matt McIrvin

      In 2008, I think the general perception was that Obama won all the debates–I guess he’d be the challenger from a partisan perspective–and his numbers did jump up every time, but I suspect that after the financial crisis hit it would have been very hard for him to lose.

  • Richard

    I know the Lichtman prediction has been beaten to death. Just wanted to note that Larry Sabato highlighted several non-poll based political science models and most if not all predict a close race in 2016, with a majority predicting a win by Dems. (Why didn’t the Washington Post do a story about all these models, rather than cherry-picking one to yank their readers’ chains? The question answers itself.) To the extent that the race looks like it will be close in the sense that there’s a good chance the popular vote spread will be within 5% (e.g. X 52 and Y 48 among those voting for X or Y) such models are doing well. However, most readers of this site, with no formal model, could have guessed that the race would be close in the above sense, which reduces the value of such models.

  • Roger

    Sam,

    Have you looked at early voting data much? So far three states have good data, with North Carolina having the most data. North Carolina is showing Democrat-registered early voting about 30% higher than it was in 2012, and Republican-registered early voting is about 20% down.

    I would think that overall, any specifically for North Carolina, this would be great news for Clinton.

    Thoughts on implications of this early voting data?

    http://www.huffingtonpost.com/michael-p-mcdonald/2016-early-voting-underwa_b_12184290.html

    • David Fry

      I’d want to know more about the R and D voting histories of those counties reporting absentee ballot requests. There are many areas of the south (such as in rural areas of FL and GA) where the great majority of registered voters are registered as Democrats but where Romney beat Obama by large margins in 2012. The early NC data could be bad news for Clinton if those conditions exist there.

  • anonymous

    The undecided category is the big stumbling block. Why would any person, at this point in this election year, be undecided? Thinking out loud:

    1. They are low-information voters, who vote based on gut feelings on election day.
    2. They cannot bring themselves to vote for either Clinton or Trump, but are not sufficiently tempted by Johnson or Stein either.
    3. They are leaning towards Clinton, but do not want to say so.
    4. They are leaning towards Trump, but do not want to say so.

    Obviously, number 4 is the most scary possibility. I also heard that pollsters switch to using likely voters instead of all voters in August. I wonder how much of a role this switch played in Trump’s rise in the polls.

    • Ed Wittens Cat

      numbah 5.
      the optimal game move at this point for youth and millenials is not to play.
      a Trump win will inexorably create backlash.
      Clinton wont give us a seat at the table– so take the table at midterms.
      that said, theres a whole lot of apathy– why bother to vote if both candidates will essentially govern just the same? both are corporatist wall street owned FP hawks enslaved by the status quo and “American Exceptionalism”.

    • trlkly

      I’m not sure 4 is the worst. There’s also “they want to vote for Clinton, but can’t bring themselves to do it.” I’ve run into a lot of people who seem to be like that.

      They also flirt with voting third party.

  • E L

    Take 2 Sam Wangs and call me in the morning.

  • Michael Levinsohn

    A couple of references have been made that if the Bayesian number is between 20% and 80% “anything can happen.” Is there a significant progression in likelihood, or is, say 79% really not that much different than 21%?

  • Trump+Democratic Congress?

    I’m a Trump supporter with a mind that is instinctually seeking to glom on to every piece of data, every interpretation and every speculation that is favorable to Trump. So, I’m not sure how much my speculations matters given my biases.

    However, I’m wondering if it matters that race has been stable over a period of time during which Trump repeatedly hasn’t met the bar for basic competence in a presidential candidate, thus alienating many traditional Republican voters (college-educated whites).

    Trump is walking over Clinton in the nation’s largest demographic, the white working class. If Trump can bring some of those college-educated white Republicans back into the fold, it could a problem for Clinton.

    • Chillax

      Pretty sure the nation’s largest demographic is women.

    • Ed Wittens Cat

      and in 2044 the nations largest demographic will be hispanics
      not that far off
      ;)

    • Matt McIrvin

      Concerning your nym: If there’s one thing these polls tell us, it’s that the one thing we are never going to have is Trump plus a Democratic Congress. If Trump’s in, he wins the whole megillah. He gets a free hand.

  • Ruth Rothschild

    Thanks for this article, Sam. It gives me some hope and some good perspectives, especially your comment about Clinton’s slight decline seeming to have stabilized. And, I sincerely hope that I don’t have to come back and mock you (which I probably wouldn’t do anyway). :-). There has been alot said about the expectations in this 1st debate being higher for Clinton than for Trump and how unfair that is. After reading your article, I think this is a good thing for Clinton. While people don’t trust Clinton because of the scabs (e.g., email server and phlegm-gazi) that Republicans continue to pick at in the hopes of swaying the vote toward their candidate, I believe that deep-down, no matter which party they affiliate with, people realize that Clinton is the stronger of the two candidates. My concern about tonight’s debate, which I’m not planning on watching, is that viewers who are in the undecided camp will be side-tracked by Trump’s showmanship, blustering, and diversionary tactics (i.e., known collectively as BS) to cover up his lack of true knowledge and understanding with the consequence that they won’t hear or pay close enough attention to the substance of each candidate’s responses. Hopefully, this won’t skew things so much that there won’t be a “regression to the mean”. I also believe that most people have already made up their minds about who they’ll vote for and that none of these debates will sway this group of people. It’s more the undecided voters who will be affected by the debates. Thanks for this site, Sam. I’d rather read a logical and objective approach than the subjective approach.

    • Ed Wittens Cat

      Im going to watch Teen Mom.
      I prefer my reality tv straight– that is unadulterated with political pretensions.

  • Tony Nickonchuk

    Hi Sam. I must say, following you and PollyVote and some other forecast sites, I’m becoming somewhat disillusioned with the 538 model. How can it possibly be so volatile? I’ve heard some arguments that it is volatile on purpose, to encourage traffic. I’m starting to agree with that and have come to the point where when one or two state polls come in and the chances of Clinton winning drops by like 5%, my eyes start to roll. Both you and PollyVote have been relatively consistent.

    I’ve also tweeted Enten and Silver repeatedly, pointing out to them that Silver himself called landline-only polls “inexcusable”, yet he includes the Emerson College landline-only polls in his model.

    So, anyways, that was an observation. Here’s my question. Given the huge polarization you talk about, which is painfully obvious to me as an outside observer up here in Canada, what do you think the chances are that the Stein and Johnson numbers are currently just parked protest votes? As in, when push comes to shove, a Democrat who can’t stand HRC will be so worried about Trump winning that they’ll hold their nose and vote Clinton and vice versa for a Republican who isn’t super excited about Trump? What are the chances that the third-party vote collapses?

    • Sam Wang

      You are asking a great question about Johnson/Stein. I do not yet have a way to answer. The high negatives for both candidates create new territory.

      Regarding FiveThirtyEight…okay, I admit that their update today has me flummoxed. I thought they basically had a model that was overhedged. But with a P(Clinton) less than 50%…what is the source of that? Kinda curous.

    • Bill

      I think that what we’re seeing at 538 this year is an over-reaction on their part due to how badly they missed on Trump during the primary. From what I hear from Nate Silver on his podcast and writings online it sounds as if he feels harried after that failure and is trying to avoid getting caught with his pants down again. Of course this is all my personal interpretation of the situation, I have no real insight into what is going on in his head.

      As far as the traffic generation argument, I’m skeptical as far as the model itself goes. I don’t see any benefit to them fudging data/calculations for desired outcomes to get traffic. If they got caught it would be a disaster and besides, they have a much better tool for generating traffic in their editorials. That is where I see a lot of the droning on about how incredibly close things and all of that.

    • Sam Wang

      I basically agree with Bill here. I am pretty sure their model is done honestly. But their essays do have to be shaped to appeal to their readership, which includes plenty of Republican voters.

      I don’t read Silver and Enten’s essays all the time. I find good nuggets…but they tend to get into the details, and then stop there. More big-picture synthesis would be interesting.

    • Roger

      On your second part, Drezner of the Washington Post makes a note on his model:
      https://www.washingtonpost.com/posteverything/wp/2016/09/22/nate-silver-says-i-should-be-nervous-about-the-election-heres-why-im-not-too-nervous/?utm_term=.0c7260d93654

      Basically he feels that Trump’s support is fixed, whereas Clinton is more expandable. He sees polls indicating that when Trump hit a high during his Convention, that many never Trump but lukewarm Clinton supports panicked and got enthusiastic about her. Then after her support peaked for a few weeks, they became less enthusiastic, toying with not voting or voting for a third party.

    • Matt McIrvin

      The pattern I’ve seen with 538 is that their model seems to try to extrapolate trends to some degree. That is, there’s some numerical version of “momentum” built into it–the now-cast particularly. (And then the polls-plus takes some of that out again.)

      It means that when the polls get a little volatile, their projections get really volatile. Looking at their models around convention time was a crazy prognostication rollercoaster.

    • Tony Nickonchuk

      Sorry Bill. I don’t mean to imply that the model is set up in order to nefariously attract traffic. But Nate himself has admitted his model is very “touchy” and responds quickly to small changes. I guess that’s what I meant. An overly sensitive model will cause wilder swings versus something more stable like Sam’s. Basically every time I go on 538 the number is different. I can guarantee you, for the poll junkies like me, that leads to increased traffic to the site.

    • Billy

      Their poll adjustments seem to be extremely volatile too. It’s supposed to adjust for house effects, but they’re non-uniform for the same pollster over time. Look at Quinnipiac’s poll adjustments in Florida, or for YouGov. The problem is that 538′s model is extremely non-transparent and non-reproducible (in addition to NYT’s), which begs the question as to why anyone pays attention to it at all.

      For all we know, 538 is hiring a bunch of interns to “unskew” the polls by looking at crosstabs, etc. Probably not, but we literally have no way to verify it.

    • Matt McIrvin

      Prediction: Hillary’s numbers are going to blip up a little now, and Nate Silver’s models will get insanely enthusiastic about a gigantic Hillary landslide, and Democrats will all love him again for a couple of weeks.

  • Ben Alpers

    Isn’t your first point not a reason to ignore “expectations,” but rather a reason to interpret them differently? You seem to be arguing that there’s a very strong correlation between expectations and a candidate’s winning the popular vote.

  • Larry Guy

    Just one word of caution regarding reliance on “regression to the mean”: it’s not clear that in a low poll volatility environment (v stable averages) regression to the mean is as strong a force as usual. Slow, inexorable drift may not self-correct.

    • Sam Wang

      Could be. I appreciate the quant perspective. To me this translates (among other things) to the question of what supporters of minor-party candidates will do.

    • Domingo Tavella

      This is precisely what we are seeing, in my opinion. The value to which the drift is moving is itself drifting as Trump becomes gradually more and more conservatives come home.

  • Ed Wittens Cat

    Data feast for Data Nerds — sentiment polling :)
    https://d25d2506sfb94s.cloudfront.net/cumulus_uploads/document/q8r0rkibs1/econTabReport.pdf#page10
    some amazing results that can only be ascribed red/blue brain hypothesis (brainscience)
    https://twitter.com/billmon1/status/780271801603592192

  • Olav Grinde

    Sam, I find your post very heartening! And I am totally on board with your prediction – I too predict that we will see a tidal wave of premature predictions. ;)

    I am also on board Alexandra Dixon’s prediction. Trump has unleashed some very ugly forces in our society, and they are not going away, nor is he.

    There is growing fragility and volatility in the financial world. We may well see a financial collapse that is far more severe and long-lasting than the 2007–2008 crisis. (Think the not-so-great Great Depression.)
    If Trump wins, we get proto-fascism now, amplified by whatever measures he takes to deal with such a collapse. (Addition prediction: He won’t be pointing the blame at himself.)
    If Clinton wins, Trump/Republicans/alt-right will be well-position to scream: “We told you so!”

    As the Chinese saying goes, we are indeed compelled to “live in exciting times”.
    .

    PS. Donald J. Trump may be the first Presidential candidate in history that has managed to turn his campaign into a money-making venture. Astounding!

  • Ravilyn Sanders

    This is what I really love about Dr Wang. He presents the data as it is. No unskewing or adjustment or anything. This is what it is. If he has personal beliefs of what is going to happen, he presents it separately as opinion.

    Taking care to distinguish data from theorizing is the hall mark of a true scientist.

  • Olav Grinde

    Sam, I find your post very heartening! And I am totally on board with your prediction – I too predict that we will see a tidal wave of premature predictions. ;)

    I am also on board Alexandra Dixon’s prediction. Trump has unleashed some very ugly forces, and they are not going away, nor is he.

    If I may make a long-term prediction of my own: Within four years we will see a financial collapse that is far more severe than the 2007–2008 crisis.
    If Trump wins, we get proto-fascism now, amplified by whatever measures he takes to deal with such a collapse. (Addition prediction: He won’t be pointing the blame at himself.)
    If Clinton wins, Trump/Republicans/alt-right will we well-position to scream: “We told you so!”

    As the Chinese saying goes, we do indeed live in “exciting times”.
    .

    PS. Donald J. Trump may be the first Presidential candidate in history that has managed to turn his campaign into a money-making venture. Astounding!

  • NKD

    Great article as always! One question: how does your finding that this election has the lowest polling volatility square with Drew Linzer’s tweet (https://twitter.com/DrewLinzer/status/779774654319689728) that volatility is higher than 2012?

    • Sam Wang

      That’s an interesting observation by Linzer. He asks if that’s about the state of the race or about the state of polling. Note that 2012 was itself an extremely stable race. Go look at my volatility graph, which suggests that his observation arises from polling variability.

      Note to readers: no more about the Lichtman keys thing please…just thinking about it is making me dumber. Your comments will not survive moderation.

  • Ken

    Dr. Wang,
    Do any forecasts and polls take into account voter turnout variances for each specific voting blocs such as Latinos, African Americans, and so obn?

  • Alexandra Dixon

    I think Donald Trump may be one of the few people in the Trump camp who doesn’t fervently care whether he wins or loses. Yeah, he has such a huge ego he has to win at everything but he’s already built in an escape hatch (“it was rigged!”) which many of his supporters will believe.

    I believe that all Trump cares about his his brand, of which he is his own biggest asset. Win or lose POTUS, he’s built his brand up in the process.

    My only regret is that he’s not going to go away after he loses. He won’t run for President again but he will glom on to any conspiracy theory or alt-right movement that he can spearhead, and attract followers. Not that he wants followers per se, not that he truly cares about any issue, but having followers gives him options – a radio show? another reality show?

    And meanwhile his toxic after-glow will poison political discourse for years to come because he has given a voice to the most deplorable impulses in the American heart.

    Sorry, this comment isn’t really about the polls or analysis thereof!

    • Gary

      This post states a common concern about the longer term impact of Trump on the electoral process. I agree that Trump will try to capitalize on a loss to his own benefit and to the detriment of our government. However, the post mortem on the election, which will be carried out for years, will have the advantage of sharp knives, detailed research, and no bias towards “fairness.” It will not be pretty, but the viewing the corpse will be beneficial.

    • 538 Refugee

      I wonder if the Republican primary system will survive Trump. I think we could sell them on instant runoff. The biggest problem is voting machine software I would think. There are open source groups working on ‘the voting problem’ that I’m sure would be happy to step in and help though.

    • Jaymes Winn

      If the GOP doesn’t radically alter their primary process, they’re basically surrendering the party to white nationalism and the White House to Democrats for the rest of the modern age.

      Which is why they’ll probably reform it to give a lot more sway to party elites.

    • Matt McIrvin

      Trump has a number of legal problems that might not have come to light were he not a presidential candidate. He’s weathered a lot of them in the past, so it might be nothing long-term. But some of them sound pretty serious.

      If he’s President, though, he can probably make them go away–the House isn’t ever going to impeach him, and might be able to pass helpful legislation.

  • trlkly

    So this race is close but not nearly as volatile as people think. It’s pretty steady.

    But, in the podcast, you said this would probably be a nail biter.

    So I am confused. I admit, I’m just a guy who actually decided to follow the election this year, and I’m no where near an expert. But I’m trying to learn what I can.

  • TJHalva

    This is my favorite example of stability…

    A plot of female respondents from New Hampshire:
    http://howwillamericavote.com/matchups.aspx?id=538&demographic=females

    Over more than a year, the support has basically been 50-33-ish. Same thing with Independents, but with slightly less stability:
    http://howwillamericavote.com/matchups.aspx?id=538&demographic=independents

    Mind you NH is supposed to be a “Swing State”; there have been 35 polls, Trump has led outright in a single poll.

  • Gelatinous_Cube

    Sam, you write,

    ‘Think of public opinion as being like a rubber band: the race has some set point, like a thermostat. If opinion gets far from this natural equilibrium, it gets drawn back to that. Statisticians call this principle “regression to the mean.”’

    I thought that “regression to the mean” was the observation that since a sample, or other stochastic event such as the continuity of traits across generations, has a random component, repeated events will tend to be closer to the historical mean than their predecessors. E.g. if I see a poll that deviates upward quite a bit from previous polls, it’s likely that a lot of the “upwardness” is due to nothing more than luck.

    The movement of polls after Phlegm-gazi and other events such as the conventions seems to be temporary, but are you saying the deviations from the mean are purely random?

    • Sam Wang

      In the statistical sense, yes, they can be treated that way.

    • Michael Ralston

      I think a high-level model that would make it look like there’s a set point is the following:

      Voters fall into one of four categories.
      1) Clinton voters.
      2) Trump voters.
      3) people who will never vote for Trump but don’t want to vote for Clinton.
      4) people who will never vote for Clinton but don’t want to vote for Trump.

      If we then assume that people mostly don’t move between groups, but people in groups 3 and 4 waffle between “yes I’m voting for X” and “no I’m not voting/voting third party/undecided” … well, you’d expect that events would generally cause some people to have something fresh in their mind (be it Phlegm-gazi, Khan-gate, “oh hey the convention made Clinton sound like a reasonable human being”, “you know, I haven’t heard Trump say anything racist for TWO WHOLE WEEKS”, whatever) to go one way, but mostly revert back to their state of “meh, I hate this election” over time.

      And then it kind of looks like category 1 is barely larger than category 2, and category 3 is significantly larger than category 4. Which then leads to a set point of “small advantage Clinton, but her numbers change more”.

      And I think under that model, devations from the mean would be pretty random?

    • DaveM

      @ Michael Ralston—

      Interesting analysis, but it seems equally possible that Clinton has a solid lead in the first two categories but no lead in the last two. If you take a look at HuffPost Pollster’s head-to-head matchup, with smoothing increased to make the point more obvious, you see that over the last month, Clinton’s numbers have been stable while Trump’s gained about 3 points.

  • Kevin

    I recall following the meta-margin over a decade ago. I was under the impression that it moves like a random walk (with regression to the mean), but that has noticeable jumps at the conventions, the first debate, and occasionally at other moments like Romney’s 47% moment, phlegmgazi, etc. Sam are you now saying that the first debate isn’t a jump moment? Am I mistaken that it was a jump in past elections?

    • David

      It’s called a post-convention ‘bounce’ for a reason. :-)

    • 538 Refugee

      Obama/Kerry was a ‘reason’ for Kerry supporters that were on the fence to ‘come home’. This year could well be the same for both candidates since they have such high negative numbers. I’m not sure who has the most to gain though. Trump has GOP deserters to bring back while Clinton has women and young voters to target. I think she has more potential upside, but that’s just a wild guess.

      Off topic. Clinton has some ads that might be very effective targeting women in Ohio. Could she be the recipient of the Bradley effect? Specifically white women thinking about their children when it comes time to actually cast a ballot and their husbands won’t know their rhetoric doesn’t match their vote?

  • Mick

    Your analysis here, I believe, is strong & sound. Readers should take heart, especially the newer ones (refugees from 538 and elsewhere). I myself (a political scientist, first in my family to go beyond high school) have dialled back my election monitoring in recent weeks, in part for sanity reasons but also because PEC’s modelling (like PollyVote) indicates a race that is actually quite fundamentally stable — if a little too close for comfort to most of us.

    I am more than curious about the aftermath of this election, especially if Trump loses (as expected). Had coffee tonight with my youngest brother, a prison guard (& former military paratrooper) who thinks Trump will win easily; my brother swears he’s seen recent polls that show Trump winning overall with 60% (!) of the vote. I avoided debate with him, smartly. But how many millions who (like my brother) sincerely believe Trump can’t lose and maybe, like Trump, think if he loses, the system is rigged. November 9 will be fascinating.

    • 538 Refugee

      Your brother may well have seen these results. I don’t know if you remember the ‘unskewing’ that took place last presidential election. Someone went through and removed all of the turnout adjustments from polls and used only the raw data. That told us that more Republicans answered polls but not much else. I’m sure people are doing it again even though it was proven a very bad idea.

  • CV

    Dr. Wang,

    Thank you for your work and analyses! I really value your approach and presentation.

    I had a question for you regarding the dissonance we are seeing between the national and state survey data, and whether you believe that changes your perspective on the outcome of the race. My feeling from the data appears that the situation can be read in one of three ways:

    1) State polls are lagging behind the apparent slight bump back towards Clinton, and will thus we may see a rebound in states in a week or so.

    2) National polls are poorly sampling, and since state poll aggregates are generally a better predictor of election day, maybe Trump is in fact > 270 electoral votes by a state or two.

    3) Clinton is over-performing in “red” states, but not by enough to actually beat Trump, whereas Trump is over-performing in swing states and is either within striking distance or is actually leading.

    If scenario 3 is correct, then Clinton might win the national vote you refer to as ‘consistent’, but the point is irrelevant since the presidency is decided by the electoral college as you know. Especially considering the degree of voter geographical segregation and polarization, how likely do you believe scenario 3 is?

    Thank you!

  • A

    Sam,

    Volatility hasn’t been great so far, but with the race being relatively tight right now, how concerned are you about the large percentage of undecideds and whether them breaking more towards Trump tips the equation?

  • ICM

    Do you lend any credence to the notion that the large number of undecideds and 3rd party voters throws an extra layer of uncertainty into these forecasts? That seems to be what some pundits with more “sensitive” models tend to point to when defending their fluctuations — outsize numbers of undecideds and likely voters who say they’ll vote 3rd party casts doubt in the forecast.

    Is that more so a flaw in the model where that isn’t baked in, or do you just feel that it’s largely irrelevant because undecideds and 3rd party supporters will ultimately break equally between D and R on Election Day?

    • Sam Wang

      It is an interesting point…but after reading this and doing some calculations, I think this year’s added uncertainty is less than 1 percentage point in terms of Clinton-v-Trump margin. I will write about that sometime soon.

  • VG

    I vastly prefer your data-only method to any “model” based predictions. But I would like to know what you think of Prof. Allan Lichtman’s non-poll based prediction?

    https://www.washingtonpost.com/news/the-fix/wp/2016/09/23/trump-is-headed-for-a-win-says-professor-whos-predicted-30-years-of-presidential-outcomes-correctly/

    • Lorem

      This was actually asked a little while ago and Sam replied: http://election.princeton.edu/2016/09/22/politics-polls-12-how-will-debates-move-the-race/#comment-102551

      My general impression is that any model-based approach that says anything other than “X is slightly favored this year, before considering polls” is overfitting and overconfident.

    • JPI

      It seems like a lot of his keys are fairly subjective observations. Some of the keys have to do with whether the candidates are charismatic and if there has been a scandal in the incumbent party’s administration. Seems like pseudoscience to me, but what do you think? He has been accurate for the last 30 years, but is it really a sound method?
      The commentary from him on his current prediction seemed a lot less than certain. Seems like he was going with the whole “Trump is a totally different type of candidate” thing to excuse himself if he’s wrong.

    • Frank

      Yeah, I would say the 13 Keys is mostly subjective. I took the test, and in my opinion, it could be 5 or less Falses (which thus favors a Clinton win). The first three questions are definitely false.

      The problem is, questions 4 to 11 become murky — Democrats will see one thing; Republicans will see another interpretation.

      Then you have 12 and 13 which rate the “charisma” or “national hero” status of each candidate. I bet there are many Democratic and Republican voters who agree to some degree that either term could apply to their party’s nominee. (And what exactly defines these two terms?) So if you flip both 12 and 13 to True, then that could only leave 3 Falses, which further puts Trump/the Republicans at a disadvantage.

      I don’t think this 13 Keys formula is working out so well for this particular election cycle due to the polarization of the electorate. The stronger the partisanship is in this country, the weaker this theory works, I’m going say.

    • Tapen Sinha

      Somebody noted on Twitter: Lichtman essentially has 9 observations and 13 parameters to estimate.

    • Amitabh Lath

      I wrote on this question last comment thread but after 15 years I’ve learned not to ignore a point that keeps coming back up.

      So let’s think about these models, with finite number of parameters and no public opinion inputs. If I can take 13 select variables and predict every American election as the country changes due to communications and transportation (telegraph, trains), immigration, depression, world wars, mass migration, civil rights, etc. etc. how much better should a similar algorithm be able to do in a homogenous state like, say, Japan, or the Scandinavian countries? And yet, there are surprises even in Icelandic elections.

      There is a reason the Physical Review stopped accepting papers on perpetual motion. You don’t have to dig into the details of every model to see it is bunk.

    • Sam Wang

      How dreary to answer and re-answer a question about a foolish meme.

    • slightly_peeved

      I think the simple answer is that while there may be some relationship between Lichtmann’s keys and the winner of a set of state polls on November 8th, there’s a far stronger and clearer relationship between the set of state polls over the past few months and the state polls on November 8th.

    • Froggy

      Also worth noting, from mid-August: “Lichtman added his model currently predicts Clinton will win about 52 percent of the vote because eight of the test statements are true.” http://dailybruin.com/2016/08/15/experts-predict-clinton-win-during-hammer-museum-lecture-2/

    • Sam Wang

      Wait, so it changes over time? That’s great. Covers the bases.

    • 538 Refugee

      I remember when I read it I didn’t find any proof of track record but I wasn’t going to waste a lot of time looking either. For all I know this is the first year he put it together correlating it to the old outcomes. Do we need a page just to keep track of these guys? ;)

    • BB

      I did the research so no one else has to.
      He’s apparently prospectively forecast the popular-vote winners of the eight presidential elections between 1984-2012. So he has that going for him.
      For 2016, Lichtman’s first prediction- Clinton win- made in Feb 2016.
      http://www.worldfinancialreview.com/?p=4865
      He coded five keys false then; since then one key ‘turned’ which crosses threshold for Trump win. The 3rd party candidate key, which I assume he changed based off Johnson’s numbers. (Note: that article was freely available right up until about last week, when suddenly paywall! Coincidence?)
      I do like the philosophy behind his model… but that’s neither here nor there. Back in Feb I read his initial prediction and noticed his interpretation on one of the keys directly contradicted how it’s supposed to be read, from [i]his own book[/i]. Cue facepalm.