A summary of the discussion follows. I will start with the key graph, which I produced in response to Joel.

**Is past performance this year a predictor of future dynamics? **Joel wanted to know about “in-sample variance”: is variance in the earlier part of a campaign predictive of what happens in the closing months? That would tell us whether it is kosher for me to use this year’s Meta-Margin history to estimate volatility from now until Election Day.

Jeremiah’s reaction says it well: *“I think of all the discussions this is the critical chart to consider….I think the way to look at this chart is to ask oneself what scenarios would point to upsetting the prediction? Even with all of the data the maximum SD for 1-90 days before the election is 4 percent and the average is much less than this. A SD assumption of 3 percent would therefore seem conservative. Also, there are no data points in the upper left quadrant of the chart and there is only one data point where the SD got much larger closer to the election and that was still less than 3 percent.”*

Bottom line: there’s no good justification for assuming that future variation will be greater than 3 percent. So I will keep it there.

In retrospect, for purposes of prediction, the graph above would have been enough. However, I think my point that polarization has come with entrenchment of opinion is still useful.

**Is 2016 different? **This leads to Mike’ general concern to my classifying 2016′s data as being similar to 1996-2012. *“I think a lot of people share an intuition that there is something about this race that should discourage us from grouping it with the other post-1996 elections in terms of volatility. It seems like it would be worthy to look for numerical support for that intuition, if only to see what the strongest argument is against the low-variability assumption.”*

Certainly I see the point of this objection. Donald Trump’s candidacy is so obviously freakish that surely 2016 is different…right? Actually, not really, from a data standpoint. The strong state-by-state correlation between Trump 2016 and Romney 2012 suggests that not all that much has changed, except that Trump is quite weak within his own party.

I see Trump as a culmination of a 20-year trend in the priorities and culture of the Republican Party. His tactics are familiar to the party base. For example, the questioning of legitimacy: of Obama’s birthplace, and of other Republicans, and even the November election itself…the list goes on. And yet he always had at least 40% of Republican primary voters on his side. I offer the following synthesis of data (2016 has been really stable) and events (crazy Trump): the U.S. is suffering from a near-fatal case of polarization, and Trump is a consequence.

**The Gary Johnson factor. **Several readers, for example NHM, raised the concern that this year, there are a lot of Gary Johnson supporters. Various hypothetical scenarios were laid out for how that could affect the race.

Here is a general way to think about Gary Johnson, who is currently polling at about 8%. Also, undecided plus alternative-party votes add up to 20.5%. The Clinton+Trump total is 79.5%, compared with 91.0% Obama+Romney on the same date in 2012. Because third-party votes are especially fluid in the home stretch, that could lead to more uncertainty in 2016 than in 2012. This is especially important because many of those voters are Republicans who might break toward Trump.

The maximum plausible range of what Gary Johnson supporters will do ranges from all going for Trump (i.e. 8% toward him) to maybe a 5%-3% split toward Clinton (i.e. net movement of 2% toward her). The approximate SD of such a range of possibilities is one-fourth of the total span. So SD_3rd_party =10%/4 = 2.5%. That’s still within the range of the 3% assumption.

>>>

There’s more good discussion. I encourage you to read it.

]]>This is an open comment period. Technical feedback is welcome. *(The comments section is rather good this time.)*

The Princeton Election Consortium’s prediction of November outcomes is based on the assumption that the race, as measured by the Meta-Margin, can drift in either direction between now and Election Day. The Meta-Margin, which is calculated entirely from state polls, is measured in units of margin between the two major candidates. Today it is Clinton +5.5%, which means that Hillary Clinton would defeat Donald Trump in the Electoral College, and polling margins would have to move across the board by 5.5% toward Trump in order to create a toss-up. If the change is larger, then Trump would be favored to win.

To understand how likely such a change would be, we have to have a measure of volatility during the general election season. That measure is the standard deviation. Since 1952, it has looked like this:

From 1952 to 2012, the two-candidate polling margin has had a typical standard deviation of 6% during the general election campaign. In other words, if you compare any two moments in time during the campaign, national polling margins at those two moments will usually be within 6% of each other – about two-thirds of the time, to be precise. If the two moments are close in time – for instance October 1 and Election Eve – the difference would be even smaller. This allows us to make a prediction about the future.

However, something changed starting in 1996. As I have written before, national politics in the United States became dramatically more polarized starting around 1994, when Newt Gingrich led Republicans to take over the House and Senate. Since then, as the graph shows, Presidential campaign dynamics have gotten much more stable. National polling margins have varied by only 3% on average. The Meta-Margin is even more steady, with a standard deviation of 1-3%.

The same point is obvious when examining the original time series plots of Wlezien and Erikson, whose data I used to make the graph above.

It is now apparent that 2016 is more like 1996-2012 than it is like 1952-1992. Data points for 2016 are included in the graph above. Even though the breakdown of the Republican Party and the advent of Donald Trump have made 2016 a crazy political year, public opinion is more stable than ever.

The Princeton Election Consortium’s initial assumption was that 2016 would be as volatile as typical campaigns since 1952 (SD=6%). This was a conservative assumption with lots of uncertainty. It was consistent with claims by pundits – and by Republican candidate Donald Trump – that the electoral map was scrambled. However, that scrambling has not materialized. Obama blue states are still Clinton blue states, and with a few exceptions, Romney red states are still Trump red states. This again shows that voters are highly entrenched in their views.

Here is what a “high variability” assumption (SD=6%) gives:

And here is what a “low variability” assumption (SD=3%) gives:

Starting today, we will use a lower-variability assumption. This makes the November win probabilities higher for the leading candidate, Hillary Clinton. The random-drift win probability goes from 78% to 92%, and the Bayesian win probability goes from 86% to 95%.

>>>

Finally, there is another step, which is converting the somewhat obscure Meta-Margin back into electoral votes, which is far better known. This has also been updated. Previously, the conversion was done using June polling data and 2012 election results, which led to a hard ceiling for Hillary Clinton because Romney’s wins were fairly large. Now that we have polls in more states, it has emerged that both Clinton and Trump have some fairly weak leads. That means that a small swing of opinion in either direction now leads to corresponding changes in EV. The “electoral strike zone” now reflects that possibility.

Going forward to Election Day, the electoral strike zone will be based on the latest polling data.

High-variability, with old Meta-Margin to EV interpolation:

Low-variability, with new Meta-Margin to EV interpolation:

The change not only makes a Trump win less likely. It also makes an electoral blowout by Clinton less likely. It might also make a switch in House control less likely.

By the way, originally I was going to wait until after Labor Day to make this modification. It would have been a logical time. However, the variability in national poll margin and in Meta-Margin is not going to change in the coming two weeks. Plus I am impatient.

Also…Labor Day is in only two weeks. Summer is nearly over, everyone!

*Update, August 22, 3:00pm: The best counterargument in the comments section is the issue of undecided/third-party voters. This year, that fraction is at 20.5% of voters; in 2012, the corresponding number was 9.5%. This raises the issue of whether this year is going to be like 1968/1980/1992/1996, years where the third-party candidate got 7-19% of the popular vote. In those years, the SD ranged from 4% to 14%.*

*Graph credits: Thanks to Prof. Christopher Wlezien of the University of Texas for the national polling data, which forms a major foundation for his book with Robert Erikson, The Timeline of Presidential Elections. The Meta-Margin data come from past calculations here at the Princeton Election Consortium.*

When the previous relationship was established in June, few states were weak for Trump; indeed, in many states we were reliant on 2012 election results, in which Mitt Romney ran stronger than Trump is now. That meant that opinion swings toward Clinton would not change the electoral-vote (EV) estimate much. That led to an asymmetric strike zone. In fact, there was a whole region around 342 EV where multi-percentage-point polling swings led to no change at all. Full disclosure, it was not an ideal calibration curve.

Now that we have fresh polling data, it is possible to recalculate the relationship between margin and seats in the prior. Trump leads are fairly evenly distributed, including some states where he barely leads. The same is true for Clinton. Therefore a swing of opinion in either direction now leads to corresponding changes in EV.

Note that this change does not affect the November win probabilities at all. That probability treats a bare EV win and a blowout the same way. The exact number of electoral votes does not matter.

*P.S. Yes, this aspect of the calculation – the conversion of Meta-Margin for purposes of November EV estimation – was not automated! It will be soon – one of several outstanding issues. I realize that it would have been more satisfying to see the change unfold gradually.*

Plotted below are median Clinton-minus-Trump margins in all states for which August polling is available, plotted against Obama-over-Romney margins at the same point in the 2012 campaign. This horizontal axis quantity is better to plot than the final Obama-Romney election margins, which include undecided voters’ final commitments – not a fair comparison.

The data come from RealClearPolitics.

Clinton is overperforming Obama in 15 out of 17 states, the significant exceptions being deeply Democratic Maine and New York. Overall, the difference is a median of 5.8 +/- 1.4% (estimated one-sigma SEM).

This is very similar to the picture in July, before the conventions. I wrote then that Trump needed to recover disaffected Republican voters in red states. At that time, his red-state leads were smaller than Romney’s 2012 win margins by a median of 9.3%. Hillary Clinton’s blue-state leads were smaller than Obama’s wins by 1.9%. The difference was about 7%, comparable to the value of 5.8% given above.

Recall that the popular vote margin in 2012 was Obama 51.1%, Romney 47.2%. If Clinton’s strong performance were to persist until the election, 12 weeks from now, the popular margin would end up at approximately Clinton 54%, Trump 44%. If we assume a disaffected 6% go to other candidates, that leaves Clinton 51%, Trump 41%, 8% Johnson/Stein/McMullin.

I find this quite amazing. After attacking Gold Star parents, advocating Second Amendment remedies, and other stuff too tedious to recount, Donald Trump will probably still end up with at least 40% of the popular vote. That is an impressive testament to the partisan polarization that has developed since 1992.

The other feature of the graph above is its correlation coefficient, +0.87. Voters are entrenched in the same positions as 2012 – and earlier, for as long as we’ve had the now-familiar red-state/blue-state arrangement. To most Republican voters, any candidate they field – whether it be Mitt Romney or Donald Trump – is preferrable to a Democrat.

]]>Game Theory Sunday. Today: Why is Evan McMullin running for President?

1) who is he? High-level House GOP staffer: https://t.co/zCxblD9AGo

— Sam Wang (@SamWangPhD) August 14, 2016

Today on The Takeaway, I discuss the impact of Evan McMullin’s entry into the Presidential race. His biggest effect will be downticket, where control of Congress is in the balance. The basic evidence: McMullin is a House GOP staffer, not a politician; and for every 1% of Republican turnout that he can salvage, they can recover up to six House seats – which could be crucial in determining control. As of this weekend, McMullin will be on the ballot in Colorado and Minnesota, where six swing districts are on the line (CO-03, CO-06, MN-01, MN-02, MN-03, and MN-08).

The show airs nationwide at various times starting at 9:00am Eastern. Find a radio station near you, or listen to it here.

Incidentally, activists on both sides are turning their attention to downticket races. Democrats, see ActBlue. Republicans, see the NRSC.

]]>You will need the free StatX app on your iPhone or Android (or search statx in Apple App Store or Google Play Store). Give it a try! Send any StatX-related suggestions and feedback to feedback@statx.io. Below, some answers regarding privacy concerns…

StatX tells me that they use the phone number as the user identifier in their system by sending an SMS to confirm the user’s identity. This is no different than WhatsApp and many other apps.

They get the camera and other permissions to allow users to upload photos to attach to a stat. There are some Android technical limitations in their permissions model (compared to Apple) that require them to get these permissions up front and some of these are bundled together so they end up asking for more than they use.

They say they keep the information secure and do not misuse it, as described in their Terms of Use and Privacy Policies, which are available on their website.

]]>For the generic Congressional preference, the displayed threshold for flipping the House is now set to Democrats +6% to +8%. That estimate is fairly rough, as detailed below. I am also experimenting with the graph format. It displays every Nth day of the median. Before, N=7; now I am trying N=3.

]]>