Protected: Two ways to estimate primary outcomes without polls (transcript)
This content is password protected. To view it please enter your password below: Password:...
Looking ahead
Going into today’s election, many races appeared to be very close: 12 state-level Presidential races were within five percentage points. But the polls were off, massively. And so we face the likelihood of an electoral win by Donald Trump. At the same time, Hillary Clinton appears likely to win the popular vote. The Upshot’s model currently projects a Clinton lead of more than 1 percentage point. If that lead lasts, it means that more American voters preferred her to Trump.
At the moment, the NYT is projecting Trump leads of less than 1 percentage point in Pennsylvania and Michigan. Even without these states, Trump has at least 268 electoral votes (depending on some districts in Maine and Nebraska). We will see in the morning how these last few states and districts will be resolved.
In addition to the enormous polling error, I did not correctly estimate the size of the correlated error (also known as the systematic error) by a factor of five. As I wrote before, that five-fold difference accounted for the difference between the 99% probability here and the lower probabilities at other sites. We all estimated the Clinton win at being probable, but I was most extreme. It goes to show that even if the estimation problem is reduced to one parameter, it’s still essential to do a good job with that one parameter. Polls failed, and I amplified that failure.
Update: I believe a likely reason for polling failure is undecided Republicans coming home in the home stretch.
This election is about to create shock waves that will make the last year of campaigning look mild. We are about to see both houses of Congress under Republican control, quite possibly with a President Donald Trump. This comes in the face of a reasonably growing economy and a popular Democratic President about to exit the White House. It is difficult to reconcile these different facts.
Thinkpieces that have been written in the last few weeks have to be re-examined in a new light. Ezra Klein at Vox has written about the weakness in U.S. democracy, in which a weak Republican Party could nominate Trump, and partisan polarization gave him a shot at the Presidency. This one-two punch appears to have landed, hard. I was correct in documenting Trump’s rise in the primaries, an easier task for polling analysis because there, his lead was considerable.
I have written about the role of partisan polarization in getting voters to choose up sides, to the exclusion of even considering a vote for the other side. The chickens have now come home to roost. Exit polls showed that most voters felt that Trump lacked the temperament to be President, and that Clinton was seen as more qualified. Yet Trump seems to have rallied enough support to get overcome these factors. All Presidential nominees have had lower and lower approval ratings, and Clinton was no exception to the pattern.
Now we see where that long trend has led. One consequence is that more voters refused to support either major candidate. Neither Trump nor Clinton is headed for winning a majority of voters in Pennyslvania or Michigan. In Pennsylvania, the NYT projects that over 3% of voters cast their ballots for Gary Johnson or Jill Stein. In Michigan, the minor-party total was over 4%. In both cases, these numbers are considerably greater than the Trump-Clinton margin.
The coming years will be disruptive ones, to say the least. Whether you are Democrat, Republican, or neither, it’s going to be a challenging time ahead. It’s Donald Trump’s Republican Party, and maybe his Presidency too. The nation belongs to all of us. Good night.
Topics:
Helpful comment re: assessing voter preference.. Thanks for being on the Take Away.
http://www.wnyc.org/story/how-polls-got-it-wrong
(~5 minute audio)
Neither republicans nor democrats came up with anything resembling the results. Neither Brexit or non-Brexit supporters had polls resembling the result of Brexit. I wonder how polling, which has been fairly reliable for decades, has gotten everything wrong lately.
I wonder — to whom would it behoove having a weaker EU (which the EU will be without the UK) and who would benefit if and when the US leaves NATO, which Donald Trump has promised to do.
I wonder……nah. It’s not as if important elections could be hacked. Servers in the US and UK are not susceptible to such things, so how could electronic vote counting be hacked? It’s impossible. We don’t get deliberately attacked and undermined by other countries.
I’m a survey researcher but not in the political polling industry. There was no industry-wide practice of cooking the books. I can’t say that with certainty, of course, but it’s just ridiculous to assert that. Incentives are to get it as right as possible, to promote ones own method as legit, preserve ones reputation for accuracy.
There is, however, a bias towards methods that are easy to execute and make money on given the tools and apparatus available. These tools allow for marketing research and to understand the relationships among certain behaviors and demographics but not be well suited for estimating small differences in preference between two candidates. Sampling frame error (in this case, perhaps failing to appropriately capture rural voters less responsive to survey invitations in some way) cannot be overcome by simply increasing sample sizes or multiplying by several surveys/companies being executed in much the same way. One putative benefit of aggregation (like Sam’s and like 538’s) is that it can leverage the diversity of methods employed, but in this case the diversity of methods was fairly limited. Or at least in retrospect, it wasn’t sufficient.
I take issue with SW’s characterization that this was “enormous” polling error. The degree to which the polling average differed from the actual result was well within historical norms, even if at the high end of the range. The only thing enormous about it was the number of polls used to generate the polling average, i.e. the number of people wrong, not the degree of inaccuracy. 538’s historical perspective on this, whence its higher projected uncertainty, turned out to be wise and correct. No use in overstating how “enormous” the error was, when the real issue was failing to account for the historical range of error and the limited sample of elections. And that’s separate from the issue of why the polls were misaligned with the vote, which I’m sure will be examined extensively.
I’m not sure why Florida is so much easier to poll than Wisconsin. What is the defining characteristic of the rust-belt that makes it hard to poll? Does this comport with presidential years past?
A coalition of states is trying to form to guarantee the winner of the popular vote wins the EC. I think they have 165 EC votes pledged at this time.
http://www.nationalpopularvote.com/
Vox makes an interesting observation here:
“The polls were quite close in their forecast for Clinton’s support. Clinton was expected to get 47 percent of the vote; she’s got 47.7 percent so far. On average, the polls understated her state-level support by about a percentage point.
Trump, however, was expected to get 44 percent of the vote, and he now has 47.5 percent; the polls undershot his support considerably.”
http://www.vox.com/mischiefs-of-faction/2016/11/9/13574824/polls-clinton-trump
Focusing on percentages is misleading. Overall, Trump got the same number of votes as Romney or McCain to a first approximation. Voter turnout was drastically lower than in 2008 and 2012. It is more probable that likely voter screens failed.
@Arun
“It is more probable that likely voter screens failed.”
It is totally this. For some reason in the Rust-belt states people who said they were going to vote for Clinton actually ended up not voting for her and/or staying home.
@Arun, @Jeremiah–
How do you explain that the same polls had Gary Johnson at 5% yet he only got 3% of the popular vote?
Likely voter screens may have failed also, but it sure seems that Donald a good chunk of votes that were expected to go to Gary.
Vox tries to explain why the Rust Belt polling is a complete mess.
“The whole region was something of a black hole for pollsters.”
http://www.vox.com/policy-and-politics/2016/11/9/13574724/donald-trump-polling
Michael Moore’s 5 point morning after action, point 2:
2. Fire all pundits, predictors, pollsters and anyone else in the media who had a narrative they wouldn’t let go of and refused to listen to or acknowledge what was really going on. Those same bloviators will now tell us we must “heal the divide” and “come together.” They will pull more hooey like that out of their ass in the days to come. Turn them off.
***
Moore had correctly predicted the race in his classic article from summer:
http://michaelmoore.com/trumpwillwin/
@SKPlatt: Intersting reading : the article from Moore from the summer. He obviously had his finger on the pulse very accurately!
Where can one find the other four points from Moore’s most recent post-election missive? I looked for it on his web site, but could not find it.
@Michael Hahn
Here is the link:
http://www.alternet.org/election-2016/michael-moore-morning-after-do-list
Sam, tough election. You’ve been great to read for the last 12 years. Loved reading you in 2004, still a joy today. I’ll be back next election and interested to see how you plan to tackle the poll uncertainty problem.
“Jonathan sighed. The price of being misunderstood, he thought. They call you devil or they call you god.”
Chin up, Sam. Get ’em next time.
I will be interested in your analysis of this mess. If polling had been more accurate, the Clinton campaign could have allocated resources more effectively.
I, too, am getting ready for SfN. I don’t know if you like sushi (do ama ebi count as bugs?), but Sushi Ota on Mission Bay Drive is one of the best places ever. I’ll be headed over there right away. They take reservations at the sushi bar. A great way to take one’s mind off politics.
Some of you are so angry at Sam for not being dead-on this time that you are missing the bigger picture. Here’s what my FB friend Jeremy P. Carlo (who teaches physics at Villanova) had to say about it:
Spending some time looking at the polls, to see how they got it wrong. Interestingly, the national polling doesn’t look to be too far off; what they got drastically wrong was the state distribution.
Most of the final polls had Clinton winning the national popular vote by 3-4%. Nate Silver’s final tally was a 3.1% Clinton margin.
Currently, Clinton leads Trump in the NPV by 0.2%. But, once all the absentee ballots are added in Clinton’s lead will probably go up to ~ 1%, maybe even 2%. (This could be right or wrong; we don’t know until we find out.)
If so, the national polls were only off by about 2% (3% if the absentee ballots don’t move the totals from their current position), which is significant, but not “yugely” bad. What they got wrong is where those voters were located.
Clinton overperformed her polls (despite underperforming nationally) in a string of solid blue states: DC, HI, CA, WA, MA, IL, NM, NY, OR, NJ, and VT. She even overperformed Obama 2012 in some.
Trump overperformed his polls – in some cases by double digits – in a string of solid red states: WV, SD, TN, ND, WY, ID, KY, OK, MO, KS, UT, NE, AK.
Add those up, and it’s mostly a wash, but probably accounts for Trump beating his polls by ~2% (call the range 1-3%) nationally.
This is simply a statement of the fact that our population is becoming more and more geographically segregated: red states are becoming redder, and blue states are becoming bluer.
However, Trump strongly outperformed his polls in a string of closely balanced states: by 4.5% in NC and MI, and by 6-7% in PA, OH, WI, and IA, giving each to Trump. (All but OH and IA were forecast to go to Clinton.)
Trump overperformed his polls in FL by 2%, close to the putative national average overperformance, but since FL was predicted to be such a squeaker beforehand (less than 1% margin), that was enough to flip FL to Trump’s side.
Those five states (the above seven minus IA and OH, which Trump were predicted to win beforehand) add up to 90 electoral votes. That’s the difference between 306, Trump’s anticipated total (assuming he wins MI and loses NH), and 216, the number many (myself included) predicted.
It’s worth noting that, with this vote distribution, even if the national polls were dead on, he still would have won all those states but NC, for a winning total of 291 EVs (and he would have lost the popular vote by 3%). He could possibly even have underperformed his polls by about 1% and still squeaked by – barely winning FL and barely losing PA for 271 EVs (and losing the NPV by perhaps 4%).
So, it was kind of a “perfect storm” of getting the big picture fairly well, but missing the finer details.
Sam – Thank you for your efforts in showing us the power and at the same time the limits of the polling industry. We are all stunned and this has lessons for all. No apologies needed, *you* did not cause the upset. The sun rises again and we will arise anew with it!
Professor Wang – I think you owe it to your readers to eat the promised bug. Maybe two of them. And post a picture. How about it? – THG
I think you should still eat a bug, but it can be a delicious one.
Dr. Wang, I just wanted to thank you for this site. In the months leading up to the election, I visited this site to get a bit of peace of mind and encouraged friends of mine to do the same.
I don’t blame you for getting it wrong. Apparently everyone got it wrong, either because Trump Voters were to ashamed to admit who they were voting for, or they were intentionally concealing it in order to trick pollsters, the fact is that Trump, being of primal chaos that he is, managed to pull a fast one on everyone.
Now we have to survive the next four years, vote in the midterms, vote in the next elections, and hopefully stop Trump from doing too much damage.
But again, Thank you Dr. Wang.
Small solace but it looks like you got the popular vote to meta-margin gap correct (2% approx) and the popular vote to Senate gap (3% approx).
So the popular vote, state and senate polls were all off by 3% on average. In the UK 2015 election the miss was partly caused by differential turnout and I suspect that was the case here. And as in the UK it was not “shy” right-wing voters but apathetic left-wing likely voters who didn’t vote that may be at fault.
A lot of lap dogs here but we do keep score . You finished last and your brand is Sam 99% Wang unfortunately for you . Welcome to the Major League where performance is what matters
Look at this white college graph– poll aggregators seemed unable to capture late movements in white college educated voters toward Trump.
https://twitter.com/adrian_gray/status/795385379197161472
lay off Dr Wang, it wasnt bad math, it was bad data.
and i guess motivated reasoning and Wang Polarization Theory.
The college educated and the intellectuals and the media simply could not believe that America would elect Trump.
This is a wake-up call for the blue half of America.
And HRC ganked us too– her VP choice signaled she didnt give a rap abt Bernies supporters. The old model for presidential campaigns was lock down ur base and pivot to the middle. No more, the Trump model is lock down ur base and then bread and circuses every day. The media is wholly complicit in this.
Trump told his base “I am ur last chance”– probably true. They took it.
Now the democratic party is a smoking ruin and we have a president who says stuff like this and is gunna kerb stomp the EPA.
https://twitter.com/KilloughCNN/status/687882626548412417
unsubtle message to the global media– say what i want or ur cut off.
So America gets the president we deserve– thats inherent in the design. The #nevertrump protest movement is now the analog of the Tea Party Patriots. warriors of a rump party. hope they can drum up enthusiasm for the midterms.
So what happens when Trump cant deliver the good manufacturing jobs he promised his base? Does he crash the global economy, start WWIII or just fake a Reichstag Fire?
We just elected a pathological narcissist to the high office.
You’re welcome.
This just occurred to me… in a year or two i may not be able say something like that publicly.
Trump is gunna inherit Obama’s massive surveillance state.
Help us Moxie Marlinspike, you’re our only hope.
And just think how great WWIII would be for the US economy.
Flynn for sec state or sec defense?
https://twitter.com/POLITICOMag/status/796648380369879040
The USC/LA Times Tracking Poll seems to have gotten it right by factoring in folks who didn’t vote 4 years. It allows them to rate, on a scale of 1-100, their likelihood to vote this year.
http://www.latimes.com/politics/la-na-pol-trump-disaffected-voters-20160831-snap-htmlstory.html
In finance we like to say that model building is part art and part science. I have no doubt you are strong in the latter, but you get an “F-minus” in the former. In the past two months, my instinct told me your probabilities were absurdly off base, especially when they reached 99%, even before Comey said on Sunday that the new investigation had revealed nothing of substance. The dangerous thing is that a total charlatan like you developed a large following and one only can wonder how it contributed to the complacency among the left that Hilary’s “got this thing.” If you are to continue with this charade, may I strongly suggest you recruit some help for the art side of your model building because you clearly didn’t have any clue regarding the assumptions, which are the key to any financial model. Now now at least a part of the bloodshed that is likely to result from a Trump presidency, both here and abroad, is on your hands. Your arrogance and huge ego both astound and disgust me, sir. Also, Nate Silver ate your lunch.
Sam, I thank you for being the best poll aggregator ever. You developed a brilliant method for combining polls from diverse pollsters in different states to estimate the outcome in the electoral college. It’s been wonderfully useful to me, as to many others.
In this election the polls were off. Way, way off. Some analysts had a gut feeling that the polls were off, and put a thumb on the scale to make predictions that were more in line with their gut sense of the outcome.
Back in the 1970’s when I was a grad student at Princeton, econometric models were used in this way–econometricians put in adjustable parameters, as well as estimated coefficients; then adjusted the parameters in order to make the models predict what the economists expect.
You kept your thumb off the scale. Your aggregation methods gave a steady reading on what the polls were telling us.
Unfortunately the polls were very far from an accurate reading of the pulse of the body politic.
The failure is not in your aggregation methods. The failure, quite clearly, is in polling. The pollsters’ methods, apparently sophisticated as they were, were as far off as the naive newspaper polls of the Dewey-Truman election.
Poll sophistication amounted to sound and fury, signifying nothing–like the creation of epicycles to account for the motions of the planets.
Your methods make it very clear that polling methods do not accurately capture voting intentions or real voting behavior.
If you decide to move forward, developing methods for assessing which polls are good predictors of voting behavior, I bet you will do a brilliant job.
And if you decide to stick to neuroscience, helping us to understand how our brains/minds work, that will be deeply helpful also.
The future is out there somewhere; here we go; where to? — Partly we can make a road map; partly we’re wandering around, making new trails in unknown territory. We make the path.
Again, thanks.
The polls were wrong – garbage in, garbage out. Please don’t eat a bug
I don’t think it was your prediction that was wrong, Sam. You said it yourself. If major storms happen, then maybe the election could be affected. Well…..fascism is a perfect storm, made only possible by the worst elements of the masses coming together with the worst elements of those in charge. It’s a cancer of government. No one could’ve predicted that America would lose it’s mind. This is a sociopathic expression of our nation, let’s make no mistake on that. All things considered (somewhat) normal, Hillary should’ve won. Food for thought….
I have enjoyed this site for the high level discussions. I hope it continues without snarky blaming and doomsday predictions. More analytical information will help in planning for the future. Thanks Sam- keep it up.
I second this! The site has been educational, and it DID keep me sane. You can’t blame the site for the incorrect data. We need this sort of attempt to have clear thinking during elections. Keep going PEC!
I also want this site to continue, but we need better source information, namely the polls themselves. I work in the automotive design and analysis field where analysis also relies on accurate basic information such as good material properties, etc. Those are often wrong, so cause design errors not visible until final testing. That’s what happened here, but in a very important national election in a superpower. Hopefully there will be an investigation into what caused the errors so that future predictions are improved. Polling is difficult because of the human nature to hide things they don’t want other people to know, so I’m not sure its possible.
… and Sam, please come back, and be our voice for good polling. Thanks for your work here on this site.
The ultimate irony is that with his 99% Clinton prediction, widely publicised at places like Huffpo, Sam might have lulled Clinton supporters into a false sense of security, and so actually been a contributing factor to the reasons she lost!
I think Sam acknowledged some shortcomings in his methodology (should have been 90%, not 99% probability. But the fact is the data was bad and everyone got it wrong, including Trump’s people. A number of sources, including
Trump’s UK firm, thought they were going to lose. I would chalk this up to polling malpractice, particularly on Clinton’s internal polling effort. They oversampled D’s. and undersampled R’s in reliance on 2012 voting data. In the middle of running all of these negative Trump ads, which were needed, they forgot to present a positive message to displaced workers from the upper-midwest. They blew the turnout effort in Milwaukee and Detroit, which would have made the difference in those states.
Indeed, there was a polling error. But a good analysis of the polls needs to take into account the possibility of such errors.
Honestly, I think that the big lesson here is that the “number crunchers” who forecast elections needs to have a more academic attitude towards their peers. Nate was criticized in 2012 for being over confident, in 2014 for being underconfident, and in 2016 for supposedly “inserting unnecessary uncertainty” into his model.
This was not a short election cycle. It was long enough for people to reflect upon their methods. Yet we can dig up articles from just a day before the election that basically lambast Nate for taking into account pollster reliability, adding unnecessary uncertainty into his models via correlated polling errors, etc.
The error here was not simply “some aggregators failed to take into account correlation” it was instead that “some aggregators refused to face a disagreement over methodology academically, and instead resorted to some kind of statistician-version of punditry”
Looking ahead:
http://www.sandiegouniontribune.com/news/courts/sd-me-trump-trial-20161109-story.html
I heard a talking head make an interesting assertion, and I’d love for Sam to comment on its validity. He said that support for Clinton had *extremely* steady all through September and October. He said that changes in the gap between Clinton & Trump were almost entirely due to people shifting between Trump, undecided, and 3rd party. The he said that this should have told pollsters that undecideds and 3rd partiers would break strongly for Trump. That is, focusing on the difference in support masked the low ceiling on support for Clinton.
I think that if Trump offered up Garland to fill the Scalia seat, he’d defang the protest. Fat chance, I know. But if he did, he’d be seen as being beyond partisanship. He’d get to 50% that way.
I generally don’t comment but read this site often and enjoy it immensely. It is an oasis of sanity in a sea of dubious methodology. And I want to thank Sam and his associates for performing a remarkable public service. Please continue!
Having said all that I would question whether the model used is effective or correct. One can say that the model works but it was just the data (state polling) that was poor. And while that may be true, the model depends solely on the median of state polls. If those polls are unreliable or inaccurate, then by definition the model is also ineffective. In the special case of a close election which this one certainly was the model may break down because the data is unable to be accurate. Just wanted to throw this out as we prepare for 4 years of utter disaster.
Michael Moore’s 5 point morning after action, point 2:
2. Fire all pundits, predictors, pollsters and anyone else in the media who had a narrative they wouldn’t let go of …
Some might suggest Michael Moore himself falls into this category.
Are you kidding? Did you read his article from back in late July titled
5 Reasons Why Trump Will Win
“http://michaelmoore.com/trumpwillwin/”
The article is astonishingly prescient, I’ll quote one particular passage from it as a TLDR:
“And this is where the math comes in. In 2012, Mitt Romney lost by 64 electoral votes. Add up the electoral votes cast by Michigan, Ohio, Pennsylvania and Wisconsin. It’s 64. All Trump needs to do to win is to carry, as he’s expected to do, the swath of traditional red states from Idaho to Georgia (states that’ll never vote for Hillary Clinton), and then he just needs these four rust belt states. He doesn’t need Florida. He doesn’t need Colorado or Virginia. Just Michigan, Ohio, Pennsylvania and Wisconsin. And that will put him over the top. This is how it will happen in November.”
Which one is he? pundit or a predictor?
No, Brian Weissman, I’m not kidding. All that’s demonstrated by your particular quotation from Michael Moore is that he can add up to 270. If that impresses you, all that I can say is that you’re easily impressed.
My understanding is that PEC’s model is pure polls; and that other models that included fundamentals such as change years, or in the case of some NY College students, were built on priors from 2012) didn’t just predict a lower degree of confidence but predicted a Trump victory outright.
Wouldn’t the correct approach, given the sparsity of data and the problems with polls, be to completely re-examine the model, and look for a model that fits the data – going how ever far back you deem valid? Fundamentals, polling, economic indicators – an agnostic search for what works.
Dear Sam,
Long time reader and very occasional commenter here. As much as a disaster this is, as a scientist, I really look forward to a good post-mortem here. Here are my speculations:
1. All the polls are wrong, n the same direction. Why? Shy trump voters do exist?
2. Maybe the polls are not that wrong, but in fact a large group of people changed their minds in the last week. Comey effect?
3. A large group of undecided voters decided in the same direction during the last week. You said before that undecided tends to fall evenly. But maybe this time it’s different. Again, Comey effect?
4. Noted by comments above, maybe a large group of supposedly unlikely voters decided to vote Trump. I guess they could either be classified as shy Trump voters or undecided.
Thanks for all that you’ve done. People are trying to say Nate Silver deserves credit for being the most doubtful. I don’t buy that. It’s much better to transparent and wrong, than to be vague and only right some times. We need good information, not voodoo.
I really enjoy the transparency and simple methodology used here. I’ve been saying and doing mentally and personally for years that we need to be using a bimodal distribution. By your pre-election post, the median absolute error in presidential results in the past 4 elections was 1.25%. So I would construct a probability distribution that combines two normal or t-distributions with whatever sigma you want, one with a mean 1.25% greater than the polling median and one with a mean 1.25% less than the polling mean. This should result in two modal predictions and significantly greater variance, but would give an average polling error of 1.25%. Your final state meta-margin was off by 3.2% I believe. A simple t-distribution with 3 degrees of freedom gives a 1.9% chance of that. The bimodal t-distribution with a 1.25% built in error gives a 3.9% chance.
For Senate elections… just as a guess I’d say the midterm error should be around 3.1%, the historical median. I’ll claim for Presidential years, maybe we should just use median Presidential meta-margin error, because the median Senate margin error seems very low. The median (directional) error among the 15 states on this page is +2.2% R. That would have about an 11.6% chance of happening in a bimodal model and a 5.4% chance under a t-distribution with 3 degrees of freedom. The bimodal distribution gives as the most likely outcomes based on polling 51 R seats, 49 D seats (although PA is D and NH R) or 53 D seats, 47 R seats. Similar to this year, in 2014 the Senate outcome extreme.
Another question with your projections is that they do not really include regional correlations in errors which do appear to be a thing.
If we learn one lesson from this polling miss, can it please be that we should all have greater respect for Nate Silver.
It seems like the general lesson of: Nate knows what he’s doing, is lost every 4 years.
I really hope that 4 years from now Nate Silver isn’t labeled as “inserting uncertainty into his model to get clicks”
This was not just a technical mistake on the part of pollsters, this was a professional mistake that failed to pay colleagues with contrary opinions due respect and deference.
Given that 6/10 voters believe Trump does not have a temperament to be President, I wonder if his approval rating will ever cross 45% mark.
I had a similar thought recently. When you combine current levels of national polarization with Trump’s uniquely high unfavorables, it’s hard to see, absent some national crisis that at least temporarily brings us all together, how his approval ratings will ever get much above 50%–especially if he makes good on even some of his campaign promises.
Moore’s First Law:
1. Midwest Math, or Welcome to Our Rust Belt Brexit.
written in summer 2016, in an article that made a *hard prediction* of a Trump victory.
http://michaelmoore.com/trumpwillwin/
Please – there are limits to how “mathy” elections are. I’m a physicist and I do not think economics, human psychology, or polling are at the level of “mathiness” yet to ascribe 99% probabilities.
It is an infantile, touchy-feely field. *Not a hard science*.
Case in point: the large ~ 13% undecided voters were not taken into account by Dr. Wang. Undecided voters that late are likely to throw a brick and they did.
Case in point 2: Polls target “likely” voters. Likely voters are traditional voters who are “likely” to give a skewed picture of calm when the country is in a revolutionary mood.
Moore got it right because he’s from the devastated mid-West, has been criss-crossing it for decades and read the mood of the people. He *is* one of them.
Please, do not diminish the sciences or areas likely computational neuroscience in the minds of the public.
It’ll. The mainstream media will grade him on a curve, making allowances for how much worse he could be. “At least he has not called Merkel an ‘Islamic whore’”
No post for the new podcast episode?
In the episode you talked about two possible reasons for the polling miss: 1) undecided voters breaking for Trump and 2) missing non-college-educated whites.
There’s been a lot of talk the last few days about the difference in raw votes between Obama/Romney & Clinton/Trump — Clinton seems to have lost a good deal more voters compared to Obama than Trump compared to Romney. I’ve seen talk of Clinton being unable to reassemble the Obama coalition, particularly among minorities and younger voters — do you think there’s anything to this idea?
Yup.
Clinton was a competent and well-credentialed candidate, but a clear establishment choice in a very anti-establishment climate, and a maladroit campaigner to boot. There was nothing aspirational or inspirational in her campaign. Her entire argument was: I’m not Trump. In other cycles, that might have been enough. It wasn’t this time.
Voter suppression tactics might also have made a difference at the margins–she only lost Wisconsin by less than 30,000 votes out of almost 3 million, and if voting were as easy there as it is in other states, she might have won. Same with Michigan. In a country where voting is much easier than it is here, I think she probably would have won.
But that’s not the country we live in, and to me, the main takeaway is that she received 10 million(!) fewer votes than Obama did in 2008, despite there being millions more registered voters. I think other commenters have it right: Republicans probably can’t get much more than 60 million votes at this point, especially because their base is shrinking. But they don’t have to worry about turnout–their voters will show up no matter what, as you can see over the past few elections. Democrats, on the other hand, have to have a reason to show. If they do, they’re highly likely to win. If not, you’ll probably end up with a Gore-Bush or Clinton-Trump outcome.
That sir, is the essence of this, and both Republicans and Democrats know it. Longer term, if they don’t appeal to younger voters and minorities, they will get substantially less than 60 million votes. Long term, they have no choice but to be inclusive. But Trump and McConnell are clearly not thinking long term, and have no reason to. Ryan probably does.
Perhaps the voters missed by the polling companies were not “undecided” but Democrats crossing over – now that would be a real monkey wrench if correct!
http://www.npr.org/sections/alltechconsidered/2016/11/10/501613521/an-app-saw-trump-winning-swing-states-when-polls-didnt
I suppose it wouldn’t be THAT shocking. There’s a fair bit of populist meat in his first 100 days plan. Term limits, tarriffs, anti-lobbying, etc. If you’re a blue collar out-of-work-er, you could get from Trump what you wish Hillary was offering.
This, along with the Guardian article about liberals and Trump, is the best take I’ve seen so far:
https://www.youtube.com/watch?v=GLG9g7BcjKs
It was the drama, not the data.
Democrats simply did n0t come out in numbers they did when Obama was running.
2008- Barack Obama- 69,499
2012- Barack Obama- 65,916
2016- Hillary Clinton- 59,938
A more detailed breakdown shows that the Millennial and African-American vote was down about 5% from 2012.
I think we Democrats did this to ourselves.
That would be in millions.
Exactly. A major failure of D’s to show up. However, for comparable numbers you do need to factor in third part candidates. R’s are pretty much flat at 60M for each of the three elections, so they are really not gaining much.
That’s the key. On the Republican side, the number is always 60 million give or take a few 100K. The difference between defeat and victory:
1. Wisconsin 27,000 votes (10 EV)
2. Michigan 12,000 (16 EV)
3. Pennsylvania 68,000 (20 EV)
To be fair, Trump outperformed both McCain and Romney in PA, but not Obama.
The key here is that Hilary was a competent and experienced person – which is fine for job interviews, but clearly lacked the inspirational appeal of her husband or Obama. Victory or defeat is controlled not by Republican turnout, which is relatively constant, but Democratic turnout.
Based on these numbers, it makes total and complete sense why Republicans would focus so much energy on voter suppression.
If the culprit was low Democrat voter turnout because of lack of enthusiasm for HRC then the question is how would Bernie have fared? Would he have kept the women’s vote?
By the time the Republicans had campaigned against Bernie as that Commie Jew, I don’t think he would have come out too well.
Much gloom and rightly so. But Hillary appears to have won the popular vote. Enough people voted for her. They just weren’t in the right place. It’s beginning to pain me a lot that my California vote is worth so much less than voters in some other locations.
The theme about enough people voting Dem but not in the right place is a recurring one. Of the last 5 presidential elections, Democrats have won the popular vote in four, but the electoral college in only two. They need to be in the right place – in swing states. Obama was able to inspire this turnout. Hillary and John Kerry were not.
Two weeks ago I posted on Sam Wang’s site a question about the quality of the data used to produce a Bayes confidence level of >99% for Hillary’s election. I simply asked a question using my own experience about polls generalized. I do not respond to polls either by landline (yes I still have that) or cell, for my own reasons. I speculated that others might non-respond likewise for reasons quite orthogonal to mine. And so I questioned the database for aggregate polls, and received no response.
With 40 hours of sleeplessness between Monday and Wednesday, I got my answer.
National polls are more reliable because they sample attitudes across the political spectrum and tend to wash out regionally isolated cultural outliers by ranging across a much larger sample. But state polls cannot do that, and so any culturally related reluctance of response is magnified as potential error by muddling the data. Perhaps some states for cultural reasons are more opaque than others in this way–concealed racism, sexism, or whatnot. That seems to be the case for my own state–WI–and MI and PA as well–especially when the outliers become statistical out-liars. And since states dictate electoral college results rather than national polls, I realized way too late why things played out as they did. Politics are indeed local–especially statistically.
Alan, I don’t believe your assertion about national-versus-state polls is correct. A state-specific sample should capture in-state heterogeneity better than a national sample of a similar size captures nation-wide heterogeneity. And besides, in-state polls can stratify by the specific dimensions present in the state — it’s easier for those pollsters to avoid the pitfalls by failing to ensure the appropriate number of respondents from an odd county or district, for example.
That’s not to be confused with huge nation-wide polls that capture enough respondents within each state to weight and report state-wide estimates. My guess is that those ended up with farcical levels of error, but we’ll see. (Looking at you, SurveyMonkey.)
State polls aren’t great but we knew that.National polls are better because it’s an easier sample to get. Most models (and polls) fail at predicting the percentages for the candidates but since it’s about picking winners it doesn’t matter. So the 98% accuracy model in 2012 done by Nate Silver and Princeton may have only gotten one state wrong, but there intervals were off in more than one state. This is why predicting elections in other countries is far more difficult because you may need to get a percentage that is accurate. The polls are always wrong, but in 2016 they were more wrong than usual.
I agree with your distrust of polls. I also share your propensity for non-response. (Our home is in a dead-zone pocket for cell, although we live in a very urban area. We also still have a land line, albeit Internet based. The “cordless” base unit plugged into it has vocal caller ID and we answer only recognized callers.)
I worried throughout the campaign season about correlated error in the polls. Still, I felt relatively comfortable when the Meta-margin was near and above 5.0. But, as the M-m steadily eroded over the last half of October my alarm increased. However, Clinton surrogates spending lots of time in Arizona buoyed my level of condidence. But when the M-m kept on dropping day after day, and then dropped below 3.0, a real sense of worry set in. Watching the Clinton rally in Philadelphia on election eve, the Obama’s impassioned pleas to Philadelphians conveyed to me a real sense of last-hour desperation. Particularly with Clinton headed to an even later rally in NC. I stayed up Tuesday until the campaign party in NYC told the workers to go home. By then, the realization had set in that demagoguery had won the day despite all the polls that said he had a near-zero probability of winning.
However, as I see it, your theory about national polls being more reliable that localized polls has two flaws:
(1) The national polls this year had Clinton’s margin roughly the same as Sam’s state-poll based projections and prediction.
(2) Virtually all state polls are intentionally weighted by national demographic models. Thus (1).
The polls were wrong because the demographic models were wrong.
The final M-m and EV predictions on PEC remind me of being lectured early in my career in data processing and computer science on the profound difference between precision and accuracy. The PEC projections were inaccurate with great precision. GIGO.
An odd coincidence – if Sam had simply switched the names Trump and Clinton in his predictions, he would have nailed it.
im up watching MTV wonderland– the consensus was that HRC was just an awful candidate that didnt care for black issues.
But its interesting to see protest rap taking on an anti-Trump shape.
Looking at protests in Cali and Colorado–I wonder if the movement will have legs.
Thank you for all your work Dr Wang– the commentariat at this site is stellar.
I was Wheelers Cat in 2012…( much luckier than Shrödingers cat). Wheeler and Einstein lived in the same neighborhood, and Wheelers Cat would follow Einstein home– so the Cat wasnt dead/alive like Shrödingers Cat but at Wheelers House or Einsteins House.
Now im Ed Wittens Cat because, well, Dr Witten!!
But it turns out Wheelers Cat was also much luckier than Ed Wittens Cat– i feel anxious and scared– likely the mirror of how the redbrains felt in 2012.
In 2020 i think i’ll be Bulgakovs Cat.
Whose name is Leviathan.
😉
oh yeah, almost forgot
the cake IS a lie…this time.
but it doesnt always have to be.
“Actually, I do happen to resemble a hallucination. Kindly note my silhouette in the moonlight.” The cat climbed into the shaft of moonlight and wanted to keep talking but was asked to be quiet. “Very well, I shall be silent,” he replied, “I shall be a silent hallucination.”
― Mikhail Bulgakov, The Master and Margarita
Lots of flaws in HRC, but pointing out not caring about black issues is a head scratcher for me.
On your Complex Adaptive Systems Dynamics link, I find one fundamental flaw:
The US does not rest power in the hands of one individual. Yes, he has vast control over the military, but not much else. The obstruction of Obama’s agenda through most of his administration was a profound illustration of those limitations. Although he found clever ways to use what executive power he had, his executive orders all die the day his term ends.
We are about to embark on a discovery of how well our system of “checks and balances” works when one party has control both legislative houses, the judicial, and an executive that is loathed by two thirds of the party regulars.
If a few senators decided to switch parties and organize the Senate under Democratic control (one of the paths possible under our system), the Trump Presidency would be contained somewhat, although his war powers would still leave me in a state of dread.
Blaming a “massive polling error” misstates what happened. The election eve polling average was about +3% for Clinton and she did eke out a small win in the popular vote. That pollsters were 2 or 3 points off is hardly a “massive” or historic polling error. It is very much within the normal range of recent years, when the increasing difficulty to get voters to respond to pollsters has made polling more prone to error. Nate Silver’s programs were correct that, on the eve of the election, Clinton’s chances of winning were only about 65%. All of the prognosticators who placed them in the 90s had no business being anywhere near that sure. Princeton, at 99%, was worst of all. Although there were criticisms of Silver for being an outlier,none contained plausible answers to his reasons for his caution. Silver’s reasons included the fact that there was a much larger group of voters who were undecided or supporting third party candidates than,for example, in 2012. Also, polling has been less accurate in recent elections,and a late trend toward Trump,or polling error,could swing all of the battleground states in his favor,so the idea that Trump had to “run the table” obscured that his winning was less unlikely than Princeton and others maintained. Add to this that every pollster can only guess about what likely voter screen to use and about the actual demographic shape of the electorate. This makes polling perilous in the best of years. In a year with historic contrasts in the presidential candidate preference of different demographic groups, guessing even slightly wrong on ,for example turnout of women vs men,white vs nonwhite,college educated vs working class could make for a decisive polling error. Not only did Nate Silver see this, but, frankly, it was so obvious that it is astonishing that other sites,like Princeton, failed to see it.
Margin of error is calculated on the basis of one sample. The “law of large numbers” says the errors of a large number of samples will cancel. They didn’t in this instance. When my statistics and probability professor was asked “what is a large number”, he replied “ten”. Needless to say, the number of polls all off in the same direction this year was massively greater than ten.
When the error of an aggregate of samplings is consistently at the maximum distance in the same direction, that is not “margin of error” chance. That is ironclad evidence of bias.
I agree on the question of turnout. As a person who has spent his life dealing with numbers, I have always struggled to understand the concept of “likely voter” numerically. Precisely how likely is “likely” numerically?
Sam, I am indebted to you for your link to the very profound Ezra Klein piece: http://www.vox.com/policy-and-politics/2016/11/7/13532178/donald-trump-american-democracy-weakness
That article should be required reading for essentially anyone with an interest in political science and a fear of demagoguery.
I was once deeply involved in (very) local Democratic party politics. (I helped elect and re-elect my wife to a town Board in NY state.) This was back before primaries became the almost universal mode of choosing party candidates. Back when candidates were ultimately chosen by party regulars, and parties were organized around specific policy agendas. The primary system was touted by the left end of our spectrum as the most (small d) democratic way to select candidates. I saw it as the ultimate destruction of the meaning of political parties. We have since evolved a system where Presidential candidates that have zero ties to any portion of a party’s policy agenda, and virtually no interest in any other part of the party “ticket” can come out of nowhere and capture a nomination. And now we have people who argue with great passion that we should go even further down this road to perdition and have totally open primaries where anyone can vote in any primary without regard to party affiliation.
My wife, American by birthright, but born and raised to adulthood in Canada, has an unshaken profound belief in the superiority of the Canadian parliamentary system over our peculiar form of government. Our ongoing discussion (and no, “discussion” is not code for argument) on this topic for our 50+ years together has given me a much deeper understanding of both systems and the advantages and flaws of each. One obvious (but far from most important) advantage of their system is the simplicity of their national elections and the brevity of the campaigns. (Her Canadian relatives were as sick of our endless campaign season as we are—and as appalled at the outcome) The most important advantage of their system is the vastly more difficult chance of a Trump gaining control, and in that unlikely event, the rapidity with which an over-reaching executive can be removed and new elections called.
This year’s outcome has left me deeply worried about the ability of our system to survive much longer, and Klein’s piece summarizes my reasoning and fears precisely.
Yep, Pres. Nobody hacked the system.
Clinton will win the popular vote by 1.7%, more than Kennedy or Nixon, not only Gore. I wish more people had rejected Nobody but the current situation should be unacceptable to everyone in the US.
Vote for change:
https://www.change.org/p/electoral-college-electors-electoral-college-make-hillary-clinton-president-on-december-19
Klein’s piece is appalling and paternalistic and exactly the reason, or one of the reasons, why Trump won. Dems though they could run a career politician with historical unlikability in a change election and win if they shouted ‘racist’ and ‘sexist’ at the other guy enough – and now that that hasn’t worked, it’s an obscure fault of the system and not their own idiocy. Nevermind that it was the very systems that make us a republic – the electoral college – that kept their candidate from winning. Klein’s piece is both vacuous and insulting, and Sam’s sharing of it makes me question his political sensibilities.
Sam, why no new blog entries? Lots of data has come out on the makeup of the electorate so far and we could use some of your mathematical analysis as to what went wrong in both the polls and your model.