A final unskewing
Mother Nature is our best teacher and the only one who is always right. – Viktor Hamburger, biologist In yesterday’s Los Angeles Times...
How likely is a popular-vote/electoral-vote mismatch?
President Obama is peeling away. As you can see from the electoral vote (EV) estimator, he is the candidate with the momentum, not Romney. In terms of EV or the Meta-margin, he’s made up just about half the ground he ceded to Romney after Debate #1. And the indicators are still headed straight up.
A few days ago, the word was that Team Romney was buying ads in Minnesota and Pennsylvania. If he wins either of those states I will eat a bug. Ohio…a really big bug. And yes, I will post a photo.
Today I’ll address a common concern among the commentariat: will President Obama lose the popular vote? Steve Lombardo is on the case with some rather bad Excel curve fitting. It does not even show the Debate #1 bounce. Hmmm, someone take his keys away. (This is basically bad time resolution, a common problem in poll aggregation.)
Anyway, the short answer: I estimate Romney’s chance of winning the popular vote at 6%, odds of 16-1 against.
Usually, I’ve used state polls to estimate the electoral vote (EV). Others have used national polls to make claims about the popular vote (PV). Both types of polls can be used to predict both PV and EV.
In the past three elections, national-poll medians have been off from the final PV outcome by 0.3% (2008), 1.4% (2004), and 2.5% (2000). Doing some extrapolation, I estimate that for perfectly tied national polls, there is a 1 in 3 chance that one candidate will win the popular vote by more than 1.75% (i.e. standard deviation = 1.75%). This is not super-informative, but it gives a little information. I will use this to make a “prior” expectation, i.e. we start looking at state polls with an expectation of how likely a particular margin M is, even before we have examined a single poll. Let’s call this prior P(M), where M is the popular margin.
State polls give a considerably sharper look. For example, in Obama v. McCain 2008, the Meta-Margin (defined as how much state polls have to swing to tie up the Electoral College) came very close to the actual outcome of Obama +7.3%:
We can combine the prior with the state-poll-based Meta-Margin to get a sharper estimate for next Tuesday’s PV outcome.
In the graph below, state polls give the black curve, S(M). To calculate the true likelihood L(M) of a given value of M, we calculate L=P*S. That is a Bayesian estimate – taking prior expectations into account. The figure shows P(M) in green and the final estimate L(M) in red.
On the red curve, everything to the left of zero adds up to the probability of a Romney popular-vote win: 6%.
Getting a popular vote and electoral vote mismatch can happen two ways: President Obama wins EV but not PV, or Romney does the same. These add up to 9%. This is a pretty low risk.
The exact popular-vote prediction today is Obama +2.1 +/- 1.1%. Allowing a little bit for Gary Johnson, this gives a vote share of Obama 50.3 +/-0.5%, Romney 48.2 +/-0.5%. I’ll update that to a final prediction this weekend.
If you want the MATLAB code to do your own Bayesian estimation, it’s here.
Here is the original Facebook comment thread from the temporary site.
Topics:
Even Trende sees the trend.
Yes, I’ve been waiting 6 months to use that one.
You’re welcome.
Welcome back! Sam, could you please explain where the PV/EV correlation is represented?
Ben: The charts are shown side by side for easy comparison.
Click on em in the right margin or use this link:
http://election.princeton.edu/history-of-electoral-votes-for-obama/
Your work has been invaluable: from your keystrokes, to God’s ears. Amen!
What are your thoughts on Silver’s comments about statistical bias on his blog today? Is there really that high of a probability of statistical bias in the polls, at this point in the election?
More! The national poll vs. state poll discrepancy is a new feature. The best bet is the state polls are more reliable. But the national polls are, after all, information. Silver’s 16 percent is without national polls arguing for the bias.
Sam, have you seen the post by zserf on reddit today?
An interesting idea occurred to me. It would be interesting in your final prediction to include a vote projection, meta-margin, and probability projection for each of the critical swing states, say Ohio, Colorado, Iowa, Virginia, and Florida. Do you have things set up to do something like this?
Why limit yourself to eating just Hemiptera?
I know, right? It should be lepidoptera for Minnesota and coleoptera for the other two, in keeping with their official state insects.
If we bet against him, we have to eat the thing.
http://fooddriveitem.blogspot.com/2010/07/canned-crickets.html
Out of curiosity, what was the Bayesian prediction like in 2008? I find it pretty incredible that Obama’s chances are > 99.5% even when the popular vote is going to be within a few percentage points… (but of course you have made it quite clear how important state polls are as we rely on the EC). Makes you wonder what it would take for the prediction to be closer to 50-70%!?
Also, I just thought about putting the odds into poker terms (texas holdem). Obama’s flopped top set and Romney has flopped bottom two pair and is going to need running cards to win. i.e. Obama holds Ac Ah
Romney holds 7c 2h
Flop: Ad 7h 2c
Turn: ??
River: ??
Sam, please don’t eat a bug. It won’t be the bugs fault if the polls are dead wrong. How about a bug-shaped turnip?
he could eat an already-dead bug
Great posts and compelling methodology, Sam. I’ve been relying on you and 538 for the statistical fix. Also reading Larry Sabato at UVA for the political punditry fix– Sabato has been calling the presidential race a toss up; his Senate predictions hew pretty close to yours, with the Dems pulling it out.
Those whose minds stray to the more esoteric on occasion might like http://www.politicalastrologyblog.com. Serious astrologers have been calling the race for Obama roughly 2-1, with all forecasting a tight race and many pointing to the possibility of recounts and delays in determining the outcome.
Sabato makes me suspicious of his intentions. Of the core prognosticators (Sabato, Rothenberg, and Cook), I like Cook.
Sam, do you mind saying a bit more about your suspicion re Sabato? I ask because he sets off my radar a bit, too, but I can’t quite put my finger on why.
Chuck Todd on MSNBC just now stated that “it could be” that we see a 50/50 split. Ha ha ha, I just started laughing and then I came here and read this!
Same here! I heard that and thought that Todd can’t be serious and that it was time for a reality check. Whew!
There are just too damn many of these people to take down.
sadly Todd seems to be what passes for a “numbers guy” on the idiot box
Pre-2009 Todd was a very sharp guy. But as we’ve seen, sharp analysis that plays down uncertainty doesn’t sell in the mainstream media.
With all the stunts that Mike DeWine has been pulling, I will grant a waiver on the Ohio bug. In a fair outcome….I’d eat a bug
Regarding Dr. Wang’s obiter dictum that the EC provides robustness against fraud: the evidence given for this (a bit of hand-waving and a nautical drawing) seems far weaker than I’m accustomed to on this site.
Is it easier to fraudulently flip 537 votes — the amount by which GWB was declared the winner in 2000 — or 500,000 votes — the amount by which Gore won the popular vote? I would say the former.
To me this suggests that the EC makes the Presidential election MORE vulnerable to fraud. If by fraud we understand also the intentional misdirection of voters, the destruction of absentee ballots, and other such low tricks, the contrast is even starker: with the EC, the hypothetical “bad guys” can, and do, concentrate their efforts on the 10-15% of the population that reside in swing states, and will succeed more easily than if they had to sway a similar percentage of the whole population.
My argument is hand-wavy too, of course; it’s not a proof. It would be very interesting to know for sure, via proper statistical analysis or via simulation, whether compartmentalization does add robustness against fraud, under various scenarios. For instance, one could hypothesize that up to 2% of the votes can be fraudulently flipped, always in one direction: how often will this affect the outcome, with and without the EC? What if the flip occurs only in a particular state or set of states, naturally assumed to be swing states, since no-one will bother trying to flip California or Kentucky? What if the direction of the flips is random?
And if compartmentalization does help, wouldn’t 500 bulkheads of roughly equal size be better than 50 of widely disparate sizes? (Hmmm, one electoral vote per Congressional district… this would at least avoid the current disparity where Alaska has 3 times more weight in the EC, relative to its population, than California!)
Inquiring minds want to know…
Best wishes,
Silvio Levy
You say there’s a 6% chance that Romney wins the popular vote, but only a 0.5% chance that he wins the electoral vote.
But you also say the chance that the EV and PV do not agree is 9%, in this article.
How is that possible? There must be something else I am not following in your argument.
Thank you.
I’m nothing official here, Bowen, but that’s a great question! The answer might be that he has to add in other infinitesimal fractions he could leave out of the separate calculations….Or maybe he didnt work these out on the same day. Or he’ll revise…. or? Wait… Outlier candidates?
Only a Princeton student could be so stupid. Get the bug ready for consumption and consider seeing if you can get the money back that your parents wasted on your education. Go Crimson!
Governor Romney, how good of you to join us. The commenters tilt Obama. Could be some persuadables. Perhaps you could consider an ad buy. Or a guest post on campaign strategy and resource allocation. I’m curious about your thoughts on Pennsylvania and Minnesota.
This is great, but what about all the un-knowables? Like how many people can’t get to the polls because they don’t have gas, or how many people just might not show up because they think BKO will win without them… Sure some of this may happen in a deep blue state where the EV is assured, but what if it’s in a close swing state? And even if it doesn’t affect the EV (which of course is the bottom line) what about the popular vote? Might it be enough to cause an almost-tie in the popular vote or even cause an EV-pop vote split?
Why the State and National Polls may BOTH be Right
One point I’ve not seen anyone raise to date about the apparent contradiction between state and national poll results: The process of extrapolating from the state level to the national level involves an ESTIMATION of the actual vote count in each state so as to properly weight its results in building up to a national estimate.
Relative vote counts across states are likely to change from election to election (for example in an off year election, how much attention states get depends on the competitiveness of local races; in a presidential year it partly depends on being or not being a swing state). This, plus population changes and changes in voter registration levels could easily add up in a way that’s hard to estimate and that could easily account for a 1-2 % discrepancy between estimates derived from state level polls—no matter how accurate those polls may be. In short, there’s no fully up to date basis for solidly estimating turnout levels in a state, yet that would be critical to going from state polls to a national vote estimate.
It thus seems far more plausible that there’s some ‘slack’ in the extrapolation from state to national results than that the diverse group of national (or state) pollsters all have some sort of odd statistical bias that no one can explain
I agree. There are very few state level polls of states that are “safe”. Texas, Illinois and Georgia all come to mind in this election cycle. According to 538 there were only 4 state-level surveys of New York during the month of October. 538 is predicting 56% Romney in South Carolina based on polls from January and December. There is also very little state-level polling in smaller red states (Kentucky, West Virginia, Idaho, Utah, …).
Meanwhile, the national pollsters are trying to get interviews that are representative of the entire population, rather than focusing on voters in swing states.
I’m unclear why PCE and 538 are discounting the national polls so heavily for your predictions of the total national popular vote. If the national polls are right that the popular vote is about even shouldn’t the probability of a popular-electoral split be considerably higher than 9%?
Hey, just clarifying my earlier question, which I don’t think it has yet been answered.
In this article, you say there is a 9% chance of an EV-PV mismatch, and a 6% chance that Romney wins the popular vote.
Combining these, there is at least a 3% chance that Obama wins the popular vote but loses the electoral vote to Romney — otherwise, there is no way the EV-PV mismatch happens 9% of the time.
So there’s at least a 3% chance Romney wins the election but loses the popular vote. But you’ve also said Romney has only a 0.5% chance of winning the election.
I must be missing something significant in your reasoning. Can you clarify, please? Thanks.
We have to make sure this election is not stolen electronically by Karl Rove. Romney did it in the 2012 Republican Primaries and McCain did it in 2008 Presidential election. This is shown in a well written paper by two statisticians who specialize in fraud analysis. Do a google on the following words:
“vote flipping large precincts michael collins part II”
Read Part I and Part II.
here is link to part II
http://www.themoneyparty.org/main/2012/10/part-ii-rigged-elections-for-romney-2/
Is there any way to factor into the analysis a Sandy factor, a reduction in voter numbers from NJ and Long Island? With large numbers of people without electricity, gas, and in many cases dislocated from their homes and communities, has anyone estimated how it will impact the vote in these heavily Democratic areas? I doubt that Romney would have the numbers to win NJ, but might the dropoff in numbers push him ahead in the popular vote?
I agree it is not likely that Romney would win PA, MN or other such states. However, could their assumption be that all of the switchable voters in FL, VA, OH, etc have already been tapped (and the difference being within a margin of error), but there may be more switchable possibilities in those (out of reach) states…therefore the new spending strategy is not to win those states but to garner more total votes in an attempt to win a national popular majority with all the bragging rights of a ‘stolen election’ when the Electoral College goes for Obama.
Dr. Wang:
Re the popular vote polls, the Pew Research Center on Sunday, has the President at 50% and romney at 47% as their final snapshot before the election and they are suppose to have a pretty good track record as to their accuracy. What say you about this?