A Presidential/downticket prediction challenge
Last month (“Using predictions in the service of ideals and profit,” Sept. 23) I asked what makes a good prediction. I made an analogy to hur...
After the storm
We shall not cease from exploration
And the end of all our exploring
Will be to arrive where we started
And know the place for the first time. -T.S. Eliot
Good morning! The day after the election is always a bit of a relief for me. We’re still waiting on a few races, but here’s a preliminary look back at how our polls-only approach did. All in all, extremely well. As Randall Munroe sent me last night: “BREAKING: Numbers continue to be best tool for determining which of two things is larger.”
President
In the races called thus far, pre-election polling medians were correct in 50 out of 50.
There is one race remaining, Florida, where Obama leads Romney by 49.91% to 49.36%, with 100% of votes counted. It hasn’t been declared yet, partly because the threshold margin for a recount is 0.5% – they are probably still working that out. Pre-election polls were a near-perfect tie. I suspect I am about to lose that coin toss. Update, 11/15: final Florida result, Obama 50.0% Romney 49.1%.
The two-candidate popular vote share is Obama 51.1% to Romney 48.9%. This exactly matches my prediction, which was derived from state polls with a little Bayesian help from national polls. Update, 11/15: popular vote Obama 51.4% Romney 48.6%.
Bottom line: I will not have to eat a bug.
Senate: Of the closest races, election returns match polling medians in 10 out of 10. Of particular note is the North Dakota race, where Heitkamp (D) leads Berg (R), 50.5-49.5%. That’s one where I had Heitkamp based on polls, and Nate Silver had Berg based on polls plus other factors. We are also waiting on the Montana race, where Tester (D) leads Rehberg, 49-45%. The upper chamber appears headed right for the median that we predicted, 55 D/I to 45 R.
House: This one will take some time to sort out. From the Republicans’ current majority of 242 seats, I predicted losses of 2-22 seats. It looks like the losses will be toward the low end of that range.
I am very interested in whether Democrats win the national House popular vote, which would mean a mismatch between the vote and the seat count. This is due in part to redistricting. It would be only the second time since World War II that it’s happened, and is antidemocratic with a small “d.”
I’ll give a more detailed look later, especially regarding our prediction challenge. For now, I think we can safely say that the following people had a good outcome: President Obama, Congressional Democrats, and quantitative poll analysts.
Finally, a trip down memory lane: the Princeton Election Consortium long-term predictor, as of August 3rd:
Back in a bit with more wrap-up…
Though, of course, the best in this case do have conviction….and won!
Dr Sam gave us the armor of conviction.
well, me at least.
True, though that’s Yeats, not Elliot.
Perhaps this has been covered in someone’s posts earlier, but last night on Fox News, during the time Fox called it for Ohio, did any of you notice how absent the usual blowhards were? I could only find Rove doing his usual deceptive thing and Chris Wallace. None of the usual cast of Fox talking heads. Plus, I think it was Chris Wallace (or was it Britt Hume?) that questioned the Fox call on Ohio. Then one of the moderators, an attractive female whose name I cannot recall, spent about 3 minutes walking across the Fox studios to some back room where the number crunchers were. They were asked if they were comfortable with their Ohio call. Indeed they were and they explained why, with reasons such as Cuyahoga county still not complete, etc. Then the lady walked across the studio again and found Rove. Incredibly, Rove was still doing his Rovian act!
Did any of you notice this?
…………………..
And, again, from an aging statistician, great work!
Steve
Yea, it was very funny. But remember Rove spent a lot of money trying to get a different result.
It was hilarious. Rove was acting like a student caught red handed while copying. Megyn Kelly is the anchor’s name.
I was watching the county results in Ohio and knew it was over. The TV networks saw the same thing. Believe it, Karl Rove.
But I was following the TPM blog, and I should like to give Fox credit for early calls and accurate ones. I had not expected that!
Thanks Sam, really great job!
Congratulations, Dr. Wang, for your great predictions and models. It looks like your initial prediction might come true, 332 EV’s for Obama. If uncounted votes are from Miami-Dade County in Florida, and Obama’s already up 46,000 votes, then he very well might get Florida. If he gets Florida, you captured that.
Yu da man!
Thank you, Sam and Andrew, for a job well done.
I was at an election party last night with about a 50/50 D-R split. People asked me why I was so calm and smiling. I said: “It’s the numbers.” and left it cryptic. Most of the Republican left early MST with mutterings of voter fraud and other dark thoughts.
The most telling comment of the night was Peggy Noonan’s: “I just don’t understand what’s going on out there.”
I never got around to reading all the comments here. I will later today.
Again, thank you and Andrew so much. I learned a lot and you offered much peace of mind. Thank you.
@E L check out Peggy Noonan’s old tweets to see why she can’t understand what’s going on. Its hilarious
https://twitter.com/Peggynoonannyc
The most telling comment of the night was Peggy Noonan’s: “I just don’t understand what’s going on out there.”
Finally, Peggy said something that is both true and perceptive.
Thank you so much, Dr. Wang, for your work and for this website. If I had to only listen to the MSM these past few months I would have gone nuts with worry. You kept me sane with rational, intelligent math! I am so grateful for your work.
Dr. Sam,
After all the obsessive visits to your website, the most joyful moment was reading the poem by T.S. Elliott.
Such a joy to have drifted into your website about a month ago. In the US, every geek always suspects that she is an ugly duckling. Then one day, she wanders by a lake full of swans!
———————————————-
PS: Now, since I work in chaos, I am finding it hard to resist the temptation to sprinkle a little mist on your parade. Looking at your track record from 2004 to 2012, I feel like commenting that
there is a hint of imperfection, in too much perfection.
“I work in chaos” is such a great phrase! Brings Greek mythology to mind 😉
PG, i love that comment so much. its exactly how i feel about PEC.
And I love chaos theory. you should have made more comments for me to admire.
The backgrounds of the commenters here are unparalleled, various and splendid.
and….it is said that absolute perfection is perfectly boring. and anti-chaotic as well.
😉
Great job, Sam! Would like for you to keep doing this at least until an election where the analysis shows a likely Rep win so conservatives can get their heads out and realize not everything is spin.
That’s a good idea, Ed, though I think memories will not be long enough for it to be appreciated.
I just wanted to point out to the “skewed” crowd that, if you take the traditional old ten states in contention: NV, OH, VA, IA, CO, NH, WI, FL, PA, and NC, you will find that the polls were actually biased 1.2-1.5 points on average in ROMNEY’s direction.
NV, VA, IA, CO, NH, WI, and PA all went to Obama in higher margins than expected. OH went lower, and FL was about dead on.
Even PPP had a slight bias toward Romney in the swing states. I credit the Obama GOTV effort.
OH. I was so relieved once they called CO and I knew, no matter what shenanigans happened in OH, PBO had won. Because if it had really come down to OH, this thing would not be over, IMO.
OH is to Dems what PA is to Rebubs, just always giving less than you think.
Possible explanations (I’m basically just paraphrasing Nate Silver, for that matter):
1. The ground game, of course.
2. Latinos with Spanish as main language.
3. PPP doesn’t call cellphones.
Some Body, you’re right on the money (as usual). I had forgotten that PPP doesn’t poll cellphones, and that’s at least as likely for the bias as GOTV efforts.
Sam, great work as always with the analysis and the entertaining commentary.
I also really want to thank you for the donation recommendations via Act Blue. I gave money to Heitcamp, Tester, and others, and I feel that is where I had the most impact in this election (I’m in CA so my vote doesn’t matter in national politics). Thanks so much!
Congratulations from Italy, Doctor Wang.
P.S..: next year we have general elections here, would you check your statistical method with an european election?
I guess it would depend on the system and the volume and quality of polling. I was entertaining the thought of getting something similar set up for the upcoming election in Israel (this January), but then I recalled that we only have a trickle of polls published, with a significant history of large (probably systematic) error, and the whole election system (electing parties into parliament with seats roughly relative to their share of the popular vote) is apt to produce much more uncertainty than the system used in the US.
The more unusual aspects of his model (and Nate Silver’s) have to do with the fact that we have an Electoral College. Other systems would probably need different mathematical approaches, though of course many of the same issues would arise.
Interesting point about the House GOP not matching the national vote. The systematic gerrymandering of Congressional districts is something that I only sporadically hear about, but which I haven’t (so far) seen any systematic analysis of.
FWIW, if you do find yourself having to eat a bug or three, there’s a grasshopper taco at Oyamel here in DC (between the Newseum and Chinatown, down the street from AARP; we have strange landmarks here). It’s really quite tasty.
I found this interesting, too. Haven’t heard it anywhere else. Does anyone know where I can find the running total of the House popular vote?
I will second that. Crunchy!
This, from CNN: “How did Mitt Romney lose a race that seemed so tantalizingly within reach just one week ago?”
Sandy did it is part of the answer.
Suja;
Well as followers of Sam and Nate can tell them
The most obvious answer is that the race was never within their reach except in the delusional minds.
OK, a little old-fashioned arithmetic in Virginis shows that Republicans won 8 of 11 house seats by taking 51.6% of the state-wide house vote.
Interestingly, Romney came within 33,437 votes of the combined R house candidate totals, while Obama got nearly 200,000 more votes than D house candidates and Kaine had almost 40,000 more than Obama. In addition to the obvious benefits of gerry-mandering, it seems that perhaps many Democratic voters simply didn’t bother to vote in their house races.
Romney came very, very close to EV parity in the immediate aftermath of the first debate. Sam (and to an even greater degree, Drew Linzer) thought that was a short-lived fluctuation and the system would return to something like the pre-convention mean, and they were right.
If that’s so, then the condition for Romney winning would have been some disaster of magnitude comparable to the first debate (or, actually, even rather less severe, given the situation that already pertained) befalling Obama in the final week or so of the campaign, giving him no time to recover.
Since the last week of campaigning is generally extremely stage-managed, it’d have to have been some external event. Hurricane Sandy was just such an event.
So I’d say the formula for a Romney win would have been for Obama to catastrophically screw up the response to Sandy in some way recalling Bush at Katrina. Obama didn’t need Sandy to win, but Romney probably needed him to fail badly at it. Unfortunately for Romney, the external event happened to be in an area of relatively great competence on the part of the Obama administration.
Unfortunately for Romney, but very fortunate for residents of NY and NJ, as well as for advocates of effective governance.
We are justly praising Sam’s number crunching, but I would also like to thank him for his Act Blue page. Because of it, I made several contributions to Senate candidates in tight races, all of whom won!
Thank you, Sam!
Great work, Sam! Cold math won over hype last night.
Dr. Wang,
You and Nate Silver kept me sane for the last few months. I will be forever grateful. It does raise some problems, though
An anecdote, I had to speak to the investor in charge of my company’s 401K. We spoke about the election in very general terms, she said essentially that she was afraid the counting would last far beyond Election Day. I reassured her that it would not. I must have sounded very confident. She looked surprised and said, “What do you think is going to happen?” I said, “I don’t have to think. It’s all in the numbers. you’re a numbers person, right? Obama will win with between 303-330 votes.” She sat back, smiled, and said, “I disagree. I think Romney will win decisively.”
I’m supposed to let this woman invest my money????
Another thing, won’t the GOP just instruct people to lie in polls next time to skew results?
If I may, I would not let this person invest one single penny of your money, you have you answer in the numbers. Google Allan Roth and follow his advice.
Similiar anecdote: I work for a very large asset management firm famed for the strength of its quantitative research. About a week before the election I asked our DC-based political analyst what she thought of Silver and the other quant modelers.
She said she thought it was all nonsense. (Although to be fair, she alao said her “best guess” was that Obama would win.)
Fortunately, I index all my 401k money, so no worries.
I watch Bloomberg TV everyday, and throughout the year, there were asset managers coming on and saying that the markets were pricing in a Romney win. Meanwhile, not once did PEC, 538 or even the gut feel aggregator Intrade ever show a Romney win. It made me wonder what models, if any, these people, who manage billions of dollars, were using.
Thanks to Professor Wang for all his hard work, both as a voice of reason and a subtle advocate for political participation. As for the results, a coin toss away from perfect is pretty damn good.
A few days ago I posted the following questions in the comments section, and am reposting them here in the hopes that someone more adept with statistics will address them:
I am curious about how Silver’s built in “conservatism” in his projections will play out on election day. Perhaps someone more proficient in the statistics can help me out?
Suppose Silver (or anyone) ended up with a large number of state (and/or Senate) probabilities in the 60-85% range . . .
— If these probabilities actually reflect reality, shouldn’t we expect some of these projected outcomes to go against the favorite?
— Conversely, if almost all of the “conservative” projections go toward the favorite, wouldn’t this be an indication that the probabilities were too conservative?
Might we end up with a situation where Silver (or anyone) could call every race correctly, yet undermine the validity of his/her own model by so doing? (Surely it wouldn’t be reported this way, though.)
[I posed these questions in the form of a hypothetical, but think the hypothetical may fit the reality of today.]
While I might have the methodology wrong, I tried to answer my own question using Mr. Silver’s probabilities for the Presidential election. Using the “multiplication rule” and simply multiplying the Silver’s probabilities of the favorite winning in each of the state Presidential races, I come up with 10.6%.
Please correct me if I am wrong, but I take this to mean: If Mr. Silver’s probabilities accurately reflected the real probabilities in each of these races, then there was only a 10.6% chance of the Mr. Silver’s favorites winning every single race.
Doesn’t this suggest that Mr. Silver’s model was perhaps a bit too conservative?
Correct, he is too cautious. I’ve made this point a few times. However, it seems like the kind of thing that suits lay readers. I have received so much heat for my 90-99% probabilities all season!
Thanks for the response. I realize you have been saying this all along, but as a “lay reader” myself I have been trying to understand how one might try to quantify his conservativism. The irony is that his “perfect” result might be the best indication that his model is not quite so perfect. (Of course his model is magnitudes better than the coin toss mythology of the pundits.)
Thanks again, and congratulations.
Simple multiplication won’t work, though, because the results in each state are not independent from the result in others, and the uncertainty on 538 includes a component of projected systematic error (shortly before the election, Jackman did basically the same on Pollster too, but he described his calculation in greater detail).
I’ll admit I was much more confident four years ago than I was this time around, but your numbers, Dr. Wang, always gave me reassurance.
Thank you.
It’s like President Clinton said at the convention: It’s math!
My son had to do a science project recently for school where he came up with ten examples of mathematical models.
I told him about your site, and he used your formula and results from 2004 and 2008 as examples. He’ll be glad to know you were only, let’s see, about 100% accurate in 2012.
Again, thanks.
Thanks also from me, Sam. You really helped me sleep at night! I know that might not be the case in 2014 but I appreciate getting the truth in a sea of spin.
The popular vote call was stunningly accurate. The power of sound statistics.
I like Nate Silver, but you go there with far less noise,which makes me think his other metrics were merely noise.
Why does Nate Silver get all the glory, and appear on CBS Sunday Morning, etc?
Congratulations. Stumbled on your site only about a week before the election but you hit the mark. Thanks. W
Just a small nitpick: the President’s popular vote margin is likely to grow considerably in the days ahead.
The biggest issue is California. The current totals available from major media outlets suggest that only a bit over 9 million ballots were cast in CA. This would be an implausibly large drop from the more than 13 million cast in 2008. What’s going on, I think, is that vote-by-mail ballots — a huge deal in CA — are not yet included in the reported totals. For what it’s worth, this interpretation is consistent with the footnotes on the CA Secretary of State election results website (see http://vote.sos.ca.gov/returns/status/).
There are also significant numbers of uncounted votes in OR and WA. Putting it all together, it’s not implausible the President could net another 1 million votes or so before all is said and done, which would push his margin of victory to around 3 percentage points. Of course, there could also be outstanding pots of votes in other states that I’m not aware of, and these could break the other way. Regardless, the bottom line is that it’s too early to accurately assess the President’s margin in the popular vote.
So, am I the only one waiting with baited breath for the explanation from the two professors in Colorado?
Crud. *bated
Well, they should be eating bait this morning. Sam set the bar high in in terms of owning up to consequences. Put their bugs where their mouths are…. 😉
They absolutely should eat some crow, but really the people on the right relying on their model should be far more ashamed. State level economic indicators is an interesting way to predict elections, but to think that this would be better than state level polling the week of the election was always silly.
If voters tell you they’re voting for the incumbent, the fact that the economic indicators in their state suggest they should vote the incumbent out is just noise. The voters know their personal economic situation, their stated intentions has to be a better predictor than aggregate economic data.
Also, models that rely solely on economic data ignore the fact that people do care about other stuff too. Economics is obviously important, but sometimes other issues matter enough to tip the scale against the grain of the economics.
YOU MATH WIZARDS CAN’T FOOL ME! Y’ALL CONJUGATED THIS VICTORY WITH YOU’RE DEVIL MAGIC!
Where does one get downloadable information on the election? Sam has a very good question about more House Dem votes than House Rep votes. I’d also like to look to see if there was a trend for more House Dem votes than for President Obama.
TIA
Well, thanks again Sam. (May I call you Sam? I feel we know each other so well at this point.)
I’ve spent the last two or three months here every day, sometimes several times a day, and this will be a bit of tearful goodbye, but you made me look like a mighty Prometheus to all of my Cro-magnon friends and family. “Chillax, righteous dudes, it’s a done deal for BO,” or something to that effect in a heavily inflected Spicoli voice. Unfortunately, they came away hating liberals even more. Don’t know why.
Anyway, I propose that in four years we all return and do it again. You’ll work 24/7 and fend off the Dean Chambers of the world, and I’ll sit on my butt and read it and appear prescient to the illiterarti. Works for me.
Thanks again.
Is anybody else waiting to see when unskewedpolls will update their site, and what they will say? Right now it looks like they are too stunned to come up with a plausible explanation for how they completely blew it.
Thank you Dr. Wang! You and Nate Silver kept me sane these past few months. In turn, I used your wiritngs and reasonings to calm my husband down every time he frantically called out to say “so-and-so (insert any pundit), said that Obama is going to lose because (insert any non-scientific reason)”. Thank you, I am eternally grateful. Nerds > Pundits any day.
Excellent job! Not only are you the most accurate, but also the most interesting and informative.
I have been following you all year and I almost wish the election were still months away because I have enjoyed your daily prognostications. Well, almost. I guess enough is enough after nearly two years! The next election cycle will start soon enough!
I heard you on WNYC. Your interview was more informative than any other pollster inbterview I have heard.
I am a statistics nut which one reason I was in Meteorology. I also love to handicap throughbred horse races. Maybe I should seek your help there as well.
Thank you for all you do here, Richard
Well done, Sam. Thanks for being an island of sane rationality over the recent weeks and months. It drove me bananas how, night after night, ABC news would say the presidential election was a dead heat or even that Romney was ahead. I guess they wanted us to stay tuned. PEC lifted my spirits every day for the last week and a half. And you were right!
one wonders if the press will remember in 2016 how right you and Nate Silver were last night.
They will only if they can profit from it, as the NYT has done with Nate. Unlike Nate Silver, Sam has a good job. I can see the Guardian or another aware media outlet picking up Sam as a columnist.
I’ve extrapolated the current vote totals from each state, and I was able to come up with the following estimate for the final Popular Vote total.
Obama: 67,580,339 (51.553%)
Romney: 63,508,751 (48.447%)
For instance, in California the current count is 5,573,450 to 3,635,571, but that’s only estimated to be 69% of the vote.
(Yes, some sites mention that “100%” of CA is reporting, but that just means every precinct has returned some portion of their votes. They have until December 7th to finish counting all of the votes, in CA.)
If we assume that the current percentages will stay about the same, e.g. 59% to 39% in CA, then we can estimate how many of the remaining votes will go to each candidate. For instance, in CA this would give a final total of 8,077,464 to 5,268,983.
Doing this for each state, we find that there are about 13 million votes remaining, with about 7 million going to Obama and about 6 million going to Romney. Of course, these assumptions may be wrong (and the “percent counted” estimates could be off for some states), but this leaves us with the totals above. 😀
Anyway… does this seem reasonable? Discuss!
Dr. Sammy put the STINK on the pundits!
He and Nate are prognosticating GODS!!!
Thanks for all you do, and for assuaging me these last two weeks. I needed it.
One of my students came in today with that quote about numbers being the best tool. We had used a lot of good sites like yours to keep track of the election, and he had kept reassuring his parents of an Obama victory. His dad saw it online and encouraged him to bring it in. I was quite tickled.
Thanks again, Dr. Wang.
The other House election where the party losing the popular vote retained the Majority of seats was 1996, where a 270,000 national vote advantage for Team D still translated into a 22 seat deficit.
This is not a coincidence. Something needs to be done about gerrymanding and state partisan officials having so much power over election administration. Far too much gaming of the system going on.
Elections in the US are spectacular. Lots of fighting during the run up – attempts to disenfranchise voters, intimidation, gerrymandering, but all based in rule of law. And the law corrected itself with disenfranchisement (the voters ID’s were thrown out) and intimidation (the billboards were exposed).
Gerrymandering is another issue. That is part of the Constitution with the census every 10 years. When too outrageous, re-districting maps are thrown out. Involvement in state and local politics – electing people who will re-district the way you want – is the answer to gaming the system. Said another way, “Eternal vigilance.”
What is the popular vote for the HR?
I love science and math. A win for all things good! Thanks Sam — YOU are the best!!!
Re: Rasmussen polls…
http://www.slate.com/blogs/weigel/2012/11/07/scott_rasmussen_explains_why_his_polls_didn_t_forsee_an_obama_win.html
Here are some preliminary numbers on the House vote. Not surprising, but it does repudiate Boehner’s claim that Republicans have a mandate.
http://thinkprogress.org/justice/2012/11/07/1159631/americans-voted-for-a-democratic-house-gerrymandering-the-supreme-court-gave-them-speaker-boehner/?mobile=nc
“Based on ThinkProgress’ review of all ballots counted so far, 53,952,240 votes were cast for a Democratic candidate for the House and only 53,402,643 were cast for a Republican — meaning that Democratic votes exceed Republican votes by more than half a million.”
Adam,
Thank you. Again, the importance of involvement in local state politics shines through.
Congratulations! But please: post a picture of the bug you would have eaten … for science.
For science! 😀
Both 538 and PEC did a great job but who was more accurate?
And why does Silver get all of the glory?