Princeton Election Consortium

Innovations in democracy since 2004

Outcome: Biden 306 EV (D+1.2% from toss-up), Senate 50 D (D+1.0%)
Nov 3 polls: Biden 342 EV (D+5.3%), Senate 50-55 D (D+3.9%), House control D+4.6%
Moneyball states: President AZ NE-2 NV, Senate MT ME AK, Legislatures KS TX NC

Geek’s Guide To The Election, 2016

November 8th, 2016, 12:00pm by Sam Wang

Here’s an informal guide to help you track results. I’ll probably update it up until 5:00pm. Please report errors in comments. Also, please list places you’ll be monitoring results tonight.

In other news, Buzzfeed has announced criteria for scoring the forecasters. They are using Brier scores and root-mean-squared errors.

Geek’s Guide Version 2.1, 2:00pm: Word and PDF.

Geek’s Guide Version 1, noon: Word and PDF.

Tags: 2016 Election

72 Comments so far ↓

  • Richard

    Why the 0.5 drop in the meta-margin between last night and this morning? Didn’t see any new state polls come out. Was there a correction to the calculation?

    • Matt McIrvin

      There were some new state polls. The most unusual ones were probably SurveyMonkey polls showing both MI and PA tied. Knowing how big a spread SurveyMonkey’s numbers tend to have, I wouldn’t read too much into them, but perhaps they pushed enough polls from before MI’s recent tightening out of the averaging window?

    • Mike

      Sam, are you going to change your prediction based on the new metamargin? That was a serious drop in one night…

    • Sam Wang

      Already changed it. That’s where the 93% comes from.

  • xman


    Thanks for all the great work. You have helped me filter out all the noise during this election cycle. Going into these final hours I feel well informed.


  • Paul Ruston

    Thanks for doing this Sam it enables a novice like me a means for understand how the evening is unfolding.

  • Barb

    I noticed a spelling error if that matters – “clustering” has some letters flipped under the House section on page three. This is a great resource, Sam! Thanks!

  • David

    Unless I’m confusing something, your “safe state” map has states with less than 5% poll margin – MI, PA, NV, NM. They’re listed on page 1 as not in the “safe state” list, but the map on page 2 has them included as blue. You didn’t do the same with the equivalent red states – GA, IA, AZ. They’re gray on the map and not in the safe state list.

  • Brian Rabbit

    Thank You not just for this but Your entire work on this site.

  • Jameson


    If it comes to bug-eating (Texas, hopefully), may I suggest these:

    Delicious and nutritious!

    • Matt McIrvin

      Hillary winning Texas would be one tasty bug.

    • Bill Ricker

      Fried crickets or locusts (Orthoptera) or chocolate covered ants (Hymenoptera) are probably the most appetizing of Hexapoda Insecta, but being not True Bugs and not named Bug, would be seen as a dodge to get something tasty. Not a lot of tasty options in Hemiptera (bugs per se): Stink bug, cicada, leafhopper, aphid, waterbugs. Maybe Cicadas nymphs prepare like Witchetty grubs (moths=Lepidoptera).

  • Avattoir

    A lobster is a bug, so to ensure no technical backsies, it’d be fairest to leave to your readers a choice over what sort of bug you eat.

    Note, I’m not referring to how the bug is prepared for your meal. It’s only fair to leave that to your own palate.

    Pro Tip: if your supporters were to choose to go for “tarantula”, I’d advise against attempting to eat one while it’s alive. It’s safer to kill it through the back of it’s ‘neck’, burn, blow & wash off off ALL hairs (Swallowing a tarantula hair could hurt you.), carefully & fully remove the head, the legs, the venom package & the digestive tract, crack the carapace, then deep fry as it or pull out the meat & roll in the batter for pan fry.

    mmmm … tastes like fried bug

    • George

      Sounds like about as much fun as suffering through this campaign has been :-P

    • Bill Ricker

      Technically Lobsters are Arthropoda
      Crustacea. True Bugs are Arthropoda Hexapoda Insecta Hemiptera. False bugs appear in other orders under Class Insecta; for purpose of wager, any Hemiptera or any insect with “Bug” in its English common name should be acceptable; pretty much anything with 6 legs even. We’d probably accept a spider or scorpion (Arthropoda Arachnida) because pretty grotty, but Arthropoda
      Crustacea is just too tasty, no; Lobster Crab Shrimp are not Bugs either literally or metaphorically — except as a racial epithet in Star Ship Troopers, in which case they’re not edible after ‘cooking’ .

  • chris

    What a great resource! You’re the only one who could raise the bar, and you’ve done it again!
    Next election (2018) I’ll be retired, and will volunteer for congressional or senate candidate based on your guidance. You inspire us.

  • Andrew

    Thank you for posting this Sam!

    As a side note, if you are interested in watching Virginia: watch the turnout in Northern Virginia (Fairfax, Loudon counties) (votes D) versus Virginia Beach (R). In past elections, the relative turnout of these two areas has been an indicator of how motivated the voters in each party actually were to actually enact their vote. (In the words of someone from rural VA: it’s almost as if none of the rest of our votes even matter…)

    • SP

      Watching Virginia increases one’s angst, as the red parts come rolling in, and Republican candidates surge to large leads. Then, NoVA (specifically Fairfax) starts trickling in, late in the game, and the lead goes POOF!

    • Matt McIrvin

      People always warn not to watch Virginia, but because I’m from there, I always do– and because I know the state pretty well, it doesn’t make me panic…

  • David Manheim

    Not useful for real-time results, but I’ll be following Danny Page and Jeremy Singer-Vine’s election forecast scores;

    Both will be grading this site’s model, plus a variety of others. Should be an interesting exercise.

  • Matt McIrvin

    A day or two ago there were really enthusiastic reports about insurmountable leads getting locked in with early voting in Nevada–but the Democratic lead from early voting in ‘s projections is only about 7,000 out of 600,000, entirely surmountable I think. Were we misled by happy talk?

    • Deep breaths, deep breaths

      I think you need to take any numbers you see on votecastr with several grains of salt.

    • Matt McIrvin

      Yeah, I think so too. I’d like to believe them, though, because Florida is looking really good there.

    • Matt McIrvin

      …Actually, on further inspection, Votecastr looks really good across the board. Not sure if I want to trust that view, though. They’ve never done this before and their algorithms are not, as far as I know, transparent.

    • Camhilfan

      votecastr already admitted their Nevada model had some issues. First, they had Jill Stein, and she’s not even on the ballot.

      Second, their registered voter data was off by 150K at one point.

    • Matt McIrvin

      Their results are weird so far: going by their turnout graphs, Hillary is blowing Obama 2012 out of the water in Florida but all the other battleground states are scary close, with Hillary running tiny and shrinking leads in most of them.

      BUT… they’ve yanked their projection maps and announced that they’re having trouble getting large chunks of the early-voting data for some states into the system, so what we’re seeing is biased toward day-of numbers. That probably makes a significant difference in Nevada at least.

      Seems like a bumpy launch, all told.

    • Matt McIrvin

      (Of course, how this relates to final numbers could also be affected by how different groups of people vote late or early in the day. I could see midday voting in some states being dominated by retirees.)

  • joec

    Sam Wang

    I could help but thing of PEC when bobbing heads began talking about momentum today. Huh?

    I can keep up with you as rule and my wife and I have both followed the Statix guys.

  • Scott H

    Good luck, and may all your snacks be bug-free

  • Greg Hullender

    The explanation for how to use the “Polling-Error Calculator” refers to a “horizontal black axis.” Should that be “vertical green axis?”

  • Gelatinous_Cube

    I raised the question before–is a Brier score “fair” as a measure of prediction quality here? A few concerns:

    1. What we really care about varies. Sam has said he’s more interested in resource allocation which is a power of one vote thing–how uncertain a given race is, the number of voters, and the impact on the larger question of the EC. Others are more concerned with getting the overall election right.

    2. Why not use RMS difference from vote share instead of binary outcome vs confidence?

    3. Brier score calculated across all states seems to reward having the right amount of confidence per state if you think of them as independent variables, but if you believe certain state results are strongly correlated you aren’t rewarded for getting that right.

    • Gelatinous_Cube

      For “confidence” read “certainty”.

    • GuyWhoShowsUpEvery4Years

      I just wanted to reply and say that your username is super awesome.

    • GM

      Indeed it is.

    • DM

      GM my ass.

      G-Cube, you have a good point, but what metric would evaluate forecasters’ correlation values from what’s observable?

      For Dr. Wang’s purpose, I’d like to score a prediction of “how close will each race be”, by absolute distance from 50-50 scaled into a voter power metric.

    • Lorem

      I think the nice thing about Brier score is that it’s relatively well known (as far as anything in this area can be said to be) and relatively accepted. I’d say it satisfies the most important criteria of being decently good and decently uncontentious.

      I think regarding 2, in this particular election/forecasting season it’s not too important. It seems like the main thing at issue is the level of confidence and everyone more-or-less agrees on raw guesses.
      For 3, something that would capture that would be interesting, but also seems difficult to measure objectively. Especially since beliefs about states being correlated are not a visible output of any of the models I’ve seen.

  • Nick

    Thanks for the election guide. I made an interactive spreadsheet that I would like to share with your readers. It is in an updated version of an Excel sheet by j.d. ruggiero that was posted on PEC back in 2012. I also added a tab for the important Senate and House races.

    In the President tab, you just enter the winner (C or T) across the top under each state and it shows the possible scenarios remaining (the map is just a jpg and does not update). For the Senate and House, you enter D and R and it updates the Senate tally and House meter respectively.

    Here is the download (Excel file):

  • Scott

    Am still trying to get my arms around how the two top dogs in forecasting. Sam Wang and Nate Silver, are at opposite ends of the spectrum and everybody else is in the middle.
    Wang has Trumps odds @ less than 1% while Silver recently had him @33%.
    They both seem to agree that polls show a 3 point advantage for Clinton, and Silver apparently is saying that 1/3 of the time the polls are low by over 3 points, which would indicate that they are also high by 3 points 1/3 of the time.
    This leaves only 1/3 of the time in which the polls would be within 3 points and I believe the actual percentage is closer to 95%–

    Can anyone shed light on this?

    • Gelatinous_Cube

      Sam has recently said that his model may be overconfident in its prediction–enough to take Clinton down to 91%. For the sake of consistency he’s not going to change it mid-process.

      Nate posted an article explaining why he has so much uncertainty:

      Sam has also commented that Nate’s model seems to have too much internal feedback in its effort to get both the states and the overall election exactly right.

      David R. of PredictWise also had some interesting criticisms of Nate’s methods.

      Two things I would discount: the HuffPost article, and accusations that 538 is trolling for mouse clicks.

  • Nathan

    If you’re not reading Jon Ralston’s Twitter coverage taking place as we speak of the Trump campaign trying to sue registrar in Nevada, well, you should. It’s amazing.

    • GuyWhoShowsUpEvery4Years

      The live feed for this is great. It’s so satisfying to hear this lawyer pulling Trumpese, and the judge to just be like, “Uh… No dude, this is not how the law works.”

  • Jay Bryant

    Thanks, Sam. Much appreciated.

    Thanks for “How To Know When The Fat Lady Sings” – I got a laugh out of that heading. :)

    I hope you’ll have to eat that TX bug, but I don’t expect it.

  • John

    They should use average of multiple dates and not just last day’s predictions.

  • GuyWhoShowsUpEvery4Years

    Oh, man. The Trump lawyer is getting BEAT DOWN in this legal challenge in Nevada.

    • JeffE

      Sort of… They are predicting what the vote looks like currently based on a number of factors (including, apparently, some exit polling?)

  • GuyWhoShowsUpEvery4Years


    Longtime occasional reader since 2008. I want to thank you for putting some sanity and mathematical clarity into a an arena where it is sorely needed. (As opposed to say, working to shoulder its way to the forefront of an arena where it doesn’t necessarily deserve that position, like, sports.)

    I’m hoping the election turns out the way the data says it should, because I think it’d be great next time this comes around if the national coverage just went and said, “You know what, top of the bill is going to Candidate A, so until we see otherwise, we’re focusing election on races down ballot.”

    I hope tonight results help you and your fellow “non-assumption scientists” nudge the stone just a little bit further on its way.

  • George

    Sam thank you for the “geek’s guide”. Excellent stuff!

  • Doctor Science

    Where will Sam be liveblogging? Here, Twitter, both?

  • RH Household

    Thanks for the efforts for this site, and the guide tonight. Monitoring results from Philadelphia suburbs.

  • Scott

    May you eat no bugs tonight.

    • Indomitable Ted

      I dunno, I’m hoping against hope that unexpectedly high Hispanic turnout down by the Rio Grande will make Sam eat that Texas bug. And Texas bugs are big bugs!

  • RA

    The fat lady sings section doesn’t have what I would consider the most important information, which is what states or even counties tend to or often report early and will give an idea of if Hillary is on track or not.

    Signs Hillary very likely won (the fat lady sung)
    Hillary announced winner of FL and NC or NV?

    Signs Trump won
    Trump announced winner of PA and NH?

    At what point, predicting before results come in, would you say we can turn off the tv and go to sleep confident of the winner?

    • Josh

      New Hampshire will have reported most of its votes early in the evening. If Hillary wins there by more than a couple percent the race is over.

  • Agent76

    “If voting made any difference they wouldn’t let us do it.” Mark Twain

    Aug 19, 2016 Electoral College and the National Archives

    Every four years the Office of the Federal Register — part of the National Archives and Records Administration — administers the Electoral College. The Office of the Federal Register informs the governments of the 50 states and the District of Columbia what is required to fulfill their duty under the Constitution to elect the President and Vice President of the United States. This video explains how the Electoral College works and the Office of the Federal Register’s role in collecting the documentation Congress needs to count the Electoral College.

  • joec

    This is not so much about this election as all. But all of them.

    I worked my first election in 1960, JFK vs. Nixon, picked up some returns, then totals by telephone taking telephoned totals to a newspaper pool in Birmingham, AL. That required a huge PBX crew. The state split its electoral vote – 6 for Harry Byrd and five for JFK.

    The notable thing I remember about far too many elections is relatively fast moving white election lines and long backups at black precincts.(Southern urban black voters were not blocked as energetically as in rural counties).

    Two potential flaws in the Republican Game plan .

    The GOP bigotry in general has antagonized ethnic communities.

    The second is Northern Virginia. A lot of those people are Federal workers or relatives – or otherwise involved.

    I would hope we would see some uniform standard for federal elections if no other.

    • Gelatinous_Cube

      Thanks for the recollections! It would be interesting to know the long-and-short-term history of EV allocation within states. What caused Alabama to move to winner-take-all? (Obvious guess: minority disenfranchisement.)

    • Prairie Pundit

      As a DC resident and employee of an agency on the Republicans’ “must kill” list, I find it remarkable that your second point is so seldom raised in discussions about Virginia’s apparent trend toward blueness.

  • SoddingJunkMail

    Reports all over the place of very high turnout. A change from expected has to be good news from the party hoping to pull the upset, yes?

    • bks

      Contrariwise high turnout is supposed to be good for the Democrats. There’s a maxim for every scenario.

    • JeffE

      Typically, though not always, higher turnout better for Democrats. Why do you think Republicans have spent last 4 years trying to limit voting?

    • Debbie Lefkowitz

      No, definitely not. Higher turnout is good for Democrats this year, since low propensity voters tend to be younger and more liberal. Unless you buy into the Trump shy voter notion.

  • Suzanne

    I will be using A trusted site for voter info and results.

    This site is for California, Illinois and New York. It’s a collaboration of the League of Women Voters California Education Fund and Maplight.

  • Scott

    “In other news, Buzzfeed has announced criteria for scoring the forecasters. They are using Brier scores and root-mean-squared errors.”

    The next logical step is to take that, as well as all of your polling models, and take a weighted median based on confidence and accuracy against past elections.

    Once we get enough models to aggregate the polling aggregators, then we create a model to aggregate them together. Repeat until you have absolutely no error margin – that’s the way statistics works, right? (this is a joke)

    • Sam Wang

      Reminds me of collateralized debt obligations – repackage things, mix them up, and you can’t see where things came from. No way!

    • Drew

      Not sure about this. It seems most models agreed (roughly) on the mean, but not the uncertainty and all got the same states wrong. Seems to warrant a distributional comparison. So if we have the histograms of multiple models, we can compare them in an honest (proper) procedure. That would give a better guess of who was “right”

Leave a Comment