Princeton Election Consortium

A first draft of electoral history. Since 2004

In which I write of paint continuing to dry

October 22nd, 2008, 10:32pm by Sam Wang


There’s just so many posts like this a guy can write. Today, Obama is still crushing McCain. Still. Crushing. McCain. The Popular Meta-margin is approximately Obama +7.5%. It would take that much shift in state polls to make the Median EV Estimator a 269-269 dead heat. The national margin is Obama +7.0+/-1.1% (n=10, surveys spanning 10/17-21). These measures are consistent.

The recent downtick in the Median EV Estimator comes from to the graininess of the Electoral College. In particular, Ohio and Florida are large and hotly contested states. A similar thing happened in June and July, with Florida-sized jumps. But the Meta-margin remains large because in other key states, the margins are stable. To see all the details, see our semi-crunched data here, derived from the Pollster.com feed. Each row contains: Dem win probability (%), median margin in %, win prob with Obama +2%, win prob with McCain +2%, state name. There’s nothing good for McCain here. Play with the pop-up map; you’ll see.

One of you suggested that since I am a partisan, I might be gaming the analysis. Them’s fightin’ words! It’s like testing hypotheses in my lab; honesty with data is paramount. But if you are worried, go look at www.electionprojection.com, which I believe is run by a Republican.

As I’ve said before, it’s time to focus on downticket races. This is, without a shred of doubt, what rational actors on both sides should do. Based on a recent poll showing Ronnie Musgrove (MS-D) and Senator Roger Wicker (R) within 1% of one another, I added this race to my list of knife-edge Senate races at ActBlue. Republicans, go to the National Republican Senate Committee page. Don’t give to the RNC, which gets blown on kicky ensembles for Sarah Palin. And of course people on all sides should read my book on the brain, an excellent diversion!

At times like this, extreme commenters on both sides become more prominent. Those of you on the left won’t stop speculating about vote-stealing, the Bradley effect – all the old hits. Sorry, guys. You’re winning this one, bigtime (excellent analysis by Charlie Cook, as usual). Meanwhile, on the right, it’s more on the Bradley effect, as well as nuttery like this and this. In the interest of rationality, I’ve been screening a few of the comments. But why don’t I let things run loose and see if anything happens. I’ll come back by after I’m done hosting RBG.

Tags: 2008 Election

42 Comments so far ↓

  • Larry

    One question about Undecided Voters…

    After reading you article it occurs to me – you say they simply require this extra time to come to correct conclusion. I don’t have any real problem with that. But couldn’t we test that theory by moving the election day without notice to like Sept 21, 2008 so they would have had to decide sooner? Force them to either use the information they have to make a choice or not.

  • Sam Wang

    The corrections by William and Michael are technically true, but they are highly unlikely to be invoked in the coming political climate.

  • AlanJae

    That’s good to hear, Sam!

  • Michael

    Well, if you want to get technical, the Senate does not impeach, the House does (on a simple majority vote, I believe). The Senate requires a two thirds vote to convict on an article of impeachment.

  • William

    “The U.S. Senate has three thresholds for action: a simple majority (51 votes), a three-fifths majority (60 votes), and unanimity (100 votes).”

    There’s actually a fourth threshold for action–67 votes, required to 1) impeach 2) pass a constitutional amendment and 3) overturn a veto.

  • Sam Wang

    AlanJae – I believe your concern is misplaced.

    The U.S. Senate has three thresholds for action: a simple majority (51 votes), a three-fifths majority (60 votes), and unanimity (100 votes). Various forms of action may be blocked by one, 41, or 51 Senators. The point of reaching 60 votes is to increase the likelihood of getting major legislation passed at all without fear of obstruction.

    Also, the Senate is an institution naturally inclined toward compromise and bipartisanship. The U.S. Congress is often likened to teacup resting on a saucer. They felt that hot passions in the majoritarian chamber (House/teacup) could be cooled when they spilled over into the consensus-requiring chamber (Senate/saucer). Majority and minority Senators alike offer amendments, which are often considered.

    Finally, there is a dynamic specific to the U.S. situation. Congressional Republicans are rather recent sufferers of the malady you mention. I saw this in person when I worked for a year as a member of House staff, then Senate staff. Congressional traditions of consultation and bipartisanship went out the window when they came into power in 1995, the year I arrived. The older traditions need to be restored. Also, the Republican Party has lost touch with its avowed basic principles. Some purifying time in the wilderness will improve their prospects in the future – and eventually help the entire nation by making them a more positive force for pushing back against Democrats. To put it another way, my country needs partisanship in the good sense. We don’t have it now.

    Finally, my local Congressman, Rush Holt, points out that bipartisanship is easier when the majority is not under threat of losing its status in the near future.

    To reiterate, if readers here want the maximum leverage over next year’s legislative action, they can achieve this by contributing to Senate races. For Democrats, go to ActBlue. For Republicans, the NRSC is appropriate.

  • Helen Good

    To Alan Jae:
    Here in the UK the polls are saying that the electorate trusts Brown rather than Cameron in the current crisis, but will vote for Cameron rather than Brown in an election. I think boredom may be a factor, and the desire to give the other side a fair go, as well as the sins of Blair being visited on Brown.

    And, because I am a pedant, to whoever was misquoting Acton:
    It is ‘Power tends to corrupt and absolute power tends to corrupt absolutely.’ Even if Acton was correct, a president with a filibuster-proof majority isn’t bound to go bad.

  • AlanJae

    Just like to throw in my two bob’s worth on the filibuster issue. In the 2004 Federal election in Australia (my home country), the conservative party gained enough seats to have the majority of members in both the Senate and House of Representatives. This allowed them to create and pass laws without the input of the opposition party.
    While conservatives all around the country were pretty cock-a-hoop about this unbridled political power at the time, it came back to bite the government in the ass at the next election. Basically, without the requirement to consult and compromise with the opposition to pass new legislation, the government became more and more idealogically extreme. Their control of the Senate and House of Reps allowed them to pass numerous contentious / unpopular laws (e.g. favouring employers over employees, hardline immigration). This use / abuse of power had the duel consequence of disenfranchising voters and removing any responsibility for bad policy from the opposition.
    Thus, when election time came around in 2007, the conservatives were trounced. Partly as a consequence of their long time in power (10+ years) but also for the perception that they stopped governing for the electorate once they gained control of both houses after the 2004 election.

    So Democrats, be careful. If everything does come up Milhouse for you in this election, the Republican party will be able to wash their hands of any unpopular or ultra-partisan policies enacted during the reign of Barack and perhaps deliver an unexpected ass kicking next time around.

  • Sam Wang

    Evans – your approach is not bad at all.

    Regarding the 30% estimate, here is how he (or anyone) would make such an estimate: start with today’s Senate polls, then take a guess about how much the polls may be off, or may change, by Election Day. As it turns out, it would take a rather large swing of opinion, >5%, to make the distribution broad enough to get to 30% for 60 votes. This is why I am skeptical. I don’t believe the assumption, and I suspect that it’s a case of Silver saying something exciting for the gallery – kind of like what all those hated TV pundits do. Most people don’t know enough about statistics to judge these statements, and simply accept the absurdity.

    gprimos1 – Are you kidding? A “wild” swing? That fluctuation is rather small. It only looks large compared to the steadiness of the overall estimate over time.

    The answer to one of your statistical questions is that the median is more resistant to swings due to its ability to reject outliers. However, it is a less efficient statistic, meaning that it gives error bars that tend to be somewhat larger. Using it is basically my version of guarding against unknown problems in pollster methodologies.

    You have also conflated your median/mean question with another question, whether to weight data. There is such a thing as a weighted median, but I decided it didn’t add that much to the analysis.

    One course of action you did not describe is to use data over a longer time scale than one week. This would require identification of the appropriate time scale. I may go down this road on Election Eve. But it involves another layer of algorithm development. I’ll wait and see if it’s needed.

  • gprimos1

    The median EV graph is fluctuating a lot yesterday and today, largely on a few polls in OH and FL. I wonder if it is a mistake for the model to use the median instead of some kind of weighted average. Wouldn’t the mean be more resistant to these wild swings?

  • Michael K

    Re: Blue Dogs: I wouldn’t go as far as Mike L. (that Republicans are better off supporting conservative Democrats). But I think he touches on an important point:

    Instead of basing our support strictly on the closeness of the race and party ID, shouldn’t we also research and factor in the candidates’ positions?

    Some Senate races feature a much starker contrast between the candidates than others.

  • Mike L

    I’l I were a Republican, my money would get best value donating to Democratic Senate candidates in close races.

  • Mike L

    I am addicted to your site for the second election now, and we all owe you a debt of gratitude for your time , energy and sagacity.

    Re: the Byzantine Generals

    I’d apply that analogy to discussion of the possible 60 seat Senate Majority. The Blue Dogs, mainly Southern Congress Folk, in both Houses in past two years on numerous major votes spoke against the Democratic Party majority .

    Adding more conservative Senators/Byzantine Generals from some of these close contests will bring in more “mavericks” who will undermine other Democratic Senators even worse on funding bills and Supreme Court Nominations than if they were Republicans voting along their own party lines.

  • Evans

    Yeah, there’s only so many ways you can say and analyze landslide, but we appreciate your attention. I’m curious about your interpretation of 538′s seat projection (>30% chance of 60 senate seats). I think you mentioned thinking Nate’s number is a bit optimistic, but what would you do differently?

    For my part, to get a super-rough estimate, when I see a Dem leading by double the margin of error, I just count it as a win for sure and add ‘em up. That gives 57 dem seats with 5 toss ups (where recent polls are within twice the margin of error). Dems appear to have the lead in 2 of the remaining races (MN and AK) albeit unconvincingly, GA is way too close to call, and the other 2 (MS and KY) appear to have a Republican frontrunner.

    So getting 60 may be a toss up, but 58/59 seems decently likely, with some chance of up to 62… so my rough estimate seems to confirm Nate’s picture. In my line of work, however, order of magnitude is generally good enough, what’s the real statistical take?

  • William

    I think with 58 or 59 we’ll still have a filibuster-proof majority unless every single Republican votes against it–and they won’t, since we can probably pull off Chuck Hagel or some other moderate Republican. And the Republicans won’t take Lieberman–he’s only useful to them as an example of bipartisanship.

  • Ron Johnson

    John:

    I agree. If the Dems are below 60, I think they will tell Joe he isn’t welcome to caucus with them anymore. I think it would be good to get rid of him once and for all. As a Repub, he will have the lowest seniority, no decent committee assignments, and the lousiest office available. Not as bad as he deserves, but close.

  • John

    Any count of 60 that includes Lieberman is no more valuable than 59 without him.

    Except from a narrative point of view, I suppose.

    Personally, I think it hurts the party more to pander to him than it would to be just short without him.

  • Frank

    Re your Calif. colleague’s argument, Republicans have an interest in denying the Democrat a mandate, but I agree with you that the agenda is better advanced by control of the Senate than by a mandate. Anyway, Obama will probably not get a mandate this time, as he’s presently below the historically average advantages in both the popular vote (which has averaged 8-10%) and the electoral vote (40-45%) (http://www.presidency.ucsb.edu/data/mandates.php).

    Re the list of Senate campaigns that you recommend for donation (Ore., Minn., Miss. [Musgrove], Ga.), I agree with Michael’s earlier comment (“Knife-edge Senate races,” 10/10) that Alaska is also worthy. I’m not as worried about Minn. and Ore., where Democrats have 3-4% leads that are widening. The Democratic candidate in Alaska has a 1-2% lead that is narrowing. I expect that if Democrats get to 60 seats, the last two states in will be Alaska and Georgia.

  • Michael Slavitch

    And (sorry):

    Today’s message is ” Obama: 364McCain: 174″

  • Michael Slavitch

    Correction:

    “Cumulative aggregation of statistics at the state level.”

  • Michael Slavitch

    Polling is not a black art, it is sampling theory. It is made much more transparent through the aggregation of polls that change individual poll bias or poll error into outliers.

    Aggregation of polls is a statistical form of byzantine agreement, one that secures the message against general adversaries.

    The Byzantine Generals problem describes a group of generals, each commanding a division of the Byzantine army, encircling a city. These generals wish to formulate a plan for attacking the city. In its simplest form, the generals must only decide whether to attack or retreat. Some generals may prefer to attack, while others prefer to retreat. The important thing is that every general agrees on a common decision, for a halfhearted attack by a few generals would become a rout and be worse than a coordinated attack or a coordinated retreat.

    The problem is complicated by the presence of traitorous generals who may not only cast a vote for a suboptimal strategy, they may do so selectively. For instance, if nine generals are voting, four of whom support attacking while four others are in favor of retreat, the ninth general may send a vote of retreat to a few generals, and a vote of attack to the rest. Those who received a retreat vote from the ninth general will retreat, while the rest will attack (which may not go well for the attackers).

    Byzantine fault tolerance can be achieved if the loyal generals have a strategy to compare notes with each other such that conflicting information is disregarded.

    In Sam’s case the comparing of notes is the cumulative aggregation of statistics. The whole point of aggregation is to eliminate lies from the system.

  • Confidence is Scary | TechnoTaste

    [...] we know now? I think the data problem is the more dangerous one. As a commenter on PEC points out, polling is a ‘black art’, and though we gain a lot by doing a meta-analysis, we’re still at the mercy of the [...]

  • Michael

    And of course people on all sides should read my book on the brain, an excellent diversion!

    Outstanding. You see, while all of us, Liberal, Conservative, Democrat and Republican alike, have no trouble at all mustering infinite amounts of righteous indignation over polling biases both real and imagined, blatant self-promotion is something we all can agree on. ;-)

  • Ginny Mayer

    I’ve noticed a lot of “we’ve got them where we want them” talk from the McCain camp, and “Things are close and could change at any moment” from some of the news media, all of which makes Democratic poll-huggers like myself nervous. Yet according to Sam Wang’s 10/22 post, I should not be concerned; I will have to keep revisiting this site until November 4.:)

  • Ron Johnson

    Thanks for your excellent work. It’s really refreshing to see someone provide the basis of their analyses. For those that don’t like the outcome and blame it on your bias, I think you provide sufficient information for them to come to their own conclusions. I don’t think they will like them either.

  • Steve Roth

    “One of you suggested that since I am a partisan, I might be gaming the analysis.”

    This doesn’t surprise me. On my blog, where I post occasionally on polling, I’ve had a huge number of visitors recently who got there by googling for “conservative polling” or similar. You too?

    The human inclination to frantically seek out ratification of delusional beliefs never ceases to amaze me.

  • Sam Wang

    Brian, that was an error on my part – I assumed undecideds would break unevenly. It was a strong lesson to me in the dangers of assumptions not based on hard data.

    The graph at left showing 2004 data is unadjusted. Bush held a narrow, unchanging lead in the closing weeks. This analysis called the final outcome perfectly.

    Elizabeth, thank you. There is nothing like flattery to keep me doing this.

  • Elizabeth Duvert

    Wonderful work and you’re still funny!

  • Brian Estabrook

    Thanks for the continued analysis, Sam. Excellent work.

    I don’t believe you to be a partisan poller in any sense.. but I do have a question for you.

    In doing a little bit of research on your site, I noted that less than a week before the 2004 election, you predicted that John Kerry would get over 300 electoral votes. Presumably, this is what the polling showed at the time. Do you have fears or concerns about this election based on what happened in 2004?

    Just curious about how the mistaken prediction in 2004 affects 2008.

    Thanks again.

  • Sam Wang

    By the way, here’s a recent comment from a colleague in California. It’s a counterpoint to my marginal-advantage point of view.

    Your arguments make lots of sense based on the marginal benefit. BUT: having Obama win by a landslide — instead of a smaller margin — has intrinsic value that is not measured just by who is elected. If he wins by a landslide, it is a mandate, a collective national agreement that change really was wanted, and a shot in the arm for a new generation of empowered young voters that they can control the future. Also a lesson for the Republican party that their lurch to the right needs to end.

    All my money is still going to Obama.

    I don’t agree, but it’s a good point. Also, the marginal-advantage argument still pertains from the Republican point of view.

  • Nicholas J. Alcock

    Dear Sam,
    I disagree that this potential big Obama victory is akin to watching paint dry(I remember some of my old university lectures!, not yours I hasten to add). Every American presidential election is a rich tapestye.g. will Obama achieve the Dems target of breaking the Reps hold on the South. In a big victory, will they be concerned that the deep South and Appalachia could have remained stubbornly Rep?As for the Senate and filibuster, FDR’s Supreme Court fiasco should warn of hubris. as Lord Acton said, “power corrupts but absolute power corrupts absolutely.”

  • egc52556

    As always, the polls (and meta-polls) are predictions of future behavior, not the actual behavior. The campaigns and partisans have no choice but to keep the excitement high (despite the numbers) until the voters have actually behaved… uh… voted.

    Love the site. It’s my main source for polling truth.

  • Sam Wang

    In general, television news is a highly unreliable source of information. This site has several advantages: the data are there for you to see, I cover a very limited range of topics (though I guess cable news does too), and the economics of reporting polls makes it easy for me to add value.

    If news organizations did aggregate polls, they could report, based on facts, that a very big story is whether the Democrats will attain a 60-seat majority in the Senate. That has bipartisan appeal; after all, the filibuster is one of the Republicans’ remaining hopes to block Obama’s plans.

  • Paul

    One point about shifting support away from the presidential campaign: Obama’s team is running an amazing, massive get-out-the-vote effort. I’ve seen it firsthand. Across the country, they are coordinating with the DNC and local candidates — so that effort floats all boats, from the top of the ticket to the bottom.

    And they desperately need every door-knocker they can find.

    I agree with Sam that at this point, money is probably best directed to knife-edge senate races. But I think that volunteer time is very well spent at your local Obama office. That time pays dividends across every race in your community. Please, everyone, get out there and volunteer at least once in these last two weeks.

  • Frank

    If campaign donations should rationally shift from the Presidential race to Congressional races, shouldn’t Obama encourage this?

    Thanks for linking to Charlie Cook’s article. Two comments occur to me.

    First, early voting — which “as many as one-third of voters” are “likely” to do this time — could be incorporated formally into models such as yours, with additional data that are probably available and a few additional assumptions. In a different year, that might importantly alter the prediction.

    Second, in one sense, it’s too bad that Obama will win in a year when external circumstances “would seem to almost seal this outcome irrespective of the candidates fielded on each side.” Great (from my point of view) that he wins, but better if he can take more of the credit for it.

  • Glenn

    Sam, as always, much respect.

    Still, I just finished with Rachel Maddow and I must add my name to the list of presidential-pessimists. With Pennsylvania’s Murtha spouting off about his redneck constituency, and Barney Frank, AKA Lou Costello gleefully declaring “there’s plenty of rich people for us to tax out there… to say nothing of Joe Biden, well intentioned or not — I want to scream at our very public surrogates: ‘SHUT THE F**K UP!” Dems have not won a presidential election IN 30-years save Clinton, so I suggest to our overconfident friends — congressional, gubernatorial, senatorial and otherwise — to follow the lead of our pragmatic LEADER and ZIP IT! Overconfidence should be reserved for sites such as this, where mikes and cameras aren’t poised to capture inane comments made by those who should know better.
    Thanks again Sam. Now, I think I’ll go paint that wall… again… :>)

  • Alan

    Love the site. And I admire your confidence. But while the analysis of polling data is a science (in which I have full confidence in your ability), obtaining that data is a black art. On Nov 4 we will learn how good is the data. Actually, not even then – as you yourself have argued, the data offers, at best, only a snapshot of the present moment. But admittedly things DO look good – and the Dems have too many pessimists like me and too few optimists like you!

    On a related note, we pessimists (and presumably the unbridled optimists as well) would appreciate some interactive maps with shifts bigger than +/- 2%. Thanks!

  • Observer

    Sam, you are about the last person I would think fiddles the data for partisan purpose. You have been as religiously dedicated [just a figure of speech] to using all polls without fear or favor as anyone could ever ask.

    Not your fault the economy blew up, or that the McCain campaign imploded, now is it.

    Of course Obama is going to win from here. That’s a good thing by me; just sad that we had to trash the economy and a couple of foreign countries before we could turn away from the Neocon heresy [not just a figure of speech].

    Chilling my champagne for the eve. of Nov. 4.

  • Marc

    Just for grins I went to http://www.election-projection.net, not to be confused with http://www.electionprojection.com.

    The former has an Interactive Presidential Election Probability Calculator. I entered the state win probability from stateprobs.csv, set the state-to-state outcome correlation factor to zero, and ran about 100,000 simulations. Although, the histogram stabilized much sooner than that.

    For what it’s worth, the Expected Electoral Votes (by which I think he mean’s median) came out to 339.8 EV for Obama, compared to 340 EV here. And, the histogram is virtually identical to the one here. It has the same four large spikes and then the ten or so smaller spikes.

    I can’t compare it completely, since there is not much of a scale on either axis of the histogram.

    It’s not clear exactly how Election-Projection.net calculates the win probability for each state. There were significant differences for about 8 states: CO, NV, MO, NC, ND, FL, OH, IN and MT.

    I also ran with their state-by-state probabilities, again with the state-to-state correlation at zero, for 100,000 simulations. It came out with 335.6 EV for Obama. But, the histogram was very different. It was still a little spiky, but much more bell-shaped.

  • MM

    Thanks for the explanation. I couldn’t understand how two states could go back to white but the meta-margin get bigger and probably still don’t understand, but feel a little reassured.

  • Sam Wang

    Of course I am still extremely interested. But I am unable to feel much suspense about the presidential race. On the other hand, the Senate has me riveted.

    In 2004, I made the point that with quantitative analysis, you can identify how to deploy your limited time and money more efficiently. I’m making the same simple point this year. I’m really glad people have given so much.

  • Reid

    Thanks for continuing to analyze trends in your data despite your waning (academic) interest in the presidential election. The tracking polls based on “traditional likely voter” projections seem to be generating lots of headlines. I accepted your point about pollsters being remunerated for promoting flawed polls that create buzz for media outlets, but the downtick in the meta analysis had me concerned.