Princeton Election Consortium

A first draft of electoral history. Since 2004

The Problem With Polls, or…Are Senate Democrats Really Doomed?

October 15th, 2014, 11:49pm by Sam Wang


TNR essay:
Here at PEC, the calculations are built on the assumption that on average, polls provide an unbiased measure of eventual Election Day behavior. This assumption is our strength and our Achilles heel, and it is the topic of my new piece at The New Republic. The supporting calculations are here.

In the 2010 and 2012 elections, Democrats outperformed state-level polling medians by an average of 2.7 to 3.7 percentage points. That’s a substantial jump from previous years. To put this in perspective, the Senate Meta-Margin, defined as how far opinion would have to swing in close races to make Senate control a perfect toss-up, is currently R+1.3%. A polling error of 2.7-3.7% would reverse that margin. I have no idea if such a large error will happen this year. That would require knowing the reason(s) for polling errors, which could be multiple. However, the fact that it has happened in the last two election cycles does make a person pause. For this reason, the probability in the banner is a fairly soft number.

Usually there are 3-4 tight Senate races per year. At this point the playing field has expanded to seven: Colorado, Iowa, Georgia, North Carolina, Louisiana, Arkansas, and Kansas. These are critical for both sides. See the ActBlue and NRSC links at left.

Update: over at FiveThirtyEight, Nate Silver takes a long view, examining all Senate polls from 1990-2012. It’s a good piece of work. However, I think a more appropriate calculation for the current situation would be to focus only on races won by close margins, since polling errors are greater in blowouts, and biased toward the winning party. He and I get very different results for 2010, which makes me suspect that a deeper look at close races would be interesting. He posted his numbers; if anyone cares to delve into this more, I’d be interested in seeing (and sharing) the results.

Update 2, 5:00pm: Maybe Silver and I have both missed the true pattern: midterms vs. Presidential years. In his results, the median absolute error in Presidential years is 0.65 ± 0.6% (SEM). In midterm years, the absolute error is 2.9 ± 0.7%. These are different (p=0.03). The difference is even larger if one replaces his 2004-2012 numbers with my close-race data. Basically, midterm polls can very plausibly be off…but in which direction?

Tweet this post:

Tags: 2014 Election · Senate

65 Comments so far ↓

  • Joe in Seattle

    Oh must just be me. Nevermind. It’s back.
    So if the Democrats outperform the polls for the third election in a row, they hold the Senate. Interesting.

  • Joe in Seattle

    Hey, what happened to this morning’s post? The one showing which party outperformed the polls in which elections? That was great!

  • JayBoy2k

    Sam,
    You are correct. I did not do my homework before I penned the last post. Just searching on the net got me this link & comment:

    http://www.esquire.com/blogs/politics/nevada-election-results-2010
    “”"…Why are these numbers so different from Rasmussen, Mason-Dixon, et al? Well, first and foremost, Nevada is unlike any other state, in that it has a comparatively transient population, with lots of newcomers, and a sizeable percentage of that population works in the gaming industry — the casinos. Traditional polling firms don’t call during the day, and so you miss most of those people, who work in significant numbers at night. And if there was ever a population that communicated with the world exclusively via cell phones as opposed to landlines, this is it. (The Reid campaign has polled cell-phone users aggressively.) The research shows that when you go strictly by the people who you reach on the first try, Reid is 2 points down. But in the subset of people who you reach on the second or third try, Reid leads by 11 points. “”

    There are additional articles just after the 2010 election that point to the mobility of the Hispanic voters, the harsh positions of Angle, and the difficulty of contacting that group using normal polling methods..

    It would seem (no guarantees) that polling firms would update their process and procedures for Nevada after the 2010 embarrassment.

    The thing that is most irritating are those polling firms who should adjust and do not. Hopefully, as a result of this election, Alaska polling procedures will be updated for 2016.

    • Sam Wang

      Voter mobility a hard thing to address by any telephone-based method, since recent arrivals may well have out-of-state area codes. For this reason, Internet-based methods interest me.

      Relevant: here‘s a map of which states have the most residents born out-of-state. And here’s my Alaska piece.

    • securecare

      I agree with you about the pollsters needing to adjust their behavior but I suspect it will take a major credible new player to shake the old hand’s behavior up.

      Time will, as always, tell.

  • JayBoy2k

    A great article chock full of data (and some analysis) from Sean Trende.
    http://www.centerforpolitics.org/crystalball/articles/what-to-expect-from-senate-polls-in-the-final-days/

    One key point is that some Incumbents (AK, LA, AR) likely need a personal black swan event in their favor just to win their state. Another is that Harry Reid’s 2010 win was basically an outlier.

    One of his comments: In fact, to predict a win for Sen. Mark Begich (D-AK), you’d basically have to predict something fundamentally dissimilar about 2014 vis-à-vis 1998 to 2012. Those arguments have been made — the potential success of the Bannock Street project to boost Democratic turnout or a potential collapse in the accuracy of polls due to declining response rates. But those sorts of objections to polls are made every cycle: We don’t really have a good basis for doing anything more than shrugging and acknowledging that this time could be different.

    • Sam Wang

      I don’t think it’s good enough to dismiss a unanimous string of Nevada polls as an outlier. Instead, ask why.

      See my Alaska analysis. I actually think similar factors are at work…but that they probably favor Sullivan, not Begich, since movable voters in Alaska are likely to be ideologically quite different from movable Nevada voters. So yes, Alaska will likely go R, but possibly by a larger margin than polls indicate.

      I am not convinced about AR being done yet – too much variance there. LA, maybe, but it’s also a bit close.

    • securecare

      “…We don’t really have a good basis for doing anything more than shrugging and acknowledging that this time could be different.”

      Translated, that is saying we are “down in the noise” so don’t be surprised at the outcome and don’t blame us if we “blow it”. Basically the same situation Zeke Hunkaburning is suggesting above.

      I think that is a totally reasonable perspective.

  • Leslie A. Joy

    Hey,

    Long story short: I have Asperger’s – just found out a year ago – anyway – elections and socioeconomic issues and how those socioeconomic issues relate to elections is my “autistic kid interest” for lack of a better word.

    I live in a swing district and in 2012, canvassed the area 3 times, but the data was lost. I’ve been following 2014 in my district disturbingly closely (I’ve spent the past years mainly unemployed) and while I’m not working on any campaigns I’ve talked with a lot of different campaigns in the area.

    I have some ideas on what you and Nate might be missing. What’s a good way to contact you?

  • JayBoy2k

    The problem on speculating on how bias changes a specific state election is not the size of the bias, but the existing single most recent example. In 2012, the only close race which favored a R was Heller in Nevada favored by 2%. The R bias in the polls was 3.4%. The other examples are in 2010. CO and NV had R close favorites lose, but in deep blue state Illinois, Kirk won.
    There does not seem to be a direct relationship between having a R bias in pre_election polling and having any R in a close race lose.

  • Zeke Hunkaburning

    The upshot or uptick (whichever pun you prefer) of all this gnashing of teeth, rending of garments and rendering of fat is simple: all this incredible statistical analysis from multiple sources has accomplished is to highlight the degree of uncertainty for the outcome of the election. The analysis from all probabilistic comers boils down to this — it’s too close to call given the margin of error and the degree of uncertainty in what we don’t know.

    In other words, “We don’t know what we don’t know so pay attention to our numbers, but be prepared for a surprise.”

    If I predict the GOP and the DP each come away controlling 50 seats that’s (about) as likely an outcome as the GOP controlling 52 and the DEMs 48.

    You can take that to the bank. (Laughing out loud)

    Now let’s wait for the “Ebola scare” to swing the polls even further toward GOP Senate control as Obama’s crises of leadership deepens. Disappointer-in-chief, yes, but still the Chief and still time to show leadership despite a fickle Democratic field.

    Better get out the vote. Don’t count on any margin of error in polling.

    • Phoenix Woman

      Of course, once the number of actual Ebola cases in the US peters out, and sanity returns (Obama himself made a point of visiting and hugging recovered Ebola patients last week) the GOP will look like idiots for pumping the fear organ.

  • Canadian fan

    The point about 2010 – the closest precedent to this midterm, and the one generally cited as a particularly terrible Democratic year – is that Democrats outperformed the polls in their collective Senate races by an average of 3.7 %. The Republicans did indeed under-perform their estimates, but by only a fraction – 0.9 %. In addition, the Republicans had a huge 7 % generic advantage to begin with. The point is not whether the 3.7 % made a dramatic difference in 2010. The point is that 3.7 % would make a dramatic difference this year.

  • Zathras

    Sam, I looked at your code files, but I didn’t see any place with historical polls/results data. Do you have that publicly available? Or is there another repository for this data?

    • Sam Wang

      Polls all come from HuffPollster, and they provide those. Results data are as archived in the .csv files here. We have 2012 files, though some housekeeping is needed.

  • Insidious Pall

    In viewing previous cycles, two data points are taken from the most recent 2012 election, a year in which African American turnout has been reported as exceeding white turnout. It seems plausible that the presence of Barack Obama on the ballot was a primary factor. Absent that factor in the current cycle, turnout may not help Dems outperform the aggregations. In 2010, Obama was also not on the ballot, but it was a wave year which would seem to introduce a higher degree of volatility. Just wondering.

  • Canadian fan

    In the previous midterm election – 2010 – Democrats outperformed every Senate race that Real Clear Politics had designated as a toss-up by an average of 4.2 % against the average estimate that RCP had finally designated for those races. Now, of course, that was the big Republican wave year, and yes, 4.2 % would not turn a race in all instances, as Republicans had a dramatic generic advantage that year. But in this year, that would change the results dramatically. What is so remarkable about 2010 is that that was the year where the Democrat electorate was more than demoralized. There was no Bannock Street Project. And yet they showed up at the polls to a degree that went well beyond pollster’s predictions. This addresses Sam’s point about close races, and how they could be dramatically affected by a modest shift ( eg. 1.4 % ). If anything, the intensity of the GOTV is quite beyond that of 2010.

    • Matthew

      However note that the swing only changed the outcome of two races, Colorado and Nevada. Democrats were ahead in three which they ended up winning and Republicans were ahead in four and they ended up winning two. Also Nevada accounts for more than a quarter of the swing. The polling error in Nevada was 8.3 and it was a raw total of 30 among the seven states. Third, all of the tossup states except West Virginia were blue in 2008. Obama won an unweighted average of 55% in those seven states in 2008. This year the playing field is markedly different. In 2012 Obama lost in seven of the ten states. His unweighted average vote total in those ten states was 44%. Maybe the “color” of the states doesn’t matter but if it does Republicans have the “home field advantage.” Fourth, 2010 was a “wave” year and so was 2006. If you look at the RCP generic Congressional vote you will find that in 2010 Republicans underperformed the generic polls by 2.6 points and in 2006 Democrats underperformed the generic polls by 3.6 points. Is it possible that in “wave” years polling overestimates the size of the wave? If that is true the polling this year might be more accurate than it was in 2010.

    • Davey

      I agree with your remarks, and add that this cycle will be a great test for Bannock. With so many races virtually tied, a bunch of them going dramatically against the polls in one direction or another could indicate whether the Dems got their $60 million worth.

  • CRM

    Georgia + Kansas + Iowa = best combo to keep Democratic control

    I call that 50/50

    • ChuckA

      50/50 of sweeping all three races?

      Given current polling, I think you’re off by a factor of ten.

  • DaveM

    A purely abstract observation regarding
    Nate’s data: though the biases appear random at first glance, they do follow a pattern of sorts.

    In Clinton’s first midterm, the bias is D+3.1. In his second midterm, it’s R+4.9—an R+8.0 swing.

    In Bush43′s first midterm, the bias is D+4.0. In his second midterm, it’s R+2.7—an R+6.7 swing.

    In both cases, the intervening presidential election year saw an intermediate bias, roughly in the middle of the overall swing.

    It’s amusing to project this pattern on Obama’s presidency: the 2010 R+0.9 (note the 2012 R+3.4 intermediate step) would imply a 2014 bias of R+7 or so; that would put South Dakota on a knife’s edge and move all the less out-of-reach races into the Democrats’ column.

    • ChuckA

      I think you’re failing to consider another plausible alternative:

      When a bias is exposed, pollsters (honest ones) seek to correct it before the next election.

      When the 1994 election happened, the pollsters adjusted their 1998 LV screens accordingly, but the electorate changed. Ditto 2002 and 2006.

      What this would imply, then, is that the Republican bias of 2010 is already being compensated for in the current LV models. How that compensation plays out vs the actual electorate will be known on election day.

      Pinning your hopes on the polls being 7 points off is not wise.

    • Sam Wang

      Agree, though I don’t think it’s pollster correction so much as general difficulty in midterm polling. A 4-point error in either direction is highly possible.

  • Tim

    So what the heck happened in mid-September? The graph sure *looks* like a black swan event … except there wasn’t one. Has there been any reflection about why the model was so inaccurately sanguine about November before that time?

    It’s certainly true that the polls may be off, but I hate that the eggs are now all in that basket. It reminds me of “undecideds break 2/3 for the challenger.”

    • Sam Wang

      Agree about the uncomfortable comparison to 2004. Especially since I think the error is large…but could go either way.

      Regarding mid-September: Iowa, Colorado, and Alaska all shifted around the same time. Seemingly it is not “fundamentals”; for example, The Upshot’s Iowa and Colorado background models are D+6% and D+7%.

    • Mike

      What happened in the middle of September? The ISIS war happened in the middle of September. The polls started collapsing immediately after Obama’s ISIS speech and never really recovered. There’s your black swan event. The base depressed by more war in Iraq and indies moving right on national security.

    • Sam Wang

      That is a plausible event. However, Obama’s approve/disapprove number did not change at that time, which is not consistent with the idea.

    • Mike

      BTW, September 10th was the black swan date. Most polls ending prior, even likely voter polls, to that date had us ahead. One week after, the numbers plummeted. September 10th was Obama’s speech. I doubt it was a coincidence.

    • Edward G. Talbot

      This may have been touched on in other posts, but isn’t September when the registered voter polls really started being replaced by likely voter polls in the model in significant numbers?

    • Sam Wang

      This is plausible, though if you look at HuffPollster, it’s nearly all listed as LV. How the LV screen changes over time, I can’t say.

      And then there are specific events such as Mark Udall’s debate performance – i.e. voters finally comparing the candidates directly.

    • Edward G. Talbot

      Yeah, I guess it’s really impossible to say. A number of explanations are plausible

  • Bill

    A non-parametric median test of the out performance numbers in the New Republic article shows the difference between median out performance numbers for R’s and D’s is statistically non-significant. This is almost certainly a function of low statistical power since there are so few data points.

    However, a t-test for independent groups shows a statistically significant difference! This test probably has higher power than the median test since it is parametric, but the sample size is way small.

    Here is a place where your preferred outcome next month can determine which of these results you choose to pay most attention to. ;)

    • Sam Wang

      Can you spell that out a bit more?

      However…I don’t think it’s all that informative to compare R-favoring years and D-favoring years in that way. The question is more whether the entire distribution has a mean (or median) significantly different from zero. Also, is it correlated with other variables of interest such as Presidential v. midterm year, whether there’s a wave, and so on. And as I said, this would be best pursued using close races only.

  • Randy Haugen

    Nate makes a good point in his article,but I have to wonder if today’s polling is more accurate than it was say 6 years ago.It could be better or worse.
    Glad I am not in the business I have enough grey hair.

  • Joe

    I consider myself a polling dummy, but trying to learn. This site is helping with that, so first and foremost thank you Sam for the lessons. I do have a question though that seems is only an after thought in some polling analysis. Is the sharp and sudden drop in polling for Democrats related to the sudden switch from RV models to LV models. It seemed like all these states had Dems in the advantage or holding, and suddenly in Sept when all pollsters move to LV screens (like CNN and FOX and Quinn), there was a sharp turn downward, and races that were close but Dem leaning turned close but R leaning almost overnight. Your graph showing the meta margin seems to be bearing out this possibility. I did some research too, and in 2008, 2010 and 2012 the LV screens for some were either too tight, or were too GOP bias. Is this what is happening now, and if this has been the case for 3 election cycles prior, why aren’t pollsters either adjusting their LV screens or perhaps scrapping them all together? Again, go easy, I’m a polling newbie, beginner, freshman, whatever you would like to call it.

  • securecare

    “…races won by close margins…”

    And I have to ask what would you likely define as a “close margin”, 2% or less or slightly more ?

    I realize this is really a gut call but maybe it would be open to some sort of sensitivity analysis.

  • JayBoy2k

    Thank you. It makes sense although I am going to have to consider implications for a while. It has been a long time since Statistics and Differential Equations in College.
    Is this difference likely to disappear by midnight November 3rd? It seems that PEC will have called all the Senate races individually.
    I had considered that this might be attributable to Kansas because the feed to Upshot has PEC calling Kansas as 81% I, which I happen to agree with , given the unusual situation and polling in that state.

    Good site and articles. I enjoy both the insights and the focus on substance over opinion.

    • Sam Wang

      No, the potential error is there to the bitter end! Too hard to predict, so we will stay with no-average-bias as our assumption for now.

  • JayBoy2k

    I have been trying not to ask this question, thinking that it was explained in a sidebar, or FAQs, etc.
    Looking right now at “the Power of your Vote” , the Rs hold all their seats for 45 total, plus gain 8 seats (WV, SD, Mont, AR, LA, AK, CO, Iowa) so why is the projected number of seats 52R instead of 53R.
    I know that I am going to feel dumb at the then obvious answer.

    • Sam Wang

      You are thinking of the mode, which is the single most probable exact combination. However, we calculate the median of all possible outcomes. For example, if two races have probabilities of 50-50, then the likeliest outcome is 1 win for each side. That is still true if the races are 55-45. Compound that across lots of races, and intermediate outcomes are possible.

  • J

    There was a piece on the Huffington Post (http://www.huffingtonpost.com/david-belk/more-drama-from-the-polls_b_5935324.html) I was reading yesterday that noted something similar, using the RCP averages in the 2006, 2008, 2010, and 2012 elections. That article and this one reaffirm why I’ve come to distrust polling in the last couple of cycles now, namely there’s a lot of partisanship and bias in the polls (or the pollsters themselves have an agenda) done these days, done for groups who have a blatant interest (going beyond the usual Republican or Democratic campaign internals). For example, a poll conducted for American Crossroads or the League of Conservation Voters should reflect the true conditions in an election, rather than telling them what they want to hear (which isn’t true polling by any stretch of the imagination). An ideal polling firm should be independent, free of any conflicts-of-interest, and release accurate polling regardless of who’s paying them to conduct the poll.

    It’s little wonder why the statisticians like Mr. Wang and others, who analyze these races and the polls to give the public some idea of what will happen on Election Day, come into some difficulty of trying to create a relatively clear and accurate picture of what will happen when there’s so much bias and inaccuracy around in the polling. If all the data available is from skewed sources, then that will affect the predictions accordingly, which is not the fault of the statistician who put the data together. (This is sadly forgotten in some instances when the partisan accusations get thrown about when they disagree with the result.)

    However, if I were to rate the most reliable pollsters, my picks would be PPP, Rasmussen, and SurveyUSA, as they for the most part provide a relatively decent and non-partisan view of the races, and they’ve been fairly accurate by comparison in the recent cycles (that’s been my observation). Just my two cents anyway.

  • Mark

    Question re:Dems tend to outperform polls–does the political leaning of a state matter? In other words, do dems outperform all polls regardless of whether the state is traditionally “blue” or “red”? Seems this would matter significantly in the current elections, since most of the states up for grads are “red.”

    • Sam Wang

      I think it’s more along the lines of whether pollsters have a good likely-voter model that year. Look at my spreadsheet. The errors seem to be across the board, in both bluish and reddish states.

      As per my comment in the last paragraph, I didn’t analyze deep-blue and deep-red states on the grounds that those errors are of no importance in gauging close elections. Nate Silver did a lot of homework for his piece – it would be worthwhile to delve into it. I am hoping an enterprising reader will do so!

  • Ben

    Why these six races and not KY or AR which have the same % margin, and a higher NJ vote power (meaning greater ability to be changed by minor changes, no?) than CO. Or NH, also close? Am I miss reading the Power of your vote table?

    • Sam Wang

      See my piece in TNR, and Josh Katz’s analysis. The idea is that the power calculation only measures a naive index of voter influence. But once a race gets outside the 4% range, catching up is much harder.

      I am agnostic about AR because the variation in polls there is large. KY is less variable, and it appears to be done.

    • NP

      If Kentucky is done then Louisiana is more so. Only GOTV and incumbency with help Landrieu there.

  • shma

    Got a question about the new state-by-state November predictions. The difference between those numbers and the state-by-state snapshot seems to be an extra error term, entered as a constant. Is the plan to reduce this extra error term to 0 as election day approaches, so the November prediction falls into line with the snapshot as election day approaches?

  • wendy fleet

    Sam — I’m phoning-from-home into IA, KY, CO, SD. Should I go with lists into IA no matter what? Is that the best use of my “power” ?
    The phoning system for the various states I listed is *much* more sophisticated, but I can probably find a way to phone IA from some phoning group each time even if much less efficient. Thoughts?

    • Sam Wang

      The power calculation doesn’t take into account the calculation that Josh Katz made – it’s a statement about individual-vote power, in the ideal case that a candidate’s margin is always movable. So one has to exercise some judgement.

      Anyway, for calling you have made fine choices, though to be entirely honest, I’d skip Kentucky. If it were me (whether Democrat or Republican) I’d focus on IA, CO, GA, and maybe SD.

  • Franklin

    From looking at the maps, polls and early voting, here is my prediction:

    1) We won’t know who controls Congress on the day after the election.

    2) The result is going to be 48 D, 49 R, 1 I (Kansas).

    3) LA and GA will go into run-off.

    4) The Dems are the ones with the Plan B already set to go for #3 above (they have the money set aside for this), the Reps are not — they are assuming they will take 50+ on election night.

    • Franklin

      In a way, the above would still align with Sam’s and Nate’s forecasts, but not necessarily predict which party controls the Senate. So this isn’t going to end after election night. It’s going to be a mess into the new year.

  • Canadian fan

    Wonderful article, Sam. Both you and Nate are in agreement, because he’s also saying that the polls could show statistical bias. The very fact that Democrats have generally outperformed the polls ( as you clearly show ) should give everyone pause. Indeed, as imposing as the meta-margin is looking these days, your point is clear – a 1.5 % statistical margin for Republicans could be very easily reversed if the Democrats were found to have outperformed the polls by 1.5 %. And as Democrats have often done much better than that, the story of this election will come down to the degree that Democrats were able to outperform the polls.

    • Matthew

      Democrats don’t generally outperform the polling. They’ve done so in the last two Senate elections. Nate Silver’s article, a link is provided by Mr. Wang, shows that since 1990 Republicans have outperformed the polls six times and Democrats have outperformed the polls six times. The average “bias” since 1990 is 0.4% in the Republicans favor. The median is zero. There is no evidence that the polls have a bias towards Republicans. If you flip a coin twelve times you are almost guaranteed of getting back to back heads at least once, the equivalent of having Republican bias in the polls the last two elections.

    • Sam Wang

      Yes, though also see my comment that Silver’s analysis should be restricted to close races. Also, Silver’s data show a standard deviation of 2.6%, very wide. So Canadian fan‘s hope is not crazy – just one of many possibilities.

      I suspect errors are not related to GOTV but to errors in estimating extreme behavior, for example in wave elections. Democratic performance was underestimated in 2010, maybe because the huge GOP wave was so hard to estimate accurately. This needs more study. Silver appears to have accumulated a lot of data suitable for exploration. I haven’t delved into it yet.

    • Matt McIrvin

      Another possibility is that the vote-suppression efforts that caused so much outrage and reaction in 2012 may actually succeed in suppressing the vote in 2014 (many of them are coming into effect in earnest only now, and voter enthusiasm isn’t generally as high). That could give a significant edge to the Republicans.

    • Bert

      The question is whether basic voting demographics have shifted much from two years ago? Rural and elderly whites usually trend Republican while minorities and urban whites usually trend Democratic. Suburbs are usually a toss up. So do we see a shift in any of those groups towards one Party or another?

      What’s fascinating is that Republicans are doing far better than expected in Colorado and Iowa, while the Democrat is doing far better than expected in Georgia. What this might signal is a general unease in the electorate with “Red/Blue” politics as usual. It may be that voting trends in the next decade go in places we wouldn’t expect.

  • Joseph

    Isn’t the mere fact that the Dems are this close to holding a majority pretty much flying in the face of earlier expectations?

    • Savanna

      I heard Joe Scarborough say that that things are this close, even if the r’s take over the majority, is already a defeat for them. This is an incredibly favorable map for them. That is, if they cannot take over the majority with this map, they are truly in bad shape.

    • Matt McIrvin

      I think the cake is baking as we speak; the chances are pretty good that the Republicans will take the Senate.

      But at this point last year, I was expecting 2014 to be a gigantic Republican wave like 2010, with a massively expanding lead in the House, sweep of governor’s races, etc.

      (At one point, just to make sure, I calculated whether the Rs could remove Obama from office in an impeachment trial if they won every single up-for-election Senate seat this year; they can’t, they’re just barely short.)

      The wave is not happening, though. It’s a game of inches, and the Democrats will take the Senate back in 2016 if general conditions are even similar.

  • Billy

    He who shall not be named has an interesting piece out: http://fivethirtyeight.com/features/the-polls-might-be-skewed-against-democrats-or-republicans/

    Basically, all this article says is that any kind of small systematic error in polling will completely push the result to either side. If you take that same reasoning to Dems/Repubs under/over-performing against polls, it would mean that we are truly on an exciting knife edge.

    There are two main issues for me: (1) whether polls accurately capture turnout — we see that it may not be the case in previous years given Democrat performance; and (2) whether median-based statistical methods are sufficient to account for outliers — the fact that democrats outperform previous polls would indicate that some stronger statistical methods are needed. For example, are the polling results bimodal over a small time window? That would be pretty strong evidence that median-based corrections would no longer be applicable. However, empirical corrections always struck me as being unreliable. Then again, there’s always the issue of “garbage in = garbage out”. With the increased amount of polling we may be indeed seeing the system being polluted with low quality data.

    • Sam Wang

      That’s an excellent piece.

    • Rosser

      This topic made me revisit the Gallup daily tracking poll from 2012. According to its likely voter results, Romney won in a close election. That poll gave me heartburn at the time.

  • JayBoy2k

    Sam,
    I am trying to understand the implications: In 2012 PEC state level projections, there was only 1 seat (Nevada) where an R was favored by less than 4% and that seat did not switch due to the bias.
    In 2010, there were 3 such R favored seats and 2 of 3 did switch (Nevada and Colorado) but Kirk won in Illinois.

    So having a bias does not necessarily flip the close state elections and there may be an incumbent relationship. In this 2014 election, currently we have 4 states (Iowa, Kansas, Louisiana, and Georgia). Louisiana is the only one with a D incumbent.