In my new essay in

At this point, the median outcome is a net gain for Democrats of one governorship. This does not fit with the idea of a Republican wave. Read on! ]]>

*…and, here’s the video. It was fun, fairly substantive, I described the incredible power of Iowa voters compared with my puny vote in New Jersey (at the moment, 1 Iowa vote > 100 Jerseyvotes). Best of all, there was hardly any nerdfighting!*

Yesterday, Nate Silver and I both examined Senate polling errors. He saw no overall bias; I pointed out that recent bias has been unusually large. Both statements are true. But neither of us pointed out that the biases follow a significant pattern: midterm-year polling is far less accurate than Presidential-year polling.

From a practical standpoint, this is good news for those of you who don’t like where things have headed lately: *in midterms, Senate polling errors are five times larger than in Presidential years. *There is bad news too: *the error can go in either direction, and a GOP blowout is also possible.*

I am interested in why midterm errors are so large. In midterm elections, voter attention is lower than in a Presidential year. In 2012, the Presidential campaigns of Barack Obama and Mitt Romney were in the news every day. This year, it’s…Ebola, which I am pretty sure none of you will ever get. Yet Ebola is in the media much more than any of the aspirant Senators, Representatives, and state officials who will affect our lives. I would guess that more of my neighbors know there is Ebola in Texas than know that Senator Cory Booker is up for re-election.

With lower voter attention comes lower turnout – and evidently, lower certainty about which voters will show up to vote. Other distractions take away from the important issue of Joni Ernst’s desire to eliminate the Environmental Protection Agency…hey, pay attention, I’m talking to you! *Ebola Ebola Ebola!* There, now you’re back. Thank you.

Here, let me show you how bad the error is.

First, let’s revisit the FiveThirtyEight and PEC analyses to see how big the “midterm error” can be. Here I have sorted out Silver’s list of polling errors according to midterm vs. Presidential election years. I have reversed the notation to show the “bonus” for each party on Election Day: for example, “R by 3.1%” means that Republican candidates outperformed the last 21 days of polling (the time window in his analysis).

The directional median indicates a bonus that favors one party over the other. Over the last six election cycles, there has been no net bonus.

However, the median of the absolute values (i.e. irrespective of direction) tells a different story. In Presidential years, the median bonus is only 0.6%. However, in midterm election years it is much larger, 2.9%, nearly five times as large.

Pollsters are good at identifying likely voters in Presidential years, when the airwaves are saturated with talk of the candidates at the top of the two party tickets. But with less information about the election easily available, voter attention can be captured by other topics like Ebola, which crowd out political coverage. What effect does this have on voting? I’m not sure, though I will say that reduced turnout generally favors Republicans.

Some of you have speculated about whether get-out-the-vote (GOTV) activity could make a difference. For example, the Democratic bonus in 2012 sticks out as being larger than other Presidential years. It could potentially have been caused by improved turnout efforts. However, that is only one data point. More generally, GOTV will only provide a bonus if it identifies people who were *not* identified in surveys as likely voters. And we don’t know how well pollsters have accounted for it.

As the historical record shows, the sum total of pollster error can be in either direction. For example, 2010 was a great year for Republicans, and Democrats outperformed the low expectations set by polls. But in 1994 and 2002, also Republican wave years, Republicans were the ones who overperformed. It is not time for either side to get excited or despondent – only to work harder.

In short, all I would be willing to say based on this historical pattern is that any race currently within 4 percentage points or less is still uncertain.

**Polling errors persist, even in the home stretch**

As I mentioned yesterday, even in the week before the election, the hidden bonus can be quite substantial. Here, again, is my analysis of races that were ultimately decided by 10 percentage points or less. This calculation is based on the last week of pre-election polling.

Even at the last minute, the bonus can be rather large. Candidates could still win if they trailed by a margin of less than 3 percentage points in the week before the election. Here are the details for 2010 and 2012:

Here, negative numbers (in red) indicate a GOP final win or polling lead, positive numbers (in blue) indicate a Democratic final win or polling lead (click to see more years). In 2010, the error was enough to flip the result to the opposing candidate in Colorado and Nevada. In 2012, no such errors occurred.

With all this in mind, here is the bottom line: for a range of possible bonuses between 3% for Republicans and 3% for Democrats, based on current polling the range of outcomes would be from 49 to 54 Republicans. That range is so wide because of the unusually large number of close races this year – and of course, it straddles the 50-vote threshold for a change in control. Midterm polling, lots of close races, Senate on the edge – all in all, it’s a challenging problem!

The PEC probability calculation above assumes a slightly lower range of errors than is indicated by midterm data alone. Any model that combines midterm and Presidential historical data might be underestimating the uncertainty. Therefore, as I wrote yesterday, I suggest that a better number to follow is the Senate Meta-Margin, which is currently R+1.4%. As long as the margin remains below R+4.0%, a polling error favoring Democrats can still reasonably alter the picture. And, of course, a significant polling error favoring Republicans would lead to a major blowout. For now, nobody on either side gets to take their eye off the ball.

]]>Here at PEC, the calculations are built on the assumption that on average, polls provide an unbiased measure of eventual Election Day behavior. This assumption is our strength and our Achilles heel, and it is the topic of my new piece at

In the 2010 and 2012 elections, Democrats outperformed state-level polling medians by an average of 2.7 to 3.7 percentage points. That’s a substantial jump from previous years. To put this in perspective, the Senate Meta-Margin, defined as how far opinion would have to swing in close races to make Senate control a perfect toss-up, is currently R+1.3%. A polling error of 2.7-3.7% would reverse that margin. I have no idea if such a large error will happen this year. That would require knowing the reason(s) for polling errors, which could be multiple. However, the fact that it has happened in the last two election cycles does make a person pause. For this reason, the probability in the banner is a fairly soft number.

Usually there are 3-4 tight Senate races per year. At this point the playing field has expanded to seven: Colorado, Iowa, Georgia, North Carolina, Louisiana, Arkansas, and Kansas. These are critical for both sides. See the ActBlue and NRSC links at left.

*Update: over at FiveThirtyEight, Nate Silver takes a long view, examining all Senate polls from 1990-2012. It’s a good piece of work. However, I think a more appropriate calculation for the current situation would be to focus only on races won by close margins, since polling errors are greater in blowouts, and biased toward the winning party. He and I get very different results for 2010, which makes me suspect that a deeper look at close races would be interesting. He posted his numbers; if anyone cares to delve into this more, I’d be interested in seeing (and sharing) the results.*

*Update 2, 5:00pm: Maybe Silver and I have both missed the true pattern: midterms vs. Presidential years. In his results, the median absolute error in Presidential years is 0.65 ± 0.6% (SEM). In midterm years, the absolute error is 2.9 ± 0.7%. These are different (p=0.03). The difference is even larger if one replaces his 2004-2012 numbers with my close-race data. Basically, midterm polls can very plausibly be off…but in which direction?*

Tweet this post:Tweet

Note that the uncertainty (1 sigma) on the probability is at least 0.15, or 15% (and it’s asymmetric; more uncertainty in the D direction). For this reason, aggregators should not be showing a ones-place in the percentage; you don’t see “39%” in weather forecasts, and those are about as accurate as what we’re doing. We could also show it as ”40 +/- 15%”.

If you want to see the precise forecast of many aggregators, they’re all available at The Upshot (NYT). They just added PEC – many thanks to Josh Katz and the team there. The calculations all point in the same direction, a very gentle lean toward Republican control. However, everyone’s using the same polls, so a polling error would make us all wrong. Ponder that!

I’ll say it again – 60% is not that certain. If you flipped a coin weighted like that in favor of heads, 2 out of 5 times it would come up tails. The show’s not over.

*Update: PEC’s November win probabilities are here, as well as piped over to the NYT.*

*P.S. For general comments use the MSNBC thread. There’s a great conversation going on there.*

In the Senate, recent Iowa polling leaves us with a median of Ernst over Braley by just 0.5±0.8% (8 polls), a dead heat. Also, what’s up with Colorado and South Dakota? Finally, look where the NRSC is putting money. A big tell that they see things the same way as we do in The Power Of Your Vote (see the right sidebar). Bottom line, Democrats+Independents seem headed for between 48 and 51 seats. Suspense!

In the House, voter sentiment is more like 2012, not the wave year of 2010. Republicans will retain control for sure, but we don’t know who will win the popular vote.

Democrats appear to be positioned to pick up a few governorships. Five races are currently within one percentage point (FL, WI, ME, KS, IL), and four of those are held by Republicans.

What’s on your mind?

]]>At this moment, PEC’s probability of D+I is 49%. Yesterday it was 52%. Obviously everything is different, a volte-face. Right? Um…

Only if you don’t have a clear understanding of uncertainty. This is common among even the most experienced journalists [NPR] [WaPo]. It makes the baby Ronald Fisher cry.

For a refreshingly accurate and insightful look at how to think about knife-edge probabilities, Mark Mellman has written an excellent article for *The Hill*.

[Some people] exchange the numbers for words. “Republicans,” they say, “are likely to take over the Senate.” Well, that depends on what “likely” means to you — and research tells us it means different things to different people. For some people, likely is akin to 90 percent; for others, it could be 60 percent. Thinking about the Senate forecasts this way results in a loss of information and precision.

What [Senate forecasts] may mean is that it is a bit more likely that the GOP wins control of the Senate this cycle than that Democrats keep it.

But consider the classic illustration of probability: coin flipping. If you flipped a coin a thousand times, it should come out heads about 500 times and tails 500 times. If you weren’t actually counting, you couldn’t possibly notice the difference between that 50-50 chance and, say, 510 heads and 490 tails — equivalent to a 51 percent probability. Which is to say, the difference is imperceptible. You probably wouldn’t even notice 590 heads and 410 tails as being particularly off the 50-50 mark.

It’s good – just go read the whole thing.

For this reason, last week’s fuss over the difference between PEC and other forecasters was overblown. Now that we’re microscopically under 50% for the Democrats, I am steeling myself for a round of “aha, now you agree!” However, anyone who has any understanding of the situation should know that **control of the Senate is up in the air.** All we know is **where activists should go to make a difference:** Iowa, Colorado, and a shifting pattern of states – today, Kentucky and Georgia.

Also, I estimate that any poll-based estimate, whether at other sites or at PEC, has at least a 15% uncertainty, i.e. the true probability today is between 35% and 65%. Of course, that does not change the fact that the overall trend over the last month has been toward the GOP.

Let me finish by quoting Alan Koczela from yesterday’s thread: “There are worrying signs, even in PEC’s model. The meta-margin tanked in late September and is bouncing around R territory. And, the likelihood D+I number is below 50%….these should be worrying signs for Democrats. No need to panic, but, if you’re not concerned, you’re a fool.” Also see my reply to him.

]]>Today let’s back up a step, and not focus so much on individual polls, or even single races. A larger picture emerges if we look at recent polls in the Senate and the House, as well as President Obama’s net disapproval rating.

Taken in full, polls indicate a continuation of recent polling trends: a House that looks a lot like 2012, and at least three Senate races within a single percentage point (IA, CO, KY), with control of the chamber going either way in November. Overall, Republicans are still headed for a good year in the Senate, driven in large part by the fact that many Senators who are up for re-election ran in 2008, a Democratic wave year.

There is one piece of genuine news, which concerns the Kansas election.

**Presidential net disapproval: a slow recovery for Obama?**

First, let’s take a look at the net approval-minus-disapproval for President Obama. At the moment, it looks like he has recovered by about 3 percentage points since summer:

This number is used by political scientists and some prognosticators as a predictive tool for Congressional elections. However, we only know a little about how well its movement is predictive of downticket races. In 2012, I estimate approximately 1 Democratic Senate seat per 1% of Obama-Romney margin. This year, it’s much lower, about 0.3 Senate seat per 1% in Obama net approval. It’s a noisy measurement, so should be considered with care. The one thing I will say for certain is that this number shows no evidence of strong movement toward the Republicans.

**House outlook: Modest or no GOP gains**

Now let’s look at the generic House poll, which asks “which party’s candidate will you vote for in your local race in November?”

This graph shows the median of the most recent three weeks, plotted over time. The current range is between D+1% (CNN) and R+7% (Fox and CBS). I wrote in August that the 2014 election was shaping up to be a ripple, not a wave. There’s nothing here to contradict that. With a current poll median of D+1%, Democrats have a good chance of winning the popular vote.

There’s a distinct possibility that the seat count might hardly change at all. Using a rough conversion factor of 3 House seats per 1% in the generic Congressional poll, the range of outcomes relative to 2012 is currently looking like zero change to a GOP gain of 9 seats. Compare that with 2010, when Republicans gained an amazing 63 seats. So 2014 isn’t a wave.

**Senate outlook: Are Georgia, Kentucky, and Kansas re-entering competitive territory?**

For weeks or longer, the Georgia and Kentucky Senate races were looking strong for the Republicans. Recent results have called that into question.

First, two new Georgia polls show David Perdue (R) leading Michelle Nunn (D) by only 1 or 2 percentage points. These were taken before the recent attacks painting Perdue as an outsourcer of jobs. The next poll will be a highly useful data point.

Second, Kentucky is looking distinctly un-rosy for Senate Minority Leader Mitch McConnell (R). The four polls in the PEC median are showing McConnell +1.0±2.2%, basically a toss-up. Compare that with the 4-5 percentage point lead he showed back in August. That’s real slippage.

I’ve been writing about the Senate field narrowing to four or five close races. At this point, the field might be shifting, or even expanding. Keep an eye on the top eight races in The Power Of Your Vote (see the right-hand column): AK, AR, CO, GA, IA, KY, LA, and SD. Also note the ActBlue and NRSC links.

**Fox polls: two or three fingers on the scale**

Some of you commented about a burst of Fox polls yesterday showing a potential bloodbath for Democrats. However, as *Canadian fan* pointed out, the Fox polls assume 2010 levels of turnout, i.e. a Republican edge of 7 percentage points. As I showed you above using the generic Congressional polls, there is little sign of a 2010-style wave.

Canadian fan said that FiveThirtyEight is using a 3.6% correction for house effects to compensate for Fox’s bias. This house-effects correction is probably a step in the right direction. However, one problem is that a fixed correction does not take into account that the effect can change over time. In this case, the estimate of 3.6% bias towards the GOP might change over time. I believe HuffPollster has a more dynamic approach, which if so might be a better idea. We can make our own estimate fairly easily.

As an exercise, browse through polls where other recent surveys are available: KS-Sen, KS-Gov, and CO-Gov. You will find that the Fox polls are more favorable to Republicans by 4%, 8.5%, and 4% relative to other polls done around the same time. This is an average bias of 5.5 ± 1.5% (SEM).

As everyone knows, I am no fan of unskewing. It allows personal bias to creep in. Also, there is uncontrolled uncertainty that goes with not knowing if a pollster has changed methods since the last poll. But *if* we were to unskew these Fox polls, we would get the following:

AK-Sen: Sullivan +4% becomes Begich +1.5%

CO-Sen: Gardner +6% becomes Gardner +0.5%

AR-Sen: Cotton +7% becomes Cotton +1.5%

KS-Sen: Roberts +5% becomes Orman +0.5%

KS-Gov: Brownback +6% becomes Brownback +0.5%

KY-Sen: McConnell +4% becomes Grimes +1.5%

Most of these are not a huge surprise: Arkansas, Colorado, and Kentucky are all very close races. Alaska looks a bit close, but at this point a number of other polls show Sullivan in the lead. Overall, the close match with other pollsters (once the bias is subtracted) means that Fox’s polls are done according to high standards…just with a rather different turnout model. Anyway, Fox’s results go right into the PEC algorithm, with no unskewing!

The real news here is both the Kansas governor and Senate races have tightened up considerably. One thing that’s changed recently is that Orman has been losing support among independents. He is likely to still be in the lead…but that could change in the coming weeks. As per usual, we have to wait for more polls.

]]>*Update: Wednesday night on MSNBC’s The Last Word with Lawrence O’Donnell. The topic: Greg Orman of Olathe, Kansas. It’s archived here].
*