Princeton Election Consortium

Innovations in democracy since 2004

Outcome: Biden 306 EV (D+1.2% from toss-up), Senate 50 D (D+1.0%)
Nov 3 polls: Biden 342 EV (D+5.3%), Senate 50-55 D (D+3.9%), House control D+4.6%
Moneyball states: President AZ NE-2 NV, Senate MT ME AK, Legislatures KS TX NC

PEC switching (as planned) to short-term forecast

September 30th, 2014, 12:30pm by Sam Wang

[Note, October 13: The probability is currently displayed as a decimal, i.e. 0.4 means 40%. The reason for this is that the precision of the probability is no better than +/-10%. This is the case for other aggregators as well – we’re just being explicit about it. – Sam]

As planned for a long time, we’re switching soon to a short-term forecast. As I wrote last month, the Meta-Margin has some predictive value for where it will be within five weeks. The election is in five weeks, so it’s time to start factoring in current conditions.

I’m traveling today, so there will be a slight delay in bringing it live. In the meantime, after reading your comments on yesterday’s thread, I have some comments of my own. [Update: after the break, I have now added a simpler summary.]

[I realize that what I originally wrote is fairly technical. Here’s a simpler version:

To make a prediction, I have assumed that October will be like the June-September general campaign, a period when Democrats led for Senate control most of the time. However, Democrats took a small dive last week, raising the question of the assumption’s validity. So I asked: what if October is like all of September? As it turns out, the math all comes out the same because Democrats have led for most of September as well. So there’s no easy way to make a different long-term prediction based on polls only. Besides, changing the math would make some people mad, so I shouldn’t.

As the election gets closer, current polls will do a better job of predicting the outcome. As time passes before Election Day, I’ll count those more and more heavily. If they show a Republican lead, eventually the prediction will flip.

Finally, I realized thanks to commenter Jay that polls can be a bit off in either direction, even on Election Eve. I am adding some math to accurately reflect that possibility.]

Changing the time window. There was a split of opinion about whether to readjust the time window for setting the long-term forecast. Currently it’s at June-now; the question was whether to switch to September-now. The argument against switching was: it’s cheating. The argument in favor was: if it’s likely to be more accurate, do it.

As much as some would like to make this a nerd-off, I think most people just want to know what is likely to happen in November. Think of weather forecasting. You don’t care what the forecasters think of each other. You just want to keep out of the rain.

At the moment, the two time windows currently give the same result, within 2 percentage points. The reason is that the June window was originally set not only by logic (Memorial Day to start the season), but also to emphasize the fluidity of the race. At the time, this meant finding a time window that favored Republicans as much as possible.

The two approaches won’t change future predictions either, mostly. The September window would get the probability moving toward current conditions a few days sooner, since it averages data over a shorter time period.

Note that in both cases, the short-term forecast will (of course) look progressively more like the current-conditions snapshot. On balance, I lean toward keeping things the same.

Unknown systematic error. Last week, commenter Jay suggested that I might be underestimating the possible discrepancy between Election-Eve polls and actual results. If you look at the discrepancies from past elections, which I listed yesterday, they showed a range of Republicans outperforming polls by 1.4% to Democrats outperforming polls by 2.9%. The distribution of discrepancies is coded in two parameters: sigma_systematic (which is symmetric in either direction) (systematic), and the long-tailed distributions that I favor using (blackswanparameter, currently set at a 1-d.f. t-distribution). My current thought is to increase the parameter systematic in the script from 0.7% to 1.2%.

Display. You didn’t bring it up, but the display is currently rounded off to the nearest 5%, since all forecasts (including ours) aren’t any more precise than that. That hid some small movements, and gave the false appearance that nothing was changing at all. The media market is set by big players like the NYT, and has established higher precision as a reporting standard. Seems like we might start rounding to the nearest 1%.


One final note: today’s forecast would hardly change, even if all of these changes were implemented.

Tags: 2014 Election · Senate

33 Comments so far ↓

  • Alan Koczela

    Dr. Wang,

    Thanks for the info. Don’t worry about the barking dogs. Your explanations seem reasonable, and, though I can see why there should be a larger systematic error component, I suspect this change it will cause the most fuss. I believe most followers of your blog have been anticipating a flip to R. Given the shorter time to the election and what the model predicted for a similar-sized meta-margin in the past, I’m speculating the model shows R with a 65-70% prob. by election day.

  • Natalie Rosen

    Love Sam Wang but truly I do not know what he just said. Sam, can you put it in simpler terms for those of us who are genetically math disabled!

    • atothec

      Waiting in the meadow.
      It furthers one to abide in what endures.
      No blame.

    • Sam Wang

      Oh, sorry! It’s like this.

      To make a prediction, I have assumed that October will be like June-September, when D’s have led for Senate control most of the time. But Dems took a small dive last week. So I asked, what if October is like all of September? The math comes out the same because Dems led for most of September as well. So there’s no way to make a different long-term prediction based on polls only. Besides, changing the math would make some people mad.

      As the election gets closer, current polls will do a better job of predicting the outcome. So I’ll count those more and more heavily. If they show an R lead, eventually the prediction will flip.

      Finally, I realized thanks to Jay that polls can be a bit off, even in the home stretch. So I am adding some math to factor in that possibility.

  • Amitabh Lath

    The meta margin is bouncing around in a what looks to be a non-random-walk manner. So either several thousand people changed their minds for no particular reason across several states, or the polls do not follow random-walk distributions.

    I know you dislike poll filtering, but I was thinking that low statistics polls (say, N<700) might have trouble with weighting as some exclusive bins might have few or no counts. Low stats might also correlate with low quality (a serious firm will get a thousand or more counts, I would think).

    Any idea what happens if one takes out low stats polls? I think IA stays tied, but CO moves D. The rest stay as they are.

    • 538 Refugee

      I’m wondering if it is going to come down to the integrity, or lack thereof, of the pollsters that we need to worry about. Look at Alaska and it appears as if there is some ‘tit for tat’ polling releases. Can you model “garbage in”? Change the likely voter model assumption and the pollster gets the number out they want. If nothing else the post election dissection will be very interesting.

    • Amitabh Lath

      Since the polls are pulled from the Huffington Post site I presume some filtering of obviously bad polls is already done.

      Something made multiple polls across multiple states go significantly Republican at the same time the last week of September. Bad polls are an explanation, but not the only one. There could be other reasons such as an coordinated ad push by PACs, candidate debates in many states, or something else.

      It is important to note here that this phase-transition or whatever it is that happened, is best seen by the meta-margin. It is by far the most sensitive probe of public opinion.

  • JayBoy2k

    I mostly would like to know how much to depend on this model in November of 2016 and 2018, etc. I am interested in a successful model rather that the specific results of one election.
    1) I think I want to learn a couple of things: How to modify the model so this issue does not come up again in future elections. That may take analysis and a while to discover.
    2) What are the visible effects of this change — One sounds like with the old model we would have taken a couple of weeks to reach the current set of polls, now more instantaneous. And the Meta-Margin range will change to reflect the current conditions. Anything else?

    I am OK with changing. I am just curious — What did we learn about RV versus LV polls or anythng else? Do we have to wait final results?

  • Randall

    IA & CO may very well go D and we’ve got polls (albeit back and forth) to say so, which leaves all the subsequent explaining to those who were always so darn sure of an R takeover. Stick to your guns, Sam.

    • RB

      IA had a PPP poll today and Ernst was up. 4/6 of the last polls in IA have had Ernst up(two by 6 points)—The other two were ties. In CO I think Gardner had been up 4 or 5 straight. With out going into great detail it had been the results of flubs by Udall and Braley that have pretty much directly correlated to the movement.

  • Teresa

    It seems to me that when it gets this difficult to make a prediction, it is because maybe you can’t. That is a keen grasp of the obvious, I know, but it also suggests maybe there is no need to turn yourself into a pretzel to predict the haze.

  • Matthew

    Re the systematic error:

    I was under the impression that the appeal of your modeling is that it is a polls only approach. If you add a systematic error number into it aren’t you “placing your thumb on the scale?” You may be right that there is a systematic error and you appear to have good reasons for making the adjustment but by adding this error factor aren’t you getting away from your polls only approach that you have been using since you started?

    • Sam Wang

      No, that is not right. You are missing the point. That’s an uncertainty, not an unskewing. It can be in either direction, but is of unknown size.

  • jd351

    Why cant we have a better press corp…. This article is so misleading…

  • Johnatrisk

    Regarding what you call systematic error…my sense is that it is coming from biases related to likely voter screens. Adjusting for this ex ante (if assignment was random between registered and likely voters) should increase the expected errors of any forecast, but research (notably Ansolabehere and Hersh) indicates that it’s not. I did some crude back of the envelope Bayesian calculations on the parameters in their paper for the last election, and it seems that your “tail” may actually be underestimating the bias rather than overestimating it. At the very least, given ex ante data on registered polls vs adjusted polls and the demographic tendencies to mis-report, you could come up with a (much wider and asymmetric) error bound that might reflect the underlying true turnout vote somewhat better. Much clearer here:

  • Canadian fan

    RB. As a matter of fact, the last six polls in Iowa included three ties, two with an Ernst lead of 6, and one with an Ernst lead of 2 ( the most recent one ). You mention the flubs of candidates. I think it is only fair to note that flubs are not the monopoly of any particular candidate. Gardner’s glaringly consistent legislative record regarding a woman’s right to choose constitutes as a flub. Ernst coming out against a minimum wage for Iowans is also a flub. I would wait until election day to determine which flubs the public deems more serious – for instance, women in Colorado, and in Iowa those who work for a minimum wage. Both of these demographics encompass a great deal of people. This is one of the reasons why there is a consistent gender gap between the two parties, including the most recent polls in Iowa and Colorado. The key to Republicans, therefore, is to get as many men to vote for them as possible. Certainly that strategy can work, and possibly may in some cases. But it can also fail. The remarks and manner of Tillis in North Carolina have turned off many women – even Republican women, for example. If you want to see what major, major flubs do, you need look no further than Terri Lynn Land in Michigan. I’m just saying that there are plenty of flubs all around. Some turn races, others don’t.

  • Canadian fan

    RB. After posting I realized I had stated something inaccurately, so I wanted to correct that. Ernst of course does not oppose the minimum wage ! What I had meant to say is that she has fashioned an argument whereby she doesn’t feel it needs to be raised. I apologize for the inadvertent error. I still think it’s a campaign misstep, though.

  • RB

    Canadian Fan-In IA the 4 th poll was also a PPP poll for the LCV…I don’t look at internals such as the one that Braley released—they are suspect and released with a motive. As for the women-I don’t think women are single issue voters. That is a policy question, but as an FYI Gallup last year found that 63%(appx). Of the country oppose 2nd Trimester abortion(clinically weeks 13-26). The number goes into the. 80s for 3rd trimester abortions. Policy positions are not flubs, they are areas of disagreements. Suing your neighbor over chickens in the yard is a flub as well as putting words in beheaded journalists. Gardner and the rest have taken the contraception issue out by favoring making it over the counter. Hey I have to pay for my baby Asparin over the counter—in fact there are a lot of things Me and my wife buy at CVS that I don’t beg Uncle Sam to pay for. I think that asking folks to pony up for condoms or birth control over the counter is reasonable(you may disagree-but that is a policy debate, not a flub)—BTW CNN had a good release for you guys on the generic yesterday-interesting to see if the other national posters match or if it is an outlier(6 point swing in 2 weeks seems extreme, but if the others point that way then that may indicate real movement)

  • Canadian fan

    RB. Well, I’m glad it’s settled that women aren’t single-issue voters. I wonder if that cancels out equal pay, as well ? In terms of Gardner and over-the-counter birth control, it only settles the issue from a Republican point of view. It does not negate the long-standing legislative record and statements Gardner has made on personhood and related issues. Personhood is very unpopular among Colorado women, by the way. And Gardner knows that. But here’s the problem – Gardner as a member of the Republican House last year co-sponsored a bill called the Life at Conception Act, that he now denies says what it says, even though his Republican colleagues who wrote that law naturally openly contradict him. As proposed constitutional law, it was a minefield, particularly with women. Naturally, the legislation never left the House. That is something, of course, that could instantly change with a Republican Senate. Gardner is now saying that he doesn’t support personhood at the state level. But at the same time, refuses to distance himself from a bill he co-sponsored last year. Not just Democrats are seeing this. This issue of Gardner’s has been splashed from one major newspaper to the next. And that is the very valid point that Udall ads make in Colorado. You call that a policy position. Indeed it is. And the more focus there is on policy positions will benefit Democrats. This is why tea party candidates have an uphill battle with women voters. Joni Ernst – also a tea party candidate – was asked in the debate on Sunday with Braley about a personhood amendment last year that she supported. She brushed it off as being just a ” statement of principle. ” This only underscores that both these candidates are well aware of their vulnerability among women on these issues. And that is why it is relevant for Democratic candidates – and journalists – to bring up these issues during a campaign.

  • RB

    Again these are policy issues and and that is one policy. As I have shown the trend on abortion is going more against, especially in the 2nd and 3rd trimester. I am sure Udall’s vote for the ACA is not helpful or opposition to Keystone. And if he is ‘Standing with Wendy-opposed to bills banning abortion at 20 weeks that is extreme by many (63% )—every candidate had pluses and minuses and while you may disagree with Ernst and Gardner on everything that does not make them extreme for the electorate even though in the circles that you may run in you find them extreme.
    My sense is that the country is not running around terrified on ‘the war on women’ and nod their heads on agreement when Debbie Wasserman gives a ‘back of the hand comment’. From where I sit I hear people concerned about the cost of day care, watching the price of food go up due to the Feds policy of printing money, and ISIS. Again I run in different circles, but it is fair to say that we Agree to Disagree—and I just don’t think people are going to the polls in IA or CO with ‘war on women’ in their mind with so many other pressing issues making an actual impact in their lives.

    • atothec

      RB you must not have read the recent Huffpo piece which indicate that IA is on track for record breaking mail in ballot requests (and remember this is a midterm to boot) and of those requests Dems crush Repibs 52 vs 26%. That is a huge margin. So you combine record votes with 2 to 1 for Dems and you have a very, very competitive race.

      Remember IA is a home base for Obama and his gotv outreach there (as well as the other states) is massive and incredibly sophisticated.

      I do believe that this election the dark horse is Bannock Street Project/ground game. We will see Obama’s advanced campaign tactics rolled out for a midterm for the first time. As 2008 and 2012 have made clear, you understimate this at your peril.

      This is new territory and I think adds to prediction volatility.

    • Insidious Pall

      Atot – While I don’t have the HuffPo piece in front of me, I remember the numbers. There were 145,000 requests at 8,000 per thus far in 2014; two-thirds are Dems. In 2010, there were a total of roughly 350,000 mail-in ballots cast. Even if requests come in at the same rate until the day before the election (not likely), that would make about 425,000 requests. More than 80% of those requests would have to be turned in to surpass 2010. Early voting ostensibly helps Democrats, but I don’t see it having enough impact to offset what is in the poll numbers. Additionally, the GOTV effort in the last two prez elections may have been given a boost by the African American vote; it remains to be seen whether that difference will translate into political realignment.

  • Canadian fan

    RB. Thank you for your comment. There is a distinct possibility you may be right, and that both these candidates will be elected. But I’ll bet my bottom dollar it will not be because of the women’s vote. And that is a troubling situation – though not uncommon in a Republican Senate that is scarce populated by women – if these issues ever come up for a vote. But despite what you say, it is relevant that Gardner explain how he could be suddenly against personhood applying to the state, but co-sponsor a bill in the House of Representatives that would have had federal binding. If it’s a war on woman, I can assure you that he has contributed to that narrative, and not his opponent.

    • RB

      @atothec-you may be right, but that defeats the entire purpose of this web site created on a polls model. That said the DMR is the Platinum Standard in IA—the voters registered and voting would be accounted for in a LV survey like the DMR or Quin. Personally I think it is close, but Ernst has the slight edge.

      @Canadian-Yes as a whole women and minorities are underrepresented by both parties in the Senate. Earnest would be the 5th or 6th women current GOP Senator and Scott is the only AA Republican Senator. I don’t think that the GOP is gunning to get the Senate to legislate on abortion. I think that they will be looking at jobs, energy, fixes to healthcare, and national security. If Obama wants to veto that is his right of course. But this is all hypothetical for now.

      Something to look for today Ras has a CO poll today(but his reputation is terrible) and Suffolk has a KS out today(personally I would have liked to wait a week to let this race settle in, but there it is)—Personally I think that Suffolk will show Orman up 4-7 but as my wife will tell you- I have been often wrong.

  • fred Neuman

    I understand that you still favor the Democrats to win because you are averaging their poll success over a period of months. Isn’t there some way of weighting more recent polls more heavily–perhaps even on a day to day basis?

  • Violet

    Sam, Sam, you broke my heart switching to a Republican projection, but I guess it’s about being accurate, rather than trying to prevent my tears. Thanks for all you do.

    • Violet

      Oh, wait, I guess the projection today is 49/51, but you’re still projecting the Dems to win? I’m clinging on to whatever I can at this point.

  • Sioux Falls Bill

    Thanks for all you do. Love your material. Heard Chuck Todd and guests this morning. They made it sounds like the REPubs were in trouble. All I know is that if 80% of low income individuals don’t vote, as is predicted, it will be terrible. Those top 400 with as much wealth as the bottom 50% of Americans will vote!! You can count on it and you know how they will vote.

  • Elvis Hunkaburning

    Is it Live?

    Sam, on 9/30, you wrote : I’m traveling today, so there will be a slight delay in bringing it live. ” referring to switching from a June-to -Now model to a September-to-Now model.

    Are you now live with the adjustment? Do the current charts reflect that change and if they do, as of what time and date?

    I may have missed the announcement.

  • AH

    Does anyone keep track of what I’ll call “the generic Senate vote” — i.e. the total number of Democratic, Republican and other votes in the 100 elections for the senators sitting at a given time? (I realize that it may be less than 100 if there are appointed senators.)

    I think this would provide a good indication of how the Senate may be skewed toward one party or the other.

Leave a Comment