Reality check: Obama net approval/disapproval
Coming into the home stretch, President Obama’s net approval/disapproval rating is at minus 8%. Not good…but 4% better than June. This is wha...
Monday morning – Senate steady, House on the move
Good morning! From a polling standpoint, there was little genuine news over the weekend. However, other sites are moving toward the Princeton Election Consortium estimate. There are several likely reasons.
First, I spy several new Senate polls. None change the picture. We have two new surveys in North Carolina, one showing Sen. Kay Hagan (D) up by 10 percentage points. That’s an outlier. But the fact that such an outlier is even possible illustrates the fact that she is ahead (Hagan +3.0%). In New Hampshire, CNN just released a New Hampshire poll showing Sen. Jeanne Shaheen (D) tied with Scott Brown (R). This is also an outlier, since the race is currently at Shaheen +6.0%. Net effect: virtually no change to either race or the overall snapshot: a Meta-Margin of D+1.0%, and a most probable outcome of 50 D+I, 50 R. [update: new poll in Alaska shows Begich +5.0%. That’s significant.]
Second, as the election approaches, other sites are decreasing the bias that they add by using fundamentals. This will inevitably make them approach the PEC snapshot, day by day. If everything converges on the PEC Election Day prediction, I would score that as an argument in favor of using polls only – or at least letting readers see the difference added by the use of fundamentals.
Finally, the House. In the last four weeks, it’s become obvious that the generic Congressional poll has slid by about 3 percentage points. With a conversion of 3 seats per percentage point, that translates to GOP gains of about 3*3=9 seats. That could change…but it is looking like the GOP will get a popular vote win, their first since 2010.
Topics:
One observation – your allocation assumes that the independents stick with the Democrats; Orman and King have been uncertain on that point.
That’s what I’ve been wondering about. I think Orman said he’d caucus with the majority. Well, what if he can determine the majority?
Logically based on his public statements, that is the one unknown scenario.
Speaking as a matter of probabilities, I feel more comfortable simply calculating “Democrats + Independents” and letting the human factors sort themselves out in 2015. That way, the histogram at right, including the green bar, represents a data-based calculation. There’s enough information there for you to reach your own conclusions.
Orman has said he will caucus with the majority and King merely flirted with it. Bear in mind that since 1980, every independent has caucused with the Democrats and that King comes from a very democratic state. Also, considering the bashing and the drama Orman is getting from the Republicans, even the mistreatment, it’s safe to assume, that both King and Orman will stay with the Democrats. The scared reaction to an Orman win shows how the Republicans feel about it too.
Also, the Dems will gain seats in 2 years, in all likelihood. If Orman goes with the R’s he will be in the minority then, and in a much weaker bargaining position if he wants to switch to the D side.
can you explain the ‘power’ number means?
See the “why care about this” link under the table.
Steve – you can find out about the power # here http://election.princeton.edu/2012/10/14/jerseyvotes/
One question: Have you done any work as to which party will “win” the popular vote for the Senate? By looking at the states with a Senate race, the only state with a large population that is likely to go R is Texas and, perhaps, North Carolina. Thus, the total votes for Democratic and Independent candidates for Senate in this election are likely to be greater than those for Republican candidates.
I don’t really see why that would be important. House districts are roughly equal in size, while states are very much not. The “Senate popular vote” tells us nothing we don’t already know (that the Senate is grossly malapportioned).
…and that the good citizens of the District of Columbia are still denied proper Congressional representation.
I only see that as being relevant state by state for the house. If a state majority vote is for one party, yet another has more representation, then that is an issue. But then again, we already know this will be the case on both sides. Yeah, ‘democracy at work’. OK, it’s a republic, for the money, the the money….
The “Senate popular vote” also depends heavily on which states actually have contested Senate seats up for election, so it’s neither consequential nor a good indicator of national mood.
The issue about what independents would do is very interesting, and I offer this as food for thought.
For part of the time Angus King was governor of Maine, the state Senate was split: 17 Democrats, 17 Republicans and 1 Independent.
The Democrats and Republicans came up with a power sharing arrangement so that the Senate presidency shifted after a year. Democrat Mike Michaud, who is now running for Governor, was the first Senate president. Then Rick Bennett, who is now the chair of the Maine GOP, took the second year.
I mention this because it could affect how Sen. King would look at a 50-50 U.S. Senate. I’m not sure what he would the results turn out that way, but his experience as governor with a split Maine Senate could figure into his thinking.
There was also a power-sharing arrangement between the parties the last time the Senate was divided 50-50, during the first six months of 2001, before Jim Jeffords left the Republican party to caucus with the Democrats.
Given the current political climate, it’s hard to imagine why Democrats would agree to power sharing. We would assume that the GOP’s turn would push a bunch of bills to the President’s desk for veto, while the Dem’s turn would send bills to the House to sit. It would seem to be a stronger strategy to put a deal on the table for indies now, pointing to the 24 seats Republicans are defending in 2016, when a woman candidate might energize exactly the voters who can give Dems a filibuster proof majority. Orman and King would be wise to take any power offered now, as a republican majority post 2016 seems unlikely.
If Begich wins, does that mean 51 D seats?
As of today, yes. Braley would be #50. However, that can change.
In case you are interested, I am playing with calculating single-race November win probabilities. (They are not necessary for the overall November projection.) I currently have Democratic win probabilities of:
Alaska: Begich 62%.
Iowa: Braley 61%.
Louisiana: Landrieu 40%.
Arkansas: Pryor 30%.
In this long-term projection, Alaska and Iowa are basically tied for #50 and #51. If Iowa goes to Braley, this would leaven the extended torture of watching the vote in Alaska and the runoff in Louisiana.
This is my first comment on this remarkable website. Sam, I have a question. On September 15 there were batch of polls released that had some great news for Democrats, particularly in a poll you deemed significant – Alaska, which showed Begich with a 5 % lead. I was expecting throughout the day that you would factor in this in with your concurrent polls averaging, but I notice Sullivan still up 2 % in your overall calculations, where he has remained for a number of days. Will you be factoring the new poll in later ? ( Although I notice you currently have Begich with a 62 % chance of winning, so my guess is that it’s figured into that calculation. ) By the way, you’re doing remarkable work. Thanks !
The model here pulls its data from what is on Pollster (on the Huffington Post site). Once Pollster posts a poll’s results in its database, its effect (if any) will show up in the next update here.
The Begich +5% poll (and another earlier Begich +1% poll from the same pollster) were entered on Pollster late last night, after yesterday’s 5:00 update here.
Nate Silver is coming around to your way of thinking:
http://fivethirtyeight.com/datalab/senate-update-democrats-draw-almost-even-is-it-the-money/
–bks
That’s good. However, does movement in that model arise from changes in polls? Or from the fact that fundamentals play a progressively weaker role in the calculation as the election draws near?
I think the real story here is that depending on how you look at it, Senate Democratic candidates are outperforming expectations – or that Republican candidates are underperforming. Pat Roberts in Kansas is a fairly conspicuous example. There are other, less obvious cases, such as Iowa.
Recall that Silver also attempts to correct for pollster house effects. This is fraught with peril since of course pollsters themselves adjust their weights to account for past misses. This is one fundamental that does not subside as we approach November.
A.L. Good point. I haven’t followed what Nate has done for a few years so this point escaped me. The strange part here is that he has written in the past that at least some pollsters do late adjustments so they can claim those final numbers in their public relations and sales. Back to my worn joke? Maybe he adjusts those pollsters adjustments? Maybe we want Sam to be correct because we are just too lazy to follow all of those important intricacies?
Refugee, you may not realize it but you just invented the pollster perturbation theory. In it, you learn how to correct the corrections to the corrections (to the corrections).
Seriously though, I think Silver et al calculate the correction once per election (you need an actual election result to calculate the offsets). However, the pollsters probably change their weights all the time, not only in response to election results, but also (probably) in response to other polls. No one wants to be a consistent outlier.
I think Sam’s conjecture, that it is useless to try to second-guess the pollsters, is basically ok. Everyone is watching, it is in their best interest to put out the most accurate estimate they can. If there is any “house effect” you can bet they try to stamp it out. Silver’s corrections may be double-counting.
Prof. Wang, do you have any conjectures as to why the Republicans are so badly underperforming in the Senate this and the previous cycle? It seems to go beyond just a couple of awful gaffes — the biggest gaffes in the national races here seem to be committed by Democrats and they’re still showing a surprising amount of strength.
I would not call it “awful” underperformance – just a few percentage points could account for the difference.
Alternately…fundamentals-based modeling of the Senate is in its infancy. Basically the NYT and 538 and WaPo are doing an interesting poli-sci experiment in real time. They differ, so they can’t all be right.
Having followed the polling data as aggregated on this site the Republicans have not underperformed MY expectations one bit. 😉
I would change that statement a little bit — have the fundamentals fundamentally changed a bit in the past 5 cycles so that they are slightly less favorable to Republicans as the last time the Republicans, at the Senate level, performed to or exceeded their fundamental based expectation was 2004 IIRC.
Here’s the WP take on this: http://www.washingtonpost.com/blogs/the-fix/wp/2014/09/16/democrats-now-have-a-51-percent-chance-of-holding-the-senate/?hpid=z1
“Worth noting: When the model showed Republicans as overwhelming favorites, our model builders — led by George Washington University’s John Sides — warned that the model could and would change as more actual polling — as opposed to historical projections — played a larger and larger role in the calculations. And, in Republicans’ defense, no one I talked to ever thought they had an 80 percent chance of winning the majority.”
Then what goes was the model to begin with? It predicted 80% chance of winning with faulty or no data and now they backpedal to say no one should have believed those numbers to begin with?
538 drops Republicans down to 53.3% as of today — down from 54.7% yesterday and 64% from a couple weeks ago.
Perhaps its just me, but I think we can probably do away with the 538 updates. We all know that their “secret sauce” will diminish and it will look more and more like PEC everyday. My guess is that it will be a less confident version of PEC on election day, just like it was last cycle.
Sorry. I just found it interesting to see the movement — especially because Nate called out Sam and Sam’s methodology.
But I can refrain from posting 538 stuff in the future.
I was a little harsh there, I apologize. It is interesting. I just worry that the comment thread could turn into a Sam v Nate thing, and this site is so much more than that.
I also think it is time to dial that down. Obviously I played a role in setting the tone, but now that it’s in the mainstream press, we should play nicer.
I will confess that I find it useful to get your reactions, not just to that site but other items in the news. Also, I actually have been avoiding reading FiveThirtyEight columns. There is too much other analysis to do here!
Wait, is he going to the trouble of harshing on PEC? That’s a step forward!
I think trying to correct for pollster bias is a futile effort, but that doesn’t mean a blind median of all polls is best either.
It strikes me that about 15% of public polls are from almost completely unknown pollsters, or obscure ones with histories of very bad predictions.
There 15% of polls strongly favor Republican candidates.
How about: (1) figuring out the 10 least accurate pollsters of each cycle (2) excluding the results of any pollster that appears on the list more than once (3) seeing if doing so would, in retrospect, be better than a simple median in prior cycles.
For the most part, the GOP has a large number of biased garbage polls (Liz Cheney just started a new company that releases GOP outlier polls).
On the other hand, it seems that these garbage polls might end up correcting a much smaller but more widespread Dem bias in mainstream polls.
Sam featured in The Daily Beast:
http://www.thedailybeast.com/articles/2014/09/16/meet-the-one-numbers-cruncher-who-foresees-democrats-holding-the-senate.html
–bks
New PPP Poll puts Orman ahead of Roberts 41% to 34%.
http://www.publicpolicypolling.com/main/2014/09/orman-davis-lead-in-kansas-races.html
I want to thank Froggy for addressing my query yesterday regarding the Alaska PEC estimate. But if PEC pools its numbers from Huffpollster, I don’t understand why Sullivan’s 2% lead still remains unchanged from before the new polls were released, as it is now a day later, and the number still remains unchanged. In other match-ups I do indeed see a correlation between Huffpollster and PEC – except in regards to Alasaka. Huffpollster gives Begich a 1.9 % lead. But because Sam gives Begich a 62 % chance of winning, I suspect he concurs with this. That’s why I don’t understand why those numbers haven’t moved. Sorry for harping on this ! I’m a huge fan of PEC, but I don’t understand the Alaska discrepancy. I guess I just like seeing the blue !
We use HuffPost data, but not algorithms. The recent median, counting one poll per pollster, is Sullivan +2%.
My November win probability comes from August-September data.
All the birds come home to roost. Messrs. Silver, Cohn, Linzer, et al are on the move. And recall it was Sam who, of the aggregators, initially pointed to Kansas as a possible determinant of the outcome. I well remember my reaction upon reading that piece: Kansas? I confess to not having closed the book on the efficacy of the use of fundamentals in aggregate polling but this latest movement is interesting at the very least.
To the question of which party Senator Orman would caucus with, I personally see a higher likelihood of a Repub result in the event of D-49, R-50. That’s a majority, albeit minimal. And Kansas is a deeply red state.
I was thinking that as well… creating a “majority” that is really a tie broken by Joe Biden is psychologically different from creating a 51-49 majority, even if it would be consequentially the same. The former might well be regarded by Kansas voters as Orman going back on his words.
This is a very good point.
What we don’t know is if there were any ‘understandings’ relating to Taylor dropping out of the race. He may already have a speech written explaining why it is better to caucus with the Dems but saying he can vote either way. Party line votes towards the center won’t be automatic either way on some issues.
Since incumbents typically make reelection their first priority, maybe the likelihood of Orman caucusing with the GOP is higher than expected. On the other hand, he might figure GOP animosity will remain so high that he’s certain to be primaried, especially if he fails to remain in lockstep. He might consider the risk of defeat preferable to being an automaton.
I am a little uncomfortable with statements that someone is “predicting a Democrat/Republican victory”. A distribution that integrates out to a 70% probability for either side is not a whole heck of a lot different than a 50% coin flip. Media talking heads tend to read a lot more into it.
I am a little confused about how Orman enters into the calculation. To see this, let me put forward a Bayesian thought experiment. Let me split the distribution into 5 parts:
(1) Democrats + I (not counting Orman) = 49, Orman wins;
(2) Democrats + I (not counting Orman) < 49, Orman wins; (3) Democrats + I (not counting Orman)= 50, Orman wins; (4) Democrats + I (not counting Orman)>= 50, Orman wins.
(5) Democrats + I (not counting Orman)>= 50, Orman loses.
According to the distribution plot, (1) ~= 40%, (2)+(3) ~=20%, and (4)+(5) ~=40%. Now let’s say we knew instead that Orman was going to lose (pretty much the case before the Democrat dropped out). That means we would be left only with scenarios (3) and (5). I could be wrong here, but I would think that P((3)) >= P((5)). This would mean that if Roberts were a sure thing, the Democrats would be an underdog in taking the Senate, which would be at odds with what was published before.
In any case, I would be curious to see a breakdown for the probabilities for each scenario above.
I don’t think Roberts was ever a sure thing. Taylor was always within five points, even as far back as July.
I can’t believe Nate took to calling you out as the only ‘bad’ model out there. Talk about holding grudges.
Sam, I know you didn’t want a lot of Nate Silver/538 updates, but in his blog post today, he specifically you and your model out:
http://fivethirtyeight.com/features/how-the-fivethirtyeight-senate-forecast-model-works/
“I don’t like to call out other forecasters by name unless I have something positive to say about them — and we think most of the other models out there are pretty great. But one is in so much perceived disagreement with FiveThirtyEight’s that it requires some attention. That’s the model put together by Sam Wang, an associate professor of molecular biology at Princeton.
That model is wrong — not necessarily because it shows Democrats ahead (ours barely shows any Republican advantage), but because it substantially underestimates the uncertainty associated with polling averages and thereby overestimates the win probabilities for candidates with small leads in the polls. This is because instead of estimating the uncertainty empirically — that is, by looking at how accurate polls or polling averages have been in the past — Wang makes several assumptions about how polls behave that don’t check out against the data.
There’s a rich record of those assumptions failing and resulting in highly overconfident forecasts. In 2010, for example, Wang’s model made Sharron Angle the favorite in Nevada against Harry Reid; it estimated she was 2 points ahead in the polls, but with a standard error of just 0.5 points. If we drew a graphic based on Wang’s forecast like the ones we drew above,35 it would have Angle winning the race 99.997 percent of the time, meaning that Reid’s victory was about a 30,000-to-1 long shot. To be clear, the FiveThirtyEight model had Angle favored also, but it provided for much more uncertainty. Reid’s win came as a 5-to-1 underdog in our model instead of a 30,000-to-1 underdog in Wang’s; those are very different forecasts.
There are a number of other examples like this. Wang projected a Republican gain of 51 seats in the House in 2010, but with a margin of error of just plus or minus two seats. His forecast implied that odds against Republicans picking up at least 63 seats (as they actually did) were trillions-and-trillions-to-1 against.36 If you want a “polls only” model that estimates the uncertainty more rigorously, I’d recommend The Huffington Post’s or Drew Linzer’s.”
Should we go through Silver’s presidential predictions and determine whether his stated levels of uncertainty are consistent with his own record of success? My impression is that he over-hedges. Your predictions should be failing at a frequency commensurate with the stated confidence level of your error bars, otherwise you’re exaggerating uncertainty.
Sam Wang did worse in 2010 than in just about any other high-profile election.
I agree, 2010 was not stellar – though I very clearly indicated a GOP takeover. Also, note that others were also off – here’s a roundup of the predictions. Finally, note that I used a different kind of polling data, which was of lower reliability than Senate/Presidential polls.
He’s assuming I didn’t learn anything from past elections. He might also be mixing up our snapshots and our predictions.