The Second Phase of Realignment: 1976-2012
Shifts in American political geography (“realignments”), which I wrote about on Thursday, can be viewed at a glance using the following diagr...
The incredibly stable 2016 campaign
Like you, I am waiting for polls to come in. A reminder: the following measures will tend to move together: the Presidential Meta-Margin, the Senate Meta-Margin, the House generic Congressional ballot, and President Obama’s net approval. In the last day, the House and Obama numbers have moved toward Democrats.
This year’s Presidential campaign has been full of drama (much of which is captured in a single current story, that of Donald Trump and Alicia Machado). Despite all the venom and extremeness, actual voter sentiment is more stable than it’s ever been. Compare the history of the state poll snapshot with past elections:
This is calculated using state polls only, with no corrections from national polls or otherwise. The full range swing in electoral votes, calculated from July onward, after the nominations were settled, has gotten smaller and smaller. You think the last month was dramatic, but it wasn’t because of changes in public opinion.
Another way to look at this is the Meta-Margin, which is defined as how far voter opinion is from an electoral toss-up. It is in the same units as the national Clinton-Trump margin, i.e. percentage points.
This is not some peculiarity of recent elections. National opinion used to be much more variable, and got more stable starting in 1996:
Measured in terms of polling margins, this year’s campaign is the most stable of any race in the era of modern polling, going back 65 years. I estimate that 2016 has been slightly more stable than 2012, though not by much. Really, those two elections are basically the record-setters when it comes to stability.
It does not seem to be a coincidence that just as campaign rhetoric has left civility far behind, opinion has become more stable than ever. For example, lots of people know how they feel about white nationalism. Their preference is pretty well set at this point.
The question is: Why is it so stable?
Probably hyper-partisinship and polarization. People have been more and more willing to never want to vote for the other side under any circumstance. That’s just me guessing though, as I have no data to back this up.
Polarization. The GOP lined up behind Trump as if he was not a lunatic (“He’s a lunatic, but he’s out lunatic”), and the Dems lined up behind HRC even though half of them voted against her in the primaries. I believe the culprit is FOX (first), the GOP (second), MSNBC (third; late to the party), social media (fourth)…everybody just listens to their personal beliefs spinning around in their heads and nothing can shake the narrative once it’s repeatedly every third minute.
1.) ever increasingly rapid dissemination of instant information for voters that is more and more top news stories on all media outlets since the mid to late 1990’s.
2.) ever increasingly diverse ethnicity of the American public and growth by sheer numbers of Americans so that information is made even more readily available by media and word of mouth. Not to mention a proliferation of electronic devices where voters get information instantaneously in the last decade.
@Andy Surely the story is a hardcore minority in the GOP were able to get a lunatic nominated. Polarization does not explain why Clinton got nominated.
Clinton got nominated because moderate Democrats were convinced that Bernie was too pie-in-the-sky to get his agenda thru Congress or to even win the election…which became a self-fulfilling prophecy. The reality is that if a single party doesn’t control both the executive and legislative branches, gridlock and the status quo prevail…which conservatives love. Truth be told, if Hillary wins, the only thing I see her being able to MAYBE accomplish is to seat a SCOTUS judge. And that’s a big maybe.
It’ll likely be at least 2 justices, and the president does a lot more than just sign bills.
I’d point to the Republican Revolution of 1994. I don’t think it is any coincidence that volatility ended after that date. Prior to 1994, there were plenty of moderates in Congress who knew how to work across the aisle. In that environment, it was possible for moderate voters to justify crossing party lines in election cycle. After that, the moderates were an endangered species and have been systematically eliminated. Elections are much more of a zero sum game now and voters have reacted accordingly.
Larger demographic trends have entrenched, but they move slowly.
<<<>>
Clinton got nominated because she got more people who actually looked like America, and specifically the Democratic electorate, to vote for her while Bernie could not.
Internet searches and delivered content being tailored to a user is causing an echo chamber for millions.
Use a generic browser that has not started tracking your likes/dislikes and see how different delivered content looks.
End results, most people think what they see is what everyone else is seeing. It is their world. Trump base supporters see a Trump oriented world and vice versus.
Marvin8:
This is false. Sanders lost the primary vote because he failed to convince minorities (particularly black Americans and Latinos) that he was the better candidate. Sanders won handily among white Democrats, but white Democrats are a minority.
If you want to know why Sanders failed to convince people of color that he was the candidate for them, it’d be best to look up what some people of color have written on the topic.
Marvin,
Given the battle lines, do you not think that Hillary would be motivated to start with a non-partisan SCOTUS candidate that would be acceptable to less rigid GOP Senators like Collins, Murkowski, Corker, Graham, Portman, Ayotte (if she makes it). A pitched battle with zero GOP support will set the pattern for the next 4 years. She might just go back to Garland. This might be smart since she could have an opportunity to nominate 2 or 3 other candidates.
Sam, still love the site. But still always flummoxed when a good round of state polls come in for Clinton, as they have today, and the meta margin and win percentage go down! I guess that’s why I’m not a mathematician.
Sam, I have been an avid follower/worshipper for years. And as soon as I think I have this figured out, I fall back into the rabbit hole on confusion. So this time I couldn’t find the answer and hoping you had the insight. More specifically, with all this “non-volatility”, one question that comes to mind is whether there is a measure how many people are not responding to the polls? And in any event, would/could that play into the analysis?
Minor suggestion, Sam: why not give all graphs in the top figure the same vertical and horizontal ranges?
Relative to what’s there now, this would mean:
(1) either switching the starting time from May to April for the 2012 and 2016 graphs, or switching the 2004 and 2008 graphs’ starting points from April to May.
(2) Switching the vertical-axis range for the 2016 graph from [200,400] to [160,380].
Change (1) is for formality’s sake, but change (2) would probably make the lack of volatility even more evident.
Marvin: If she has the Senate, she has the judges. And she’ll have the Senate.
Phoenix: Even IF she has the Senate, the Dems might need to use the nuclear option to get her SCOTUS judge through. The GOP’s had a conservative court for 45 years and will NOT roll over on this one. Some may even be willing to sacrifice their careers to filibuster her appointee indefinitely.
Sam’s model suggests she won’t have the Senate.
Senate snapshot puts that at 49D/51R and has been stable for quite a while. Maybe She’ll have the senate, maybe not.
She shouldn’t need a majority of the Senate to get pretty much whatever Supreme Court justice she wants. Historically, it’s very rare for the Senate to prevent the nomination of a justice: the current vacancy due to Scalia’s fortunate death is highly unprecedented (you have to go back to pre-Civil War US to find similar-length vacancies in the court).
My bet is that an Obama nominee for the Supreme Court will be confirmed after the election (once the Republicans realize there’s no point in waiting), and Clinton will probably replace some combination of Anthony Kennedy, Ruth Bader Ginsburg, and/or Stephen Breyer.
As for the Senate makeup after this election, the polls are really too close to call. But I do expect that as Trump’s momentary lead wanes, the probability of the Dems taking back the Senate will increase.
I think the opposite: I do not think a Republican Senate will ever again confirm a nominee from a Democratic President, until the party fundamentally changes. The norm is going to be that, at least if your a Democrat, you need a Senate of the same party or you get nothing.
However, the filibuster for SCOTUS nominees will probably disappear the instant the President and Senate are of the same party, whichever party that is.
They have a constitutional duty, and the excuse for not fulfilling that duty was that they would let the next President decide who would fill Scalia’s seat.
They don’t need to roll over. The American people won’t listen to more excuses. The judges will get confirmed. They have no choice.
“Measured in terms of polling margins, this year’s campaign is the most stable of any race in the era of modern polling, going back 65 years.”
It’s fascinating how volatile 538 has been, while this model has been much more stable. Why is that?
I think Nate Silver was peeved at Matt Yglesias implying that his forecast was too unstable. It seems Sam and Nate Silver are saying a similar thing though – the race this year is quite stable. No argument here.
Just musing, but I think I disagree with you. People can engage in civil, reasoned conversation while still ultimately disagreeing. (Put differently: civility and tolerance are not synonymous–a conclusion most of American history would seem to support.)
I can be civil to you while still vehemently disagreeing with your worldview. But nobody in American politics bothers to be civil anymore. I’d argue this is at least in part a function of election incentives–since people have sorted, and continue to sort, themselves largely based on political preferences, the people they elect have no incentive to “talk nice” about the other side.
This, combined with ideological polarization, has created a society where every person “knows how things are” and their politicians know the same; they have no reason to engage civilly with those who see the world differently.
Apparently Americans are also self-segregating. Liberals are moving to cities and coastal areas, conservatives to the rural middle and south. That can tend to reinforce the FOX/MSNBC effect as it’s easier to demonize the other side since more people associate personally with only one side. When I was brought up, one didn’t talk religion or politics as it was impolite and you realized it would start a fight. Now since you’re more likely to be living next door to someone who substantially shares your views, you’re safe to talk about such topics and can often be led into extremes.
And gerrymandering has given a lot of people in Congress the ability to put party ahead of country to an even greater degree…they can’t be “punished” by voters when they refuse to give the other side a political victory.
Actually the stick is probably more instrumental than the carrot in this case: threat of being challenged in primaries (the real election of those gerrymandered districts) keeps them strongly partisan.
I see no reason for civility when responding to arguments that we should treat some people as second-class citizens.
I think this is because politics have become more about social issues than anything else. It’s easy for voters to be undecided about what exactly needs to be done to fix the economy, for example, or to find peace in Syria. These are complex situations with complex answers that no one really agrees on. Social issues are comically simple in comparison. You either support gay marriage, or you don’t, and you’re probably going to vote for the candidate who shares your views.
Speaking for myself, I have trouble justifying a vote to a party that uniformly chooses to ignore the body of scientific information on a number of key issues. Even if I liked a particular congressional candidate, for example, I would be reluctant to vote for them as another GOP seat gives the party itself more power to legislate. Until something changes I find myself a straight-ticket voter, at least on the national level.
Why do you and Nate Silver differ about this?
Not sure. He is probably using the output of his model to calculate SD. If that model has correlated errors, for instance by using national swing to “correct” state swing, then it would tend to amplify fluctuations.
My calculation has no such hidden sources of variation.
Hey Sam, it seems you probably intended this to be a response to the earlier question posed by Kevin under the “Post-debate pundit spin” thread, wherein he posted:
“I don’t know where the methodological difference in calculating standard deviation comes from–is it the use the two candidate margin versus absolute movement? RealClearPolitics average versus 538′s adjusted numbers?
Link to Kevin’s earlier post here: http://election.princeton.edu/2016/09/28/post-debate-pundit-spin-lives-up-to-its-promise/#comment-102975
Thank you, thank you, thank you for this website and all you’ve done here.
BC
@Sam: But this was his method since 2008, so he should have counted the previous races as even more volatile, which he didn’t really (surely not in 2012, and only the GOP convention bounce through him off a bit in 2008). Also, his inclusion of “fundamentals” (also in previous models) should in principle have a stabilizing effect. Of course, we don’t have the full entrails of his models to examine for possible other reasons. But (as noted in my other comment) simple national poll averages also seem to be indicating relatively large volatility (by recent standards, at least). Also, I’m not sure the Wlezien et al. data on past polling is comparable with the poll averages available for recent campaigns. There may be something about how they collected their data that amplifies volatility (or something about polling aggregators that reduces it).
Hmmm. Well, then I am puzzled.
I am not sold on what you are saying about national polls. If you manually pick out the Clinton-Trump margin in the HuffPollster aggregate at 2-week intervals and calculate statistics, the SD is low.
In regard to 2008 and earlier…this data is available online. I encourage you to go analyze the data. I have previously posted Wlezien and Erikson’s data before, which for 2008 gives an SD of 3.4%.
Another approach would be to consider the peak-to-peak range, which is convenient though not exact. See the 2008 RCP chart, which varies between McCain +2.9% and Obama +7.6%, a span of 10.5%. The span for Clinton-Trump 2016 is 9.0% (RCP, July to now).
Finally…I think that fussing over the 2012 vs. 2016 difference misses the big picture, which is the massive change from pre-1996 to 1996-2012.
His polls+ model has very similar variations as the PEC model. The other two (nowcast and polls only) get torqued about by the national toplines.
Average Clinton margin since early July:
538 polls+ 3.7 +/- 1.5%
PEC 3.7 +/- 1.2%
Looking at the 2008 RCP chart, yeah, it moves a lot more than I remembered. Stopping Sept. 30th does shave half a point off the range, but that’s still a 10-point range. With 2016, actually, it strongly depends on where you do the cutoff point. At his best, Trump was at +1.1 straight after his convention. At Clinton’s best—well, it depends how long back you are willing to go. She was +19.6 when Trump announced in July 2015, but even as recently as March and April, she was in the +10–+11 range (max point: +11.4 on Mar 23). The 2008 chart only goes back to Sept 2007, but has nothing outside the 10.5 range even that far back.
(For comparison, with 2012, we have a max point of Obama +7.6 way back in June 2011, a brief Obama +6.1 in February 2012, and Obama +4.6 in August. Romney briefly pulled to +0.7 in October, which was his best result, so the range would be 5.3, 6.8, or 8.3, depending on when you start counting. And, guess what, they have 2004! Only going back to March 2004, ranging from Kerry +2.7 to Bush +6.8, for a range of 9.5)
The upshot (by this one measure)? 2016 seems to have settled, eventually, within the normal for post-1996 campaigns, but was considerably more volatile if you also count early stages. 2012, not 2016, was the exceptionally stable campaign.
As for the pre-1996 / post-1996 divide, the point seems valid (though, with the exercise I just did, I’d like to see how far back from election day do historical data go; that actually matters a lot). But then, when you’re writing a post about 2016 being so very stable (and arguing about why Silver doesn’t show it to be all that stable), I think your readers would be making a comparison with recent campaigns, not with the norm of a generation back.
Electoral-vote.com’s model also shows more volatility for 2016 than for 2012. Their model uses the single most recent poll unless there are multiple polls for a state in the same week. My impression is that, for whatever reason, there is just less polling going on this year than in 2012; it may be that the relative sparseness of state polls is making a most-recent-poll model more unstable.
…It seems to me that in both the 538 and electoral-vote.com cases, the main difference comes down to Sam’s model not showing as large a dip for Clinton’s margin in mid-September, or (maybe) in the weekend between the debates. 538 actually briefly showed Trump ahead a couple of weeks ago; on e-v.com it was a near tie.
Thank you. The graphical comparison is very helpful.
More stable among partisans but more undecideds could make outcome less predictable?
I’m wondering about the possibility that this election could have lower turnout than any Presidential election since people on the Internet started aggregating votes. If this ends up more like a typical midterm election, it could affect the historically excellent performance of these methods in Presidential elections, and we could be in for a big, possibly nasty surprise.
On the face of it, the unprecedented debate viewership would seem to imply a high level of interest in this election. (I realize that alternative explanations do exist, such as debate-as-entertaining-TV…)
The Gallup poll that came out a little while back said that a very high number of people said they had given a lot of thought to the election, but an unusually low number said they were certain to vote. So they’re deeply interested but they don’t like the candidates.
2000 was the opposite, interestingly: in September, a fairly high number said they were certain to vote but a remarkably high percentage said they’d given little thought to the election.
The data from early voting is pointing towards high turnout.
http://www.cbsnews.com/news/early-voting-surge-benefits-hillary-clinton/
Adding the Web & Fox news to an ascendant Rush L seems to be corelated to the timing of flatlining volatility. Given the country has become more evenly divided AND is more quickly informed it seems there are no more ‘surprises’ capable of moving the needle. Echo chambers are bearable when they echo different solutions for the same facts. When the “facts” are wildly & widely divergent…we get Trump.
Two more possible factors:
1) End of the Cold War, no more enemy to keep everyone in line
2) Civil rights legislation. This is a slow burn but it seemed easier to get along when it was just a bunch of white people, through others into the mix and problems begin to arise.
I was about to ask the same. Maybe “volatility” is not range.
“The full range swing” seems clear enough.
Is it a coincidence that this higher degree of stability commenced after the 1994 Newt Gingrich Congressional revolution? Gingrich was the prophet and path-blazer for excessive obduracy and purist ideology which defanged Bill Clinton. Then Bob Dole could play the “good cop”, negotiating from strength. The Defense of Marriage Act, the reckless financial deregulation, the shutdown of government, even the nuclear option of impeachment constitute the Gingrich legacy which the Net microcosm has since ossified into our society’s ubiquity of entrenched positions and incivility.
Now the demonization of opponents and their views has become part and parcel of engaged voters political discourse. And like herbes, it might lie dormant for a short time but breakout egregiously when we are under the stress of election campaigns. Accordingly, this P.E.C. identified trend of election stability is here to stay.
I agree in broad terms. The quality of the discourse in the public arena took a tremendous dive since Gingrich. I think the precipitating event, however, was the election of Bill Clinton 2 years prior. The GOP considered that loss “an affront” and the”demonization and de-legitimization” of an opponent became their MO. It worked in spades in 1994. Continued in with Whitewater, then Lewinsky (sometimes it does not work 🙂 ) and then swift boating, and now Crooked Hilary…
Why was it an affront? Because Clinton got <50% of the popular vote? Does that ever come up?
Because Clinton was a Democrat. But the fact that he got <50% of the popular vote definitely did come up, repeatedly, as a reason that he was somehow illegitimate. That suddenly stopped after 2000.
I agree. After winning five out of six elections, the Republicans saw the White House as their birthright. They resented losing, but especially to a draft-dodging, philandering hillbilly. The fact that he did not win a majority of the popular vote was just an excuse, as of course GW Bush proved eight years later. This was the beginning of conservatives delegitimizing the outcome of elections that they lost.
Actually, I think it goes even further back. Demonization of opponents began, at least in my concious lifetime, with Spiro Agnew and his “effete intellectual snobs”. That period is when the Republican party began purging itself of its moderates. People like Mark Hatfield or Nelson Rockefeller began to be forced to the sidelines if not outright pushed out of positions of responsibility within the party. Perhaps it goes even further back to Goldwater, but I was too young to fully appreciate what happened in the ’64 election. But certainly I have witnessed the “purification” of the Republican party within my voting lifetime. And I worry that the same thing might happen to the Democratic Party. Diversity is a GOOD thing, as it leads to friction, which when properly applied, leads to new ideas.
It’s because of media fragmentation – people self sort now and only listen to other people with whom they agree.
I think the media fragmentation plays a big role here. I’m a fan of Prior’s work in this field: http://ijpor.oxfordjournals.org/content/20/3/398.extract
I wonder to what extent the removal of the Fairness Doctrine has contributed. I’m sure it’s not the only factor, but I bet it contributes.
The fairness doctrine’s concept still has a deleterious effect because networks that aren’t blatantly partisan like Fox go out of their way to create false equivalencies so as not to be accused of “bias.” “Fair and Balanced” means nothing if it gives way to “Accurate and Informative.”
The NYT’s Upshot likes to help readers visualize its current estimate of the probability of Clinton losing by comparing it to “the probability that an N.F.L. kicker misses [an n]-yard field goal” (right now, n=46).
Anyone else find this irritating? I think what bothers me is the equating of the collective behavior of millions of voters—casting millions of votes, in hundreds of thousands of locations—with the performance, under pressure, of a single player in a single event.
In this context, does a Black Swan Event constitute a bad snap?
But the election -is- a single event: same as a field goal kick, or at-bat, or shot from the free-throw line. Most sports fans still don’t have an instinctive appreciation for how chance influences such an event, but the average football fan knows that a 46-yard FG attempt is more than a chip-shot, but also that if you’ve got a decent kicker and there’s two ticks on the clock left and your team is down by two, that’s a good position to be in. As such, I think the football analogy gives a great point of comparison. But my bias is that I like analogies to help understand difficult or counterintuitive concepts.
Bad snaps are not black swans because their frequency can be inferred from past data. Black swans are events that are so rare that you don’t know they could potentially happen.
Five stars for Brad’s comment. I dislike the misuse of “black swan.”
I have always had mental hang-up with a Monte Carlo simulation on polls. I understand that is just a high simulation of how odds would be distributed in practice, but I am not sure if odds of how people poll (which is a human intent/planned action) is the same as odds for something, much closer to being random without intent, like coin flipping.
For example if polls show a candidate up +5, the only variables that might give the other candidate a chance are sample errors, and likely voter composition modeling errors but that intent by those people would not likely be very variable, yet these simulation that produce the overall odds in Monet Carlo simulations would treat the intent as a variable.
Roger, that’s a good comment. The assumption of course is that as N gets large, what you call “intent” behaves like any other variable subject to the Central Limit Theorem.
I expect this is not just used in estimating vote share but also macro and micro economics, marketing. It may be a little humiliating to realize that free will at the individual level can in aggregate be modeled like thermodynamics, but yeah, science.
Paul Krugman has said that he got into economics because when he was a kid, he wanted to be a “psychohistorian” like in Asimov’s Foundation Trilogy.
<>
I’ve often wondered why an entrepreneurial journalist doesn’t really do a deep dive on the data modeling techniques used by the Obama/Clinton Campaigns and how they use them to project the electorate. My understanding is they’re fundamentally different than how the public pollsters do it using fundamental weighting techniques. They go into every individual registered voter’s file and use an algorithm based on voting history and demographics to grade them on a scale of 1-100 their likelihood of being a Clinton Voter or Trump voter, and then creating a collective aggregate of the projected electorate. That aggregate is fed new data daily from phone banking, polling, etc. to create their up to date forecast modeling of the projected electorate. This is different from the traditional public pollsters just coming up with educated guesses of what the electorates will look like, translating them to weights and applying them to their poll responses (sometimes with non-optimal results, ie Gallup 2012).
Maybe the campaigns are just super secretive of their proprietary technology (but Plouffe and Messina have talked about it at a high level). Maybe it’s considered just a little too data nerdy for the everyday reader, but given the amount of traffic the poll modelers are driving this election season there has to be some appetite for it. But generally, I think people just still find it off putting that humans can be so predictable.
Perhaps the problem with sports analogies is the assumption of “major league ability” on the part of kickers, batters, free-throw shooters, etc. I think what has Democrats nervous is how the assumptions about a typically professional candidate seem not have applied to Trump.
I find it fascinating to read this discussion, especially in light of the Foundation Trilogy +++ written by Isaac Asimov many many years ago. Actually, it appears that he was prescient!! The basic argument is that while at the individual level, human behavior is indeed unpredictable, when you aggregate, and the numbers get large enough, at least some predictability emerges. Computing power has now become sufficiently large and inexpensive to make it possible do the sort of analyses that InmanRoshi refers to. As I recall, a very similar argument is made in Vol. 3 of the original Foundation series!!
Still it is worth keeping in mind that if one organizes oneself into a sufficiently large aggregate, then people can still make a difference. So there is no reason to give in to despair that everything is becoming predictable; quite the opposite, get organized, dispel complacency, and work to effect change!!
I believe there’s a high level political data modeling outfit right in Sam’s backyard. Fifty One Percent, which is co partnered by astrophysicist from Princeton, who helped get another physcist from Princeton, Rush Holt, elected into Congress.
https://www.campaignsandelections.com/campaign-insider/obama-style-modeling-down-the-ballot
http://www.fiftyone-p.com/
Inman, I assume both campaigns are running multiple models of varying complexities. And looking skeptically at all of them. But, as we all know, more complex does not always mean more accurate results.
(In fact, quite the opposite sometimes)
It’s difficult to say that Obama/Clinton models are somehow “better” since the aggregation that Sam, Drew Linzer et al performed in 2012 hit the nail pretty accurately on the head. And in the end, election results are the only observable we have.
PS: As to InmanRoshi’s comment “people just still find it off putting that humans can be so predictable”, I would say there is a difference between the behavior of a single entity and a large collective. I could not begin to predict the behavior of a single air molecule, but in aggregate I can define pressure, volume, temperature and 19th century thermodynamics works pretty well.
I find it fascinating.
I am annoyed by it because it assumes we are all football fans, and even if we were, that this figure would mean anything to us. It also seems kind of gender specific, like having a continuous forecast based on the likelihood that the rubber band holding your ponytail would snap.
Why has the Clinton meta-margin dropped since the debate?
Never mind — as I posted, it went from 1.7 to 2.4. That’ll teach me.
Jim Messina has said he would fire any of his staff if he saw them looking at public pollsters, because they were ‘garbage’. Remember Suffolk pulled out of polling VA and FL in October because Obama couldn’t possibly win there, was derided and mocked by Plouffe for doing so, and then Obama promptly won there. And, there have been many stories of Romney’s problems with their internal polling, mainly overweighting the white electorate. So yeah, I think there is pretty good evidence that Obama’s Campaign and data modeling teams have a better track record than the competition.
And for good reason, it’s hugely expensive to run these kinds of data modeling operations and costs tens of millions of ollars. Most survey operations have been traditionally funded by newspapers who can barely keep the doors open and keep staff counts, much less fund a new survey technique where they pull every single registered voter’s individual file and analyze it.
http://thehill.com/blogs/ballot-box/polls/261189-pollster-pulls-out-of-fla-nc-and-va-says-obama-cant-win
https://newrepublic.com/article/110597/exclusive-the-polls-made-mitt-romney-think-hed-win
In many states the median still includes pre-debate polls, I think.
Sam & PEC readers — It’s been fascinating to watch the disconnect between a data-appreciating and data-wealthy sharp like David Plouffe (who confidently, and quite publicly, predicts inevitable Hillary’s win) and the apparent CW of the chattering class that a “slam dunk” election will cause voters not to show up. I’m not suggesting Hillary herself or top campaign operatives should be dancing in the end zone (mostly because I just think it’d be bad form), but is there any publicly available empirical data that voters behave like this? It’s easy to see why this is the intuition of nervous Nellie Democrats, but my own intuition is that supporters and volunteers on the side of the “doomed” candidate would be even LESS likely (on the margins, anyway) to turn out for their losing effort. There is, after all, a pretty attractive human appeal to being on the bandwagon with the winners. Yet this assumption pretty much goes unchallenged.
Ed Kilgore–a pretty respected Democratic strategist–is pushing this message, in direct contradiction to Plouffe’s confident declarations.
Obviously, nothing will be official until Election Day. But is this a Schrodinger’s Cat situation? Is the cake pretty well baked, and/or have the events been set in motion leading to the inevitable election of Hillary Clinton on Nov 8? In 2012 the Obama campaign commissioned top researchers in human behavior and sociology to conduct experiments. I’d venture to guess that Plouffe knows more about what that research says than Kilgore et al. Any thoughts from you sharps out there? Thanks.
People love to show up to vote for a winner and aren’t inspired to show up for a loser. That is why the Denver Broncos sell out every game and the Tampa Bay Buccaneers don’t.
I think a big difference between 538 and this site, which uses Pollster’s data, is Pollster is more selective in the survey’s they publish while 538 takes every poll available – but then they rate them in their overly complex model.
My hypothesis is that there are more fly by night operations this year than years passed that are giving outlier results. That makes his model less stable than a model built around high quality surveys from groups who publish their methodology.
I am a physicist. Michael Hahn’s comment about unpredictable individual behavior vs more predictable ensemble behavior is the fascinating fundamental idea behind statistical mechanics. Modern economics and quantitative political science are coming closer and closer to physics. Not so surprising that actual scientists are doing well in these pursuits.
Not quite Hari Seldon, but getting there? 😉
Isn’t a reason for stability this cycle the fact that both candidates are well known and were well known for months before their respective nomination s?
Scott,
While HRC and Trump certainly both had extremely high name recognition from the beginning of the campaign, I think that the stability results not just from them being well known, but from both of them having unusually high unfavorable ratings. I’m guessing that a sizable majority of the electorate has been basically unmovable since the nominees were decided because they dislike Trump or Clinton so strongly. The stability means that fewer people are being persuaded to change their minds and I think that if both candidates were popular, more people would be willing to consider switching, not less.
These phenomena are part of a pattern. Candidate net favorability has been on the decline for the last 20 years. Also, note that Trump is especially unpopular. HRC is “normally unpopular” given the overall trend. And yes, it does seem that dislike of a candidate would stabilize opinion.
I note that the Bayesian and random drift statistics have begun to move towards one another. Is this a function of the passage of time, or of the stability of the race, or both?
Sam,
How do things such as hacks into a candidate’s email and the subsequent publishing of that information influence the stability of an election and influence your meta-analysis? I’m wondering if these hacks cause wild swings toward or away from a candidate, even with your meta-analysis getting rid of noise. And, does the widespread publishing of such damaging leaks stretch so far away from the equilibrium that there’s a point of no return to the regression to the mean? Approximately 4 weeks before an election, do these types of hacks and leakage even affect the numbers and the overall stability of a given election cycle? I’m particularly thinking about Julian Assange’s recent statement that he had been planning to release more of Hillary’s emails he had hacked, but has put that on hold because of threats to his life. But, that’s only for now. I would expect that with or without threats, he’ll release those emails within a short time before the election. And, we know what havoc Assange’s publication of his hacks wreaked within the DNC organization this past summer.
Thanks. And, thanks for the great articles.
Ruth,
Any release by Assange is probably going to have very little effect on the people who have decided that Trump is unacceptable and it will have no effect on those who already believe all the baseless accusations, false equivalences, and double standards that have been used to smear Hillary. The question is how this will play with the voters who are not committed one way or the other.
Will they see Assange as an anti-American opportunist peddling what will probably amount to either gossip or manufactured evidence—possibly with the assistance of Russian intelligence—or will they believe the type of narrative that will be put out by FOX News? I don’t think that Assange’s disclosures “wreaked [havoc] within the DNC organization”, nor do I think they would be remarkable—even the content which was forged—if they could be compared to what some people where saying about Trump in emails to the RNC. I expect the release, if it comes at all, to have a less pronounced effect than the previous one as Assange will have lost credibility with some of the voters he is trying to reach and some of them will have made up their minds or even voted already. On the other hand, it will probably be played up by the news media to strengthen the horserace narrative and thus will take focus from Trump being Trump.
I would also be very interested in Sam’s opinion on this also, as, if this is the result of Russian meddling in our elections, whatever its effect it will have serious implications for our republic. This would be fascinating if it weren’t so potentially terrifying.
Slartibartfast:
Thank you for your insightful response. I agree with your statements about the baseless accusations, double standards, and false equivalences applied to Hillary’s
campaign. I don’t understand this idea of Hillary being viewed as the lesser of 2 evils when, in my opinion, she’s one of the most qualified presidential candidates we’ve had in decades (From my perspective, there’s only one evil– Trump). What’s probably driving this is the fear that a competent and intelligent woman can do an awesome job of leading a country, plus the sexist idea of the woman as the happy homemaker. Interestingly, for a supposedly advanced country, the US is very backwards socially.
I had had the same thoughts as you brought up (i.e. effect of hacks and leaks on voters, both decided and undecided; peoples’ ability to distinguish between opportunists
trying to confuse people and muddy the waters; media hype aimed at boosting ratings;consideration of likelihood that the content of the emails was forged).
These thoughts are what prompted me to pose my questions to Sam. Unfortunately, people are quick to believe things without educating themselves or vetting the statements and
considering the source. This results in people becoming reactionary, instead of using logic. For sure, I agree with you that if Assange follows through and announces his hacks, the media will play it up, especially Fox News and CNN. This, in turn, will distract people from Trump being Trump– hopefully only temporarily. In any given campaign, though, not just the 2016 one, it would seem to me that if people become reactionary vs considering the source of the hacks and realizing that they’re meant to discredit rather than being factual, this
could affect the polls which would then have an influence on the stability of a campaign. While Sam indicated in his meta-analysis article that noise is accounted for and eliminated to a degree in the analysis, I didn’t get the impression that the “October surprise” type of noise is part of this accounted-for noise. Or am I mistaken? If this sort of leak revelation event isn’t accounted for as noise, it would seem to me that it could influence the stability of a campaign, as well as affect the analyses. Also, while many people have already made up their minds and some have begun to vote early, given that early voters can later change their vote, I also wonder how many of this group who has
been decided could/would be influenced by a leak’s propaganda– possibly another hit to the stability of a given campaign? So,it will be interesting to see the effects of Assange’s alleged revelation on both the campaign’s stability and returning to equilibrium. And, your point about the current leaks possibly being linked to Russian meddling are especially timely given today’s announcement that the US has broken off diplomatic relations with Russia on Syria (fortunately not on other fronts, but it has
overall further strained relations between the two countries). Definitely, if there’s Russian meddling it will have serious implications for our country.
Ruth, Clinton Derangement Syndrome has been a project of the Republicans since before the Big Dog was caught doing what the Speakers of the House who pursued him were doing (or worse). Mrs. Clinton is an exceptional candidate who is a real policy wonk. She doesn’t have her husband’s political touch, but she’ll make a great President.
The media, now including online websites whose survival depends on views, clicks and traffic, are desperate to keep this election close so that they can stay in business. Couple that with a “change” election, and it stays close when Trump should have been deemed unfit over a year ago.
The Orange Menace is the media and the GOP’s baby. And they continue to nurture him or the hatred that allowed him to become the standard bearer of the party of Lincoln. GOP leaders should be ashamed. But sham,e is very low on the list of emotions in Washington, D.C.
Scott,
I agree with you. Unfortunately, the Republicans have done a good job of discrediting one of the most qualified candidates in many decades. I believe that at least a part of this is that the Republicans and right wingers are afraid of a woman who is much more competent and intelligent than all of them put together. If not for the Republicans’ constant penchant for digging up dirt on Hillary just to make sure their candidate wins the election, Trump’s election bid should have been put away a long time ago even before he became the party’s nominee. But, as you said, the Republicans and mass media need to keep a story going and keep the election close to stay in business. It’s pretty disgusting, in my opinion. Unfortunately, people are very naive and easily swayed. And, I realize that anything that looks like National Inquirer material will bring lots of attention and may sway peoples’ opinions. Assange and people like him, as far as I’m concerned, are footnotes and not very credible. But, there are people who believe these hacker trolls and don’t bother to delve, taking things at face value and believing this garbage. So, I’m curious about how their gullibility/naivete affects the metrics that are used at this and other sites that use statistical analysis. Does this type of hype really matter or influence the metrics? If so, how? Can it cause the equilibrium to shift so far, especially this late in the election, that it wouldn’t shift back (i.e. regression to the mean) before Nov 8th? Or, this late in the election when many people have already made up their minds and may not be so easily swayed, would an “October surprise” revelation be no more than a slight rumble that doesn’t even register on the meta-analysis meter as anything more than just the usual noise that it addresses? I’m mainly curious about the effects of this stuff on the overall stability of a campaign and effects, if any, on the meta-analysis.
Ruth,
You’re welcome. I’d say that the gullible people are a part of the electorate, so they’re baked in to the polling. Unless we somehow have reason to suspect that they have been sampled in a biased manner, then any movement in their position should be reflected in the meta-margin.
My question is how many of them are going to move? I think the answer is very few. Gullible people, at least in this race, are unlikely to remain undecided, in my opinion. To do so they would have to be exposed to (and hence believing) an equal amount of anti-Trump and anti-Clinton information, something that doesn’t seem likely to me. Much more likely that they were exposed to more of one or the other initially and made up their minds early. I would expect them to be amongst the core supporters and predominately supporting Trump.
The question is what the low-information voters who didn’t really “tune in” until recently are going to think. They probably weren’t paying much attention to the earlier leak or other events that moved the meta-margin and won’t look into the minutia or buy the hard line on either side. Which, of course, makes it difficult to determine how many of those people are out there and what they’ll do.
Ultimately, we’ll see it play out in the meta-margin as the debate bounce peaks and starts to revert to mean and the Assange bounce (however big it is) goes through its life cycle. If it is announced this morning as planned, will it be big enough to overcome the 2.8% meta-margin Clinton now enjoys (plus any of the bounce yet to come) and slow enough to persist to election day? That seems like a lot given the stability of the race.
Assange seems to have pulled a Trump and released a bunch of self-congratulatory puffery in place of any actual revelation. Anti-Clinton people are complaining that they got played. I doubt he’s got anything of sufficient significance to take the spotlight off of Trump’s continuing meltdown.
Matt,
I noticed the same about Assange. Now, he’s saying he’ll release the documents either late this week or by the end of the year, that Wikileaks is being reorganized, and that it is now accepting membership. All of this implies to me that the organization doesn’t have sufficient funds to pay Assange’s moles. I also agree with you that he probably doesn’t have anything of much significance that is any different from what the FBI and State Dept already have in their possession. He’s a Trump clone– a pathetic narcissist trying to draw world attention to himself. Unfortunately, there are alot of suckers who fall for his BS hook, line, and sinker. Maybe they’re the uninformed voters that are mentioned in Slartibartfast’s posts. During one of Bill Clinton’s campaigns (I think it was his 2nd one), there was a similar type of hack into some documents, the difference being that the Internet wasn’t quite as pervasive in the mid-90s as it is now. But, Clinton played out the clock and it turned out they had nothing on him. I believe that Hillary will do the same by keeping Trump’s meltdown and his other shady dealings in the forefront of the news right up until Nov 8th. Hopefully, people’s focus will remain on Trump’s shady dealings and his meltdown, rather than on some pathetic failure who is locked inside of an embassy because he’s too much of a coward to come out and face the consequences of his several transgressions. A hack into WikiLeaks that exposed Assange for what he really is, in my opinion, would be ironic.
Slartibartfast,
I agree that the gullible people are probably baked into the polling and that the meta-analysis probably reflects this movement. So, if this is the case, hacks and leaks to the public probably wouldn’t significantly affect the stability of a campaign. I suppose that if there was a significant amount of movement in one polling phase, it may be reflected in the meta-margin such that there’s a temporary shift away from the mean until voters tire of the news and stop paying attention to it. Then there’s a return to the mean.
Re: your comments about gullible people and movement: Given how polarized this country currently is, I agree with you that voters on both sides long ago decided which of the two candidates they’re going to vote for. It’s mainly the impact of the undecided and uninformed voters that I wonder about and their effect on the stability of the campaign. In this particular campaign, there seems to be a higher number of undecided voters than in previous campaigns. But, does this significantly shift the meta-margin and have any significant impact on the stability of this year’s campaign given the number of undecideds? But then again, maybe the meta-analysis has also already accounted for this group? And, given that the undecideds will likely have decided which candidate they’ll vote for or will just decide not to vote, I suppose that as we close in on Nov 8th, there will be a return to the mean. I suppose, too, that the meta-analysis also accounts for voters who indicated that they won’t be voting.
Matt,
Apparently he’s got some stuff and it’s YYUUUGGEEE, but it’s got to be released in bits (and closer to the election). Which makes it pretty clear that he doesn’t think that the material can swing the election on it’s merits (or he would have dumped it now and in full) so he’s decided to maximize it’s impact.
I can’t imagine how Mr. Assange could appear less ethical than he does right now.
For the first time in 8 yrs, I looked at the “scrubbed csv”. I know there is an obvious answer that I am at a loss to come up with – duplicate rows of the exact same poll, population, and date with different results. NC – Sep has NYT, Fox, and Quinn going back in time two rows each with same data except results. The duplicate rows of PPP look like the second is the result with the undecided broken out. Are the other polls representing with and without Johnson and Stein but no one in NC is voting for either? Ditto FL Sorry for such an elementary question!
If a pollster releases multiple versions of the Presidential question (e.g. two-way and four-way including Johnson and Stein) HuffPollster enters them all into its database. PEC takes whichever one was entered first on the grounds that it is the pollster’s “top-line” report, reflecting a judgment of which numbers are to be taken more seriously.
Sam et al.: I note that Pollster is now doing their own predictions and using something that sounds similar to Sam’s approach, with their projections being a tad less “optimistic” (for us Dems) than Sam’s. Does anybody understand what they are doing the same or different? Seems maybe they don’t ever kick a poll out, i.e., just keep adding to the data? But if that is so – are they differentially weighting newer polls vs. polls from much earlier in the cycle? If there is really any movement, seems to me you’d have to overweight most recent polls over older polls. If you don’t, then all you are doing is collecting more datapoints on an unmoving target? Thoughts from the stats and data geeks?
Hi Sam,
Could you share the data used to create the chart “The Disappearance of Presidential Campaign Volatility” please? I’d like to reproduce the chart in a report and due to formatting I cannot just copy and paste the image.
And needless to say, the source will be appropriately cited.
Thanks!
All of the important poll results currently visible (Tuesday afternoon, October 4) on the left side of this page are bad for Clinton. They show her behind in Virginia, Pennsylvania, Ohio, and North Carolina. How did the meta margin go up today?
AAF,
If a poll is better for Clinton than the one it replaces then it can move the meta-margin up even if it has Trump winning the state.
A couple things: note the dates on those polls, which include some of Clinton’s worse polling periods. Second, those were from one pollster. A pollster, I might add, that had Trump up 2 in a national poll in the field from 9/26-10/2, a time where no other national poll showed Clinton with a lead less than 3. Hence those polls might be on Clinton’s low end in the aggregate. There are other polls contributing.