Graph of the day

June 29, 2016 by Sam Wang

On most news days this month, there has been some pointless story about a single poll. Journalists’ instincts to report on the exceptional event are totally inappropriate for following polls, where the median result is the one that is most likely to be true. After 12 years of poll aggregation, wouldn’t their profession have adopted better practices by now ? Anyway, Clinton has been up by 5 to 7 percentage points all month. There is nothing else to say about that. Also, we are starting to get state polls, which will fill in the picture considerably.

Meantime, this is of at least equal significance for November:

45 Comments

truedson says:

Ah, as this post was being written TPM has announced their PollTracker is out that sends a message that a new poll has come out!
I really like Josh Marshall’s commentary but his site is so bloated with stuff like that.

bks says:

I despise Josh Marshall’s unctuous commentary and his site. To me he is proof that there is always someone on your side that you wish was on the other side.

Gopalan says:

The y-axis of the approval plot says “Obama net disapproval (%)”. The label on the y-axis suggests that +y-axis is disapproval, and negative y-axis is approval.
If you click on the graph, it shows a description that says that it is a plot of “net job approval minus job disapproval.” Assuming what is actually meant is net job approval (which is equal to job approval minus disapproval), this suggests that + y-axis means net approval, and negative y- axis indicates net disapproval.
The message is confusing. Is the y-axis “net approval” or “net disapproval”? From polls elsewhere, his approval rating is up, so my guess is, the y-axis should be “net approval”.

Sam Wang says:

Correct, thanks, will fix.

truedson says:

And then the Atlantic has a headline today because of one poll! It’s so ridiculous.
‘Voters Are Divided Over Hillary Clinton and Donald Trump’

Bela Lubkin says:

That one is at least *true*, since it’s a tautology. They’re always divided unless one candidate polled at 100%.

Mark F. says:

Agree about the irresponsible use of polling data by the media. Clinton almost certainly would win if the election were held today. Trump can rally, but he’s well behind at the moment. Obama’s approval ratings also bode well for Clinton.

A New Jersey Farmer says:

NYTimes also has the latest Quinnipiac poll on its website, showing Clinton leading by 2. It will be interesting to see if the dynamics change as polling incorporates the effects of the British vote to exit. Even the leaders of Leave are having second thoughts.

Jay says:

I look forward to seeing how newer state level polls affect the win probability.
Does anyone know if HuffPost Pollster will incorporate today’s Ballotopia swing state polls?

Matt McIrvin says:

I wouldn’t think they would affect November win probabilities that much, since the top-line probabilities are still dominated by the simple fact that it’s early. (And the “if the election were held today” probability is just 99+% either way.)

Steven says:

Also a contender for myopic, single-poll story-of-the-day: http://nyti.ms/295qkWu
Couldn’t a place like the NYTimes have a standard editorial rule that a) single-poll stories should be avoided and b) if it must be done then it must cite as background an aggregated average of polls? (R.C.P. average = Clinton +6.2 today) Not to mention how great it would be if they cited PEC’s meta-analysis.
The NYTimes headline (Hillary Clinton and Donald Trump Are Deadlocked, Poll Shows) is irresponsible in light of those other numbers.

Froggy says:

Actually the headline is irresponsible just in light of the single poll they are reporting. A poll with Clinton ahead by 2% shows a close race, but likely a real lead of one candidate over the other. It’s not a deadlock, a virtual tie, or a (gack) statistical dead heat.

Craigo says:

Right. A 2% lead with a 2.4 MoE implies a nearly 95% chance that Clinton is actually leading by any amount.
(The “statistical dead heat” for any topline result other than an actual tie is my nemesis.)

Matt McIrvin says:

Especially if you average together 20 polls that show a consistent lead, compare it to the MoE of any one of those polls, and still call it a “statistical dead heat”.
(With large poll aggregates, the major bugaboo to look out for is a consistent systematic bias, which has absolutely nothing to do with the reported margins of error.)

Matt McIrvin says:

That’s nothing: Rasmussen just put out a national poll showing Trump four points ahead! Gonna be a lot of Sturm und Drang about that one. Rasmussen has a long history of this sort of thing, of course, but the best way to deal with it is still neutral methods like median-based averaging.

Matt McIrvin says:

I actually think it’s kind of interesting that the PEC Meta-Margin is a couple of points *lower* than the balance of national polls. That may just be a stale-poll effect, though, since state polls are not very frequent yet.

jtr says:

There’s definitely a trend in the internet age that every small data point needs to be blown up into “the big story” by journalists. We see the same thing with scientific studies, where any paper with a slightly exciting conclusion (chocolate makes you lose weight! drinking red wine is really good for you! red meat is going to give everyone cancer!) gets blown up in the media even though it’s just one data point in a vast body of scientific data.
I also think there might be a slightly more insidious motivation for journalists in this case. Elections are big business for news outlets, and they aren’t going to get many clicks by publishing the same “Clinton continues to have substantial lead in aggregate polling” story for the next four months. News publications want a close race, or at least for the public to perceive it to be a close race, so they naturally jump on any poll that makes things look closer than they are.

anonymous says:

A troublesome fact in all these polls is that about 10% of the polled population is undecided. This means that they have seen all of the Trump shenanigans and are still not willing to rule him out as a choice. This suggests that a November surprise (perhaps large increases in ACA premiums) could push a larger percentage of these towards Trump than Clinton.

Todd S. Horowitz says:

In a normal campaign, that would be a problem. But every unexpected event seems to provide Trump with another opportunity to show how unpresidential he is. Remember when all the pundits were saying that a terrorist attack would shift things in Donny’s favor, as if he were a generic conservative candidate? Two terrorist attacks later, things are not looking good for him.

Joel says:

The undecided numbers are pretty similar to 2012 polling at the same time in the cycle. I think it’s (generally) another reason to refer to Sam’s post on poll reliability.

emmy says:

No, they are higher.

Josh says:

Usually undecideds break about 50-50. Unless you can produce a good argument as to why this time would be different, it’s hard to imagine that 9 out of 10 undecideds would go to one candidate over the other.

Doctor Science says:

I would LOVE to see your commentary on Brexit polling, as reported by yougov or otherwise.
In the UK, phone polling seems to have been disastrously misleading, while online polls were more accurate. Is anything similar happening in US?

A New Jersey Farmer says:

Here’s something from RealClear that makes more sense than the one-poll-wonders.
http://www.realclearpolitics.com/articles/2016/06/29/is_there_a_state-national_polling_discrepancy_131054.html#2

Matt McIrvin says:

It looks as if he’s groping toward something like the Meta-Margin, though Trende’s number is more useful for exploring possible discrepancies between national and state polling, whereas Sam’s is more useful for figuring out what it would take to win or lose. I think his observation about the time lag inherent in state polls is well-taken.

Turgid Jacobian says:

So, are you actually updating the calculations of the EV, etc? Looking at the code, data, etc link on the sidebar the last updates appear to be from mid June (as of last night)

Sam Wang says:

There might be an issue. Stay tuned…
…okay, the poll parser (which does initial intake before handing off to MATLAB) crashed. Fixed now, Thursday 2:00am.

Turgid Jacobian says:

Awesome, thanks!

Gene V Glass says:

What’s mildly troubling about the Brexit case is that polls and prognostications missed the outcome because people may not be willing to express nativist (or worse) opinions outside the voting booth. Might the same thing be operating here?

Sam Wang says:

Other than the fact that UK polls did not really miss, the ballot question there was sui generis, US polls do fine, and any Bradley-like effect is no more than two percentage points, sure.
In other words, no way.

Suvro says:

I don’t see a way to reply to Sam in his reply below.
IF there is a shy-Trump effect:
http://www.nytimes.com/2016/05/11/opinion/campaign-stops/how-many-people-support-trump-but-dont-want-to-admit-it.html
(or reverse Bradley) effect:
http://www.politico.com/blogs/ben-smith/2008/10/a-reverse-bradley-effect-012952
I am wondering if this would not have been evident in the primary polls. In some place Sam has argued that the primary polls were accurate when predicting Trump’s support. But those were the most rabid Trump supporters.
Is there data showing that the actual “shy” Trump supporter does not exist among the general electorate, rather than in the primary electorate?

Joel says:

@Suvro
Not to flog Taleb, but those examples sound like a couple of “Fooled By Randomness” case studies.

Michael Hahn says:

I was wondering when someone was going to get around to noticing the Obama approval ratings again!!! I have been tracking this since the first of the year, and it is a remarkable climb!!! And one that COULD bode well for the Democrats in November. But only time will tell!!

BillSct says:

I mentioned this before. I have been following the Favorable/Unfavorable ratings for the Republican and Democratic parties. Currently Pollster shows:
Republican Party: Favorable 30%, Unfavorable 60%
Democratic Party: Favorable 45%, Unfavorable 47%

Marvin8 says:

I sure wish there was a fairly accurate method of estimating the strength of closeted Trump voters…ie…those that won’t admit they’re voting for him. I have many GOP friends, who, although they don’t like to talk about him, will definitely vote for him in the booth. I have a sneaking suspicion that there are more than enough of them to sway the election.

Sam Wang says:

Dan Hopkins studied the Bradley/Wilder effect. He found that in years when it appeared to exist (pre-1996), it was a median of three percentage points. That could be used as an estimate of any such effect.
Generally, this kind of question is why I have constructed +2%/-2% electoral maps in past years. It is also why I calculate the Meta-Margin.

Matt McIrvin says:

The other side of the coin: I suspect there are a lot of closeted Hillary voters. Many women, particularly.
Even in liberal political circles these days, it’s still just a little bit taboo to say you enthusiastically support Hillary Clinton, especially on the Internet, because the Bernie Sanders dead-enders will dogpile on you and it’ll be a fight. It’s easy to make the calculation that it’s not worth the grief. Yet she won the primary, so the support is out there.

Michael Coppola says:

My completely subjective view is that there are a whole lot of people who fancy themselves independents, but in point of fact consistently vote one way or the other. These folks tend to tell pollsters that they are undecided, especially this far out from an election. Sam can tell you all about the dangers of assuming that you have some special insight into how the undecideds are going to break (hint, they tend to break the way the decideds have already broken).
Anyway, if there is a shy-Trumpster effect, it would have to be absolutely huuuuge to make the current polling “wrong.” And finally, as a Virginian I’ve always preferred to call it the Wilder effect.

538 Refugee says:

Most breakdowns I see have the ‘independents’ breaking for Trump. I’ve gradually started getting the feeling that the Tea Party folks don’t like to be considered Republican. If party identification is self reported in the surveys this could explain why Clinton is ahead of Trump even though ‘independents’ seem to breaking Trump’s way.

Matt McIrvin says:

Some of the polls do re-weighting according to some model of how partisan identification ought to fall–and the responses to these polls are so polarized that that can determine the whole result. In most recent polls I’ve seen, “independents” break very weakly for Trump, Republicans strongly for Trump and Democrats strongly for Clinton (whether more or less strongly depends on the poll, but it’s not hugely more strongly). But more people self-identify as Democrats, which ends up giving Clinton the edge.

Sam Wang says:

We take whatever the pollster provides. Note that third-party candidates usually fade in the final weeks of a campaign.

Michael Coppola says:

Speaking of statistical malpractice, there’s a new Rasmussen poll out today.

bks says:

When there are no outliers, someone has his finger on the scale.

WildIrish says:

Is there something wrong with the graph? Obama disapproval is now over 60%?
Love this website! An oasis of sanity in these turbulent times.

Sam Wang says:

Yeah, there’s something weird in the data feed. We’re working on fixing it.

Leave a Reply

Your email address will not be published. Required fields are marked *