Protected: Two ways to estimate primary outcomes without polls (transcript)
This content is password protected. To view it please enter your password below: Password:...
The polls are always bouncing to the left and to the right
For those of you not into adolescent pop culture of the 1970s, today’s title is explained at the bottom.
Updated to reflect new polls. The bounce is smaller than the original estimate.
We have five polls that measure a post-convention bounce for Presidential candidate Donald Trump (R):
- NBC/SurveyMonkey: steady at Clinton-Trump 46%-45% (July 11-17 to July 18-24).
- CBS: an increase from Clinton-Trump 40%-40% (July 8-12) to 43%-44% (July 22-24), a 1-point swing from a tie to Trump +1%.
- YouGov: A 1-percentage-point change from 40%-37% (July 15-17) to 40%-38% (July 23-24).
- Morning Consult shows a 6-point swing from 41%-39% (July 14-16) to 40%-44% (July 22-24).
- CNN shows a shift from Clinton-Trump 49%-42% (July 13-16) to 45%-48% (July 22-24), a 10-point swing from Clinton +7% to Trump +3%.
The measure of the post-RNC bounce so far is a median swing of 4 percentage points 1 percentage point. For stragglers, see HuffPollster.
One point is not an impressive change. Recall that in states won by Mitt Romney (R) in 2012, Trump has been lagging by about 9 percentage points. A CBS crosstab (can’t find at the moment – perhaps a reader can help) reports that Trump’s progress was made entirely with Republicans – whose support went up by 2 points. This suggests that with many reluctant Republican voters, Trump did not close the sale. Also, note that some of the change may be changes in how likely people were to respond to the survey. And of course, it remains to be seen whether any increase in support is lasting.
Current numbers do indicate that the race has closed up a bit. As of today, the election could possibly go to Trump. However, the election is not today.
Convention bounces aren’t what they used to be. Shown below are patterns that come from Gallup data, 1984-2012 in the net change in direct support for a candidate.
As you can see, the median change in candidate support in modern times is only 2 or 3 percentage points.
This reflects what I wrote about over the weekend, decreases in net impact, i.e. change in “likelihood of supporting candidate”, which allows favorability to be measured without forcing voters to change their minds:
In the CNN/ORC poll (see Q13), 42% of respondents said they were “more likely to support Trump,” and 44% said “less likely.” That’s a net difference of negative 2 percent, which is worse than any value in the graph above. By that measure, the Republican convention was a failure.
In both graphs, a notable shift occurred around the time that national elections became more polarized, in 2000. We are in an era of government shutdowns, endless Congressional investigative hearings, criminalization of political opposition, and ever-more-contentious judicial nominations. Voter entrenchment appears to be just one more symptom.
In the coming week you may be surprised to see relatively little change in the Princeton Election Consortium electoral-vote tracker and November win probability. There are two reasons: (1) We use state polls, which take time to reflect national shifts. (2) The Bayesian-win probability listed in the banner uses polls over the entire 2016 campaign to set a prior expectation for where things are likely to head. The second assumption also has the more traditional name of “regression to the mean.” Effectively, these two mechanisms prevent the calculations from spinning out of countrol whenever there is a momentary bump in polling. Therefore, today’s November win probability is 80%.
Of course, if the race shifts in a lasting manner, it will show up eventually. Just to state the obvious, now is not the optimal time to gauge where the race is headed in steady state. Recall that in 2008, the Republican convention and the addition of Sarah Palin to the ticket led the race to briefly appear tied.
If you want to see the prediction without the Bayesian prior, the assumption that polls can drift equally in either direction, toward Clinton or toward Trump, is the random drift probability. Today, that probability is 65%.
Which brings us back to the title of the post, which is adapted from a lyric by AC/DC. It’s my belief that one big poll should be held every night:
P.S. A reader, Dmitriy, wants to know if this is a normal swing in the course of past elections. In terms of state polls, we don’t know yet. I would guess that once we have enough data, the answer will be yes.
Here is a view of the last three rollercoasters (click to enlarge):
Topics:
It’s my belief that my big polls should be held every night.
Sounds like a great idea. Trouble is polls ain’t free to run.
If you want the party cross tabs for the two latest CBS surveys (and there’s very little change), look at questions 3 and 4 here:
https://www.scribd.com/document/318275701/CBS-NYT-POLL-Hillary-Clinton-and-Donald-Trump-tied-going-into-conventions
https://www.scribd.com/document/319228383/7-25-16-Toplines-monday-Morning-CBS-News-Poll
Are you pointing out that the independent voters when given the choice of Gary Johnson are more likely to vote for him over Trump? And or are you showing that like Sam Wang states the convention bump is small and that was what was most likely expected?
Sophia, I’m just trying to be a helpful amphibian in response to Sam’s “A CBS crosstab (can’t find at the moment – perhaps a reader can help).”
Not easy being green, is it?
It’s one of the few things that Jill Stein and I agree on.
Thanks, Froggy. I am having trouble finding a crosstab about changes in Republican-voter-only sentiment, which I saw flash by in my news feed…
The last part of the meta-margin and median EV look like a last heart-beat before a flatline (death). I hope it is not an omen.
For ’08, think the Lehman bankruptcy was a marker.
Does the Donald know that AC/DC is claiming to have “the biggest balls of them all”
Do you have any thoughts on CNN’s claim that this is the first convention bump they’ve seen since 2000?
http://www.cnn.com/2016/07/25/politics/donald-trump-hillary-clinton-poll/index.html
“Donald Trump comes out of his convention ahead of Hillary Clinton in the race for the White House, topping her 44% to 39% in a four-way matchup including Gary Johnson (9%) and Jill Stein (3%) and by three points in a two-way head-to-head, 48% to 45%. That latter finding represents a 6-point convention bounce for Trump, which are traditionally measured in two-way matchups.
There hasn’t been a significant post-convention bounce in CNN’s polling since 2000. That year Al Gore and George W. Bush both boosted their numbers by an identical 8 points post-convention before ultimately battling all the way to the Supreme Court.”
The critical words are “in CNN’s polling”. If you only look at two polls, the difference between them is going to be dominated by random statistical noise.
It reminds me of what happens if you try to take the first derivative of noisy data by differencing without thinking about it. (You just get junk.)
…and here’s a three-minute video that illustrates the issue generally. CNN is essentially focusing on a peak in that noise.
McCain got a significant convention bump in 2008 that temporarily put him ahead of Obama. If CNN didn’t see it, that’s just their sampling issue.
Nate Silver says that Trump is now ahead in popular vote and electoral votes. I’m beginning to really doubt his methods.
That’s the “now-cast”, meaning he would be ahead if the election was held today. Which is possibly true, given the “bounce” he got. The current polls show him winning.
Of course, that’s like saying a team is going to win if they score a run in the top of the 1st inning. It helps, but it’s not definitive in any way. Long way to go yet.
Well, they’re publicizing and hyping the “now-cast” movement for attention, but to be fair the forecasts don’t put Trump ahead (yet).
Interesting to compare today’s posts from Sam and Nate, actually. The core facts being pointed out are actually nearly the same (a ~4-point bounce that needs careful interpretation), but you can see how his writing is a bit more, uh, “news-y” for the eyeballs. The “real” takeaway about loss of accuracy near the conventions is buried in the bottom of his text update today, rather than being the headline.
The 538 forecast assigns a finite probability to Trump winning over 500 electoral votes. Enough said. Wonder if the Russians have any August or September or October or November surprises in store for us.
Whereas here the “now cast” would point to a 95%+ chance of a Clinton win.
The latest national PV polls pretty consistently show Trump slightly ahead; I have no real trouble believing that.
Silver’s “now-cast” does a lot of work to try to extrapolate state results to the present day without actually having sufficiently recent polls, so it needs to be taken with a bigger grain of salt. And he knows perfectly well that convention bounces don’t last, so the projections for November are different.
…Whereas Sam’s “now-cast” is just directly using poll results that could be days or weeks old, so it’s not going to catch a rapid convention bounce in real time. If the bounce fades rapidly or is neutralized by the DNC, it may never fully manifest in the PEC model.
If readers want a snapshot of current national conditions, I think the HuffPollster aggregate is sufficient.
Nate Silver wants traffic. Sam Wang wants to be right.
Obviously he wants to be right as well! However, with the pressure to attract traffic, he does not get rewarded for being right too early.
I would argue this is driven by two considerations:
1. He got badly burned recently in sticking by his ‘Trump won’t win the nomination’ claim until as late as February.
2. 538 isn’t just him doing politics anymore, it’s a much broader and more time-intensive concern. However, it’s still built in large part on the reputation he built by predicting the 08 and 12 elections. If he keeps getting burned, that could be very bad for 538.
The result of these is that he is playing things very conservatively (non-political meaning) right now. Trump definitely got a bump, and if the election was held today, that bump would definitely matter. Those are two indisputable facts. Everything else is conjecture, and I think he’s going to limit his conjecturing until late August or so, when polling actually starts to get meaningfully predictive.
It makes sense that a “now-cast” that takes into account Trump’s post-convention bounce would show him ahead in EV.
But we don’t actually have a lot of state polling that is new enough to incorporate that.
On this site, the “Trump +2” map probably comes pretty close to showing what it ought to look like. (Right now, what you get is the absolutely minimal Trump win, unless you count throwing it to the House: exactly 270 electoral votes for Trump.)
538 has also, for all its past merits, hit some pretty significant clams in the last year, just not here: Scotland out of UK, UK remains in EU. Therefore, the recent track record is not strong, at least in areas outside a sphere of supposed expertise, which may point to another peril of “intangibles,” that as awkward as they can sometimes be locally, they travel exceptionally poorly.
I feel like if one just averaged the polls and eliminated Rasmussen and Gravis the predictions would be very accurate.
The elimination of Rasmussen and Gravis obviously based on their poor performance from the 2o12 election, see this from Drew Linzer: http://votamatic.org/another-look-at-survey-bias/
“I feel like if one just averaged the polls and eliminated Rasmussen and Gravis the predictions would be very accurate..’another-look-at-survey-bias/’”
But if Dr Wang just eliminates polls based on his own opinions, isn’t that survey bias? Not that he wouldn’t do a better job than almost anyone. Some pundit sites state percentages by which they find certain polls skew, largely to the right I’ve read. to the extent that that can quantified it seems reasonable. I infer from the New York Times summation of um predictors that only one or maybe two are considered data based: PEC is definitely one.
I trust everyone here will correct me if I’m mistaken 😀 if not PECk me to death, but the method you suggest likely has an eerily similar result to what PEC’s basic algorithm does: Show a median result rather than the average. So in contrast to clickbait news, outliers are self-de-emphasizing. Also, Sam doesn’t make predictions per se. This site displays calculated likelihoods. This election (or the aggregated polls about it) is been going strangely smoothly. I fear that pollsters arent getting great info re who really will vote and why. More polls should be open to answers about more than two candidates, in case it matters. Hope there’s a good voter turnout. The better the turnout, the more accurate random surveys/polls should prove, no? What’s intriguing here is Sam’s commitment to analyzing what little IS known.
What I am suggesting is that if you absolutely know some pollsters are biased (as demonstrated by Drew Linzer using a lot of data) then eliminating them and then taking the average of the rest could potentially give you a more accurate prediction.
This is a little like Nate Silver at 538 but I feel he goes over the top in applying sophisticated algorithms which the data probably doesn’t warrant.
There are always decisions to be made as to the algorithm to use (e.g. how far back do you include polls and how much etc.) but simpler and poll data driven is probably better, much like here at PEC.
@Sam:
Wow, that’s a pretty sharp drop in your Generic Congressional Preference, from +7% D to +3% D. Can you shed some light on that?
Two new surveys spanning July 22-24. It is equal to the 4-point post-RNC bounce. I am a bit surprised to see it, since I thought disaffected Republicans were expressing their downticket preference all along. It makes me believe in (a) the coattails idea, or (b) the idea that response rates are variable, and are guided by factors like voter enthusiasm.
Professor Wang, you said: ” It is equal to the 4-point post-RNC bounce.” But then I see the post-RNC bounce has been corrected to 1 point. To me, that suggests that choice (b) is likely to be the correct one. Voter “enthusiasm” probably correlates with “hope”, and I’d say hope of a Republican win was at a low going into the Republican convention. As the wind drops out of that particular sail, I expect enthusiasm to wane.
I’d say that’s going to be the story of this election; waxing and waning hope/enthusiasm on the Republican side.
That congressional tracking chart. Wow. Big move.
As a newbie to PEC and enjoy it very much, but not a newbie to Bayesian inference, I have a question. 538 uses a form of Bayesian model averaging accounting for past performance of the various poll used in the averaging. BMA is known to provide optimal predictive performance under a certain type of scoring rule. PEC uses the performance of the polls over a period of time to provide a prior that is constantly updated, but it is not clear how the uncertainty in each individual poll is accounted for, given that over time each poll’s uncertainty will change. I’m curious if PEC is doing some kind of model averaging in the background. A bit more on the methodology of the Bayesian forecast for PEC would be fun to read. Thanks.
You can find the code at
http://election.princeton.edu/code/matlab/
(you want the file EV_prediction.m)
It’s fairly easy to find where the priors are calculated even if you don’t parlez MATLAB. Look for the comments
“% What was long-term prediction? (the prior)”
and
“% All hail Reverend Bayes”
Although I am a stooge of Big Polling, nonetheless I could not bear to delete this particularly choice comment entirely. It came from “Luigi Valentino.” -Sam
The poll data reported has been faked. Source- http:/www.inf__ow_ars.com Trump is actually about 12+ points ahead of Hitlery. With all the exposed corruption I’m surprised it isn’t 20+ points. No one in their right mind could possibly vote for her except for the diehard lobotomized Clinton zombies who live in an alternate universe where laws aren’t enforced, crimes are not punished, unethical facts are fiction, Blue Lives Don’t Matter and the concept of email etiquette is primitive or completely nonexistent.
HA! Let me guess, he was one of the folks from UnskewedPolls.com in 2012? 🙂
Bravo, Luigi.
You’re call Sam, but after about a week when the comedic value has diminished I’d delete the URL to rob him of the little victory of SEO placement. 😉
Good call. Fixed.
After listening to two days of the “Bernie-or-bust” crowd, I’m afraid I don’t find Luigi particularly funny.
Thanks for the news and for Luigi’s comment, which my husband loved. I hope Luigi reads the site’s background articles, learns where your numbers come from, then comments again.
==> Today Neilsen reports 3 million more viewers tuned in to the DNC yesterday than saw the RNC, many in an important young demographic. Has anyone correlated historic Neilsen ratings to see if larger numbers suggest a larger post-convention bounce? This either bodes well for Democrats or I’m missing something. http://www.joemygod.com/2016/07/26/democrats-top-gop-in-convention-tv-ratings/
I read somewhere that ratings are just slightly inverse to outcome. Something like 7 of 10 elections have gone against the convention with higher ratings? Not sure of the exact number now. Maybe more telling but unsure of which direction? 😉
http://www.usatoday.com/story/news/politics/onpolitics/2016/07/28/dem-convention-gets-people-googling-voter-registration/87653020/
We may need Sam to the “N”th power for this election.
Interesting, I do seem to recall last election your model definitely lagged behind others. It got into the right spot but it persisted in giving dems better chances than anyone else for some time.
I’d imagine this has something to do with state polls being older. Its been sometime since anyone polled PA and that was at a HRC high mark for example.