Update: Natalie Jackson from HuffPost’s pollster.com responds at the end of this post. Again I thank her for all that Pollster.com does.
Holidays are over. I see that journalists, including poll aggregators, are still focused on the Presidential horserace. As Zenger at Electoral-Vote.com has pointed out, sites such as FiveThirtyEight are under economic pressure to attract traffic. And there is nothing to attract eyeballs like a crazy Presidential race. Still, from a substantive standpoint, it might be more appropriate to spend efforts on, I dunno…issues? See this excellent critique of media coverage by Jeff Jarvis, which includes a good hard whack at the media obsession with “balance” and polls – basically, tricks to let reporters escape engaging head-on with substantive issues. If journalists insist on horserace coverage, at least focus on downticket races in Senate, House, and even state legislatures – and maybe write about some issues along the way. These races will determine the power dynamic in 2017 under the new President, whoever she may be.
Just to remind everyone, variations in this year’s race are quite narrow, consistent with the last 20 years of partisan polarization. Polarization has made both the GOP and Democratic nominees unacceptable to nearly all supporters of the other party. In addition, Donald Trump is radioactive to about one-fifth of his own party. As a result, this year’s race is full of melodrama, but numerically stable. In 2016, the Princeton Election Consortium’s state poll-based aggregate has only varied between a median outcome of 310 and 350 EV for Hillary Clinton.
The Meta-Margin, which is defined as the front-runner’s effective lead using Electoral College mechanisms, is a very low-noise and stable measure – as opposed to single polls, which can be all over the place. You should generally ignore single polls, especially ones that surprise you. The Meta-Margin has varied between Clinton +2.5% and Clinton +6.5%, and is now at Clinton +4.0%, close to the season average of 3.8%. If it left the 2.5-6.5% range, that would be interesting. That has not occurred yet.
>>>
Now, a brief note about the virtues of casting the widest net possible when it comes to polls.
As in past elections, we rely on the good people at Huffington Post’s poll-aggregation operation for our polling feed. In 2008-2014, their quality control consisted of making sure a survey met disclosure standards such as those set by AAPOR. They also provide readers with tools to include/exclude polls by checking/unchecking boxes. I find the combined approach to be democratizing – and one of the great virtues of HuffPollster. It is also well-suited to the analysis approach used here.
This year they are being more aggressive. They are excluding from their database polls that are done on landlines only, as well as some Interactive Voice Response (IVR) surveys (“robo-polls”). The rationale is that these categories of poll are less reliable than newer methods. PEC’s current approach for data scraping limits us to whatever they provide, so this effectively excludes these polls from our calculation.
I would have preferred for them to continue to include the maximum diversity of polls, and then set their default checkboxes to leave out landlines and IVR surveys from their own averaging procedure. That would take into account their professional expertise, but would still allow users to have a single data source. The resulting feed would make them the unambiguous preferred choice over other sources such as RealClearPolitics and FiveThirtyEight.
Natalie Jackson of Pollster.com responds regarding IVR surveys: We are checking each IVR poll to determine whether it’s all landline or if it includes some cell and/or online panels. If they are all landline, we do not include it. If there’s some effort to account for cell users by supplementing with live callers and/or a web sample, we will include it.
I should say at this point that I had formed the impression – apparently false – that they were expanding their “no-landline-surveys” policy to also exclude all IVR surveys. Her response clarifies that this is not the case. I withdraw my concern!
Leave a Reply