Princeton Election Consortium

A first draft of electoral history. Since 2004

Poll geekery on All Things Considered and MSNBC

October 7th, 2014, 4:06pm by Sam Wang


NPR interviewed me and my lab members here at Princeton. It was fun, and it did get across that polls-alone might be enough. However, it didn’t have time to focus on one nuance, maybe because it’s a bit dull: PEC and other sites are not that different in their predictions. We’re talking one or less Senate seat of difference on average, well within the uncertainties. It’s just that Senate-2014 is a close election. It aired on NPR’s All Things Considered at 4:20pm, 6:20pm, and 8:20pm Eastern. This should work: [link to audio]

Update: Wednesday night on MSNBC’s The Last Word with Lawrence O’Donnell. The topic: Greg Orman of Olathe, Kansas. It’s archived here].

Tags: 2014 Election

15 Comments so far ↓

  • Edward G. Talbot

    The other thing is if the bias is targeted as opposed to systemic. If a pollster is specifically attempting to show certain results as opposed to merely using biased methodology, then an historical house bias becomes less probative. The median, on the other hand, retains its value – though of course the median still has its limitations in underpolled races.

    Keep up the good work, Professor!

  • Canadian fan

    Regarding the batch of Fox polls, 538 was up with an article on them yesterday stating that Fox is using a Republican generic edge of 7 percentage points – in other words, close to 2010 wave levels. This has been basically the Fox argument for some time. 538 finds that Fox polls have currently a 3.6 % Republican tilt. 538 takes that into account, and as a result, the Republicans have- according to their model – actually microscopically less of a chance to capture the Senate – from 56.5 % to 56.4 %. Now I know Sam doesn’t do that – which is why the meta margin reacted as strongly as it did. But Sam’s right. Take the longer view. See as many polls as possible. But with a generic bias as strong as the Fox polls, little wonder they were what they were.

    • Sam Wang

      The house-effects correction is probably a step in the right direction. One problem is that a fixed correction does not take into account that the effect can change over time. In this case, the above estimate of 3.6% bias towards the GOP might change as the election draws near. I believe HuffPollster has a more dynamic approach, which if so would be a better idea.

      As an exercise, browse through polls where other recent surveys are available: KS-Sen, KS-Gov, and CO-Gov. You will find that the Fox polls are more favorable to Republicans by 4%, 8.5%, and 4% relative to other polls done around the same time. This is an average bias of 5.5 ± 1.5% (SEM).

      As everyone knows, I am no fan of unskewing, both because of possible bias and because of the uncontrolled uncertainty that goes with not knowing if a pollster has changed methods. But if we were to unskew these Fox polls, we would get the following:

      AK-Sen: Sullivan +4% becomes Begich +1.5%
      CO-Sen: Gardner +6% becomes Gardner +0.5%
      AR-Sen: Cotton +7% becomes Cotton +1.5%
      KS-Sen: Roberts +5% becomes Orman +0.5%
      KS-Gov: Brownback +6% becomes Brownback +0.5%
      KY-Sen: McConnell +4% becomes Grimes +1.5%

      Most of these are not a huge surprise. The real news here is both the Kansas governor and Senate races have tightened up considerably. I judge Orman to still be in the lead, but that could change in the coming weeks.

    • Kevin

      Other problems: Fox could be modeling the race correctly, while the other pollsters are askew. Also, this is a situation where observing the behavior may change it. When someone like Silver publishes an article announcing that Fox polls have a consistent, measurable lean, Fox reads that article.

  • Steve

    Looking on the bright side, Nate’s decision to challenge Sam’s model has helped generate a lot of media interest, and even some excitement. The attack appears to have backfired as well, in that, Sam’s and PECs credibility seem to have only been enhanced by the debate, and by the media coverage.

    If you’re in the audience, a bumpy debate is better than a boring debate, or no debate at all.

  • Marvin8

    Nice interview with Lawrence, Sam!!!!!

  • Joe

    Sam, What’s your make of the slew of polls from FOX today showing +5 or more leads on some important races, particularly KS which went from +5-7 for Orman now showing a +6 for Roberts? House effect, statistical noise? Because they released them on a day where it was good poll after good poll for Dems, and suddenly in one swoop, all the medians came down, as well as the meta-margin.

    • Sam Wang

      These polls have a problem. Go look at the HuffPost data (clickable, right sidebar). A tell here is that the time window on the Fox polls overlaps with two other surveys showing Roberts+1% and Orman+5%. In other words, Fox is off the median by 4 percentage points. The race has probably tightened, but it’s not clear yet who’s ahead. Hate to say it, but wait for one more poll.

      It also seems odd that the news in these polls is so uniformly good for the GOP across the board. It could be true…but nothing so sudden has happened all season. It would be quite a black-swan event. At times like this, the median is an invaluable tool. It does the same thing as house-effects corrections.

  • Philip Diehl

    On a different subject:

    Sam, are we seeing a Kansas redux in South Dakota with the sudden emergence of Larry Pressler, a former GOP Senator who has twice endorsed Obama? There’s no chance he’d caucus with the Rs. See http://m.dailykos.com/stories/1335143

  • John Parenteau

    With all due respect to Dawkins, enough with “meme.” When and why did the good old English words like “idea,” “thought,” “concept” become inadequate?

    • Kenny

      I think meme implies its “infectious.” The term is not commonly used in “internet-speak” to talk about a particular (usually short-lived) trend in social media. So for example, the Rick Astley “rick rolled” videos from a few years ago, or the “sad Keanu” pictures.

      These weren’t merely ideas, thoughts, or concepts — they were this large shared common experience.

      Google described it better, “an element of a culture or system of behavior that may be considered to be passed from one individual to another by nongenetic means, especially imitation.
      a humorous image, video, piece of text, etc. that is copied (often with slight variations) and spread rapidly by Internet users.”

  • A New Jersey Farmer

    The media story is you vs. Silver with very little nuance involved.

  • securecare

    Disappointed they focused on the polling “debate” instead of the lab. *Sigh* maybe they will do something on the lab post election.

  • Alan Koczela

    Congrats! Looks like the lil dust-up with Nate is starting to pay off. I enjoyed reading your latest response to Silver and I am delighted to hear your hard work is paying off — an “overnight media sensation” after only ten years. All the best.

    • Sam Wang

      Actually, the interview was weeks ago. It was originally going to feature my lab’s research prominently. I’d say the dispute has been *bad* because I think it crowded out our brain research a bit…