Princeton Election Consortium

Innovations in democracy since 2004

Outcome: Biden 306 EV (D+1.2% from toss-up), Senate 50 D (D+1.0%)
Nov 3 polls: Biden 342 EV (D+5.3%), Senate 50-55 D (D+3.9%), House control D+4.6%
Moneyball states: President AZ NE-2 NV, Senate MT ME AK, Legislatures KS TX NC

NYT forgets basic statistics; Bloomberg Businessweek forgets 2012

May 31st, 2013, 11:22pm by Sam Wang

I’ll take a brief break from fluorescent protein design, human genetics, and autism and dive back into the fray.

First, a small item: last week in the NYT, a history instructor from a university in Washington state argued against looking at social statistics to learn about one’s own situation. Basically, she gets stuck on the idea of an average, and thinks statisticians are forcing her to be Jane Average.

This is horrifyingly silly. The obvious answer is to look at the standard deviation to get a clearer view. I explain today in the NYT Letters section. The philosopher Sissela Bok supports the I-fear-averages camp…though I think “Russian families” might be her version of standard deviations. In other words, maybe we agree if you look hard enough at what we wrote.

And today in Bloomberg Businessweek, Joshua Green gets the 2012 campaign partly right.

OFA data compared with pretty much the suckiest pollster of 2012[/caption]

First, he points out that Obama for America’s internal data was far better than Gallup’s. Well, sure…but that is because Gallup was one of the worst-performing polling organizations of 2012!

Then Green claims that OFA had better data than pollsters did. Well….no, no, no. Does he forget this?
2012 EV history - Princeton Election Consortium
The meta-analysis here at the Princeton Election Consortium caught the major pivot points, such as they were. We had far better time resolution than the OFA graph. Throughout 2012, we¬†pointed out that the race was basically unmoving since June. And we reported all of this live, day by day.¬†In fact, I would argue that the principal justification for Green’s graph labels comes from during-the-campaign poll aggregation.

The point is not about how great we are here at PEC. It’s that publicly available data during the campaign told this story as it unfolded. OFA data is certainly interesting, but it is not a Rosetta Stone.

P.S. Of course, the Meta-Analysis has a certain time lag that one must be willing to visually “un-lag” or un-filter. The turning points are the key. If you want to see sharp, single-day turns, that is also available. National polls are easier. Here is one example; there are others on this site if one digs.

Tags: 2012 Election · President

17 Comments so far ↓

  • Gold Star for Robot Boy

    “…a history instructor from Washington State argued against looking at social statistics…”

    Quick clarification: That “Washington State” is not Wazoo, but Evergreen State U. in Olympia. A few hundred miles distant from Pullman, but light-years apart in attitudes.

    • Sam Wang

      Sorry. I meant the state of Washington, as opposed to the District of Columbia. I chose wording to avoid confusing people who do not know where Evergreen State University is.

    • Tsuyoshi

      Evergreen State College, actually.

  • John Parenteau

    People try to take shortcuts using statistics. Even expanding from mean to median & standard deviation loses information. To really understand, you need to know the distribution – just saying “one standard deviation” for a Gaussian distribution means one thing, but for a Rayleigh of Uniform it can be quite different and quite misleading.

    • Sam Wang

      John Parenteau, I don’t think this is the most pressing need for this audience. If all people know is the average, then lesson #1 is median and SD! Also, we’re talking about social data here, which can long-tailed but is usually not that weird.

      As for Rayleigh distributions…the limit was 150 words. Couldn’t quite figure out how to explain a skewness of 2sqrt(pi)(pi-3)/(4-pi)^3/2 in that space…

    • Jan Galkowski

      It depends whether the statistics quoted are sufficient for the distribution in question or not. If they are, you don’t need the distribution, by definition.

      The thing is, of course, and here you are closer to what’s there, few empirical distributions fit neatly in as an idealized one. This does not mean the idealized ones are useless, merely that there either needs to be some way of capturing that complexity within the idealized frame, or it means there needs to be a way of doing estimates and inference with the empirical distribution.

      That sounds easier than it is, because, almost by definition, estimating a distribution accurately is much more difficult than estimating a point statistic, especially on its tails. Often, in terms of decisive impact, the character of the tails are the most important features. So, by all means, go with the empirical distribution if that sounds better but, if you do, expect to spend a LOT of time and work obtaining a good figure for that empirical distribution.

  • Amitabh Lath

    I too thought about the PEC estimator when I saw the OFA plot by Joshua Green in Bloomberg.

    Look at the sharp edges (infinite slope) at the DNC, 47%, and first debate.

    I remember being disturbed by the sharp drops in the PEC, post Ryan VP-pick and first debate. My thought was there was no way the system (the electorate) could respond that fast to an input.

    But the OFA curve makes the PEC look like an overdamped harmonic oscillator.

    I suspect the OFA model is quantized in some way.

    Arguably, the 1st derivative is as important to a campaign as the actual value of the function, and speed is of the essence when you are doing ad buys.

    • Sam Wang

      The Meta-Margin/EV estimator is temporally filtered by the availability of data. One could deconvolve it – or look at the national data at the bottom of the post. I am curious about the time resolution of the OfA data, i.e. are those plateaus really as noiseless as indicated? Really?

    • Amitabh Lath

      Not clear from the Bloomberg article if the OFA curve had any polling inputs. Polls don’t do step functions.

      It could be a self-contained simulation, or a selected sample of people meant to represent entire groups, who get repeatedly questioned.

  • Some Body

    Actually, if you look hard enough, I think you also agree with Stephanie Coontz. Here’s the main point of her article and the study (by Anthony Mancini) it touts: “treating the average response as if it was the normal or typical outcome can lead to bad social policy and inappropriate therapeutic responses”. Simplistic, silly, and manipulative uses of statistics (and averages in particular) are too widespread to ignore, and Coontz is right in calling attention to such misuses of statistics in certain fields.
    On the other hand, I don’t think it would be fair to read Coontz’s article as rejecting statistical analysis altogether. I also don’t think it was fair of you, Sam, to suggest that she is unaware of medians and SDs. There’s nothing particularly silly about her article once you take into account its actual aims and its actual target audience.

    And as for the OFA model vs. your median EV estimator–well, there is in fact no way to tell which model is better. The only hard data point to compare the models against is the actual vote. You may claim your model responds faster to events, and the OFA people may respond that their model shows the events actually didn’t move the dial at all, so there was nothing to respond to.
    Moreover, it’s not at all clear putting these two models one against the other (or the OFA graph against the Gallup numbers, for that matter) would be comparing apples to apples. Joshua Green does not actually tell us what the OFA model was a model of. Was it a model of the popular vote or of the EV margin (one would assume that the particular graph in question reflects a popular vote estimate of some kind, but there are ways around that conclusion)? Was it modelling the results “if the election was held today”, or was it a forecast of the Nov. 6th results? Was it based only on shifts in voter opinion, or did it factor in GOTV in some way? There’s just too little information about it.

  • Amitabh Lath

    I just read the Coontz article (and Sam’s response) again. I think I understand what Coontz is getting at.

    When you examine an ensemble of systems, there are going to be outliers, and outliers to those outliers… and so on.

    Coontz points to unwed mothers. They do poorly in society. Encouraging marriage might help, but expose a subset to worse outcomes.

    I know little about social sciences, but an apt analogy might be changing the oil in cars. The average car needs an oil change every 3k miles. But that rule exposes a sub-population (cabs, cop cars, racers etc) to damage since they need more frequent oil changes. So you make a 1st order correction for cars that frequently rev high. They get changed every 2k miles. Then you make a 2nd order corrections for cars that never speed into high revs. They get changed every 5k miles.

    At some point you stop as the sub-populations get too small to matter.

    And so on. What Coontz is saying is that if you do not do the higher-order corrections for the sub-populations that differ from the mean, they could actually end up worse (say, if the higher order correction had the opposite sign to the zeroth-order one).

    Social sciences are really difficult. To some extent, every unwed mother is sui generis. As a society we cannot even agree on how to allocate resources to tackle the “mean” much less the subsets.

  • Will Hutchinson

    Also, fivethirtyeight showed essentially no movement over the summer and fall, except for the 47% and debate 1. Nate Silver and Sam Wang helped me keep my sanity during the period and provided me the intellectual backup of their methodologies to convince me their numbers were broadly correct.

    Thanks again for the great public service.

  • Matt McIrvin

    We’re seeing an interesting miniature replay in the Markey v. Gomez Senate race in Massachusetts right now. All the respectable polls and the poll aggregators show Markey 7-12 points ahead, but these splashy news stories from conservative outlets keep popping up about how the race is a tossup or Gomez is actually somehow ahead. I guess it might keep Markey on his toes.

  • mediaglyphic

    Dr. Wang, (and anyone else)

    Did you see the Nate silver article on rdistricting.

    “Geography, Not Voting Rights Act, Accounts for Most Majority-Minority Districts”

    any thoughts?

  • Matt McIrvin

    …And Markey beats Gomez by ten points, just like the polls said (and in spite of very low turnout).

  • Avattoir

    Marshall generally doesn’t provide for reader comments on his own postings at TPM, so I can’t link your piece here to his latest brag:

    but SOMEONE should correct the record.

Leave a Reply to John Parenteau (Cancel)