Princeton Election Consortium

A first draft of electoral history. Since 2004

An open-source thank you

November 13th, 2012, 12:00pm by Sam Wang


Benchmarking of Election-Eve snapshots continues. Drew Linzer wrote me: “basically we all succeeded in different ways,” as seen at Applied Rationality and Margin Of Error. His update is here, and a recent interview with the two of us on KUSP-FM is here. A separate topic is genuine long-term predictions, which I will evaluate later (“A Presidential/downticket prediction challenge,” October 28).

Today I want to focus on what has been distinctive about the Princeton Election Consortium – and the role you played as readers.

Here are two indicators for how we did.

  1. This year PEC got over 4 million page views. That’s three times our traffic in 2008, and occurred despite several brief service outages in September and October, including one just before the election. It is consistent with an estimate (see survey) in which we had three new readers in 2012 for every reader who started following us in 2004-2010.
  2. Same as 2004-2010, I used simple statistical methods to identify knife-edge contests where readers could maximize their leverage. Because President Obama was ahead all season, this meant downticket races. The ActBlue thermometer at left shows one metric. This exceeded expectations by over 6-fold. The Crossroads GPS link does not indicate how much PEC readers gave Karl Rove. I imagine it was a small fraction of that organization’s $300 million in expenditures.

We did well on questions that affect next year’s legislation and messaging: Presidential electoral-vote and popular-vote outcome, specific Senate outcomes, and House seat and popular-vote counts. In all areas, the predictions were very close.

The ActBlue list did identify genuinely close Senate races, with the exception of Nevada (Berkley v. Heller, decided by a margin of 1.2%). Specific House races were harder to identify because of the absence of polling data – DCCC/RNCC was generally a better bet. Correctly identified knife-edge races should have been won 50-50 by each side. Although this was true, further refinement of this process is needed.

But these outcomes are just part of the picture. The real story is the mechanism of this project successes. Three principles were at work:

  • Elementary statistical methods were all we needed to reveal truths about the 2012 race. Medians and other outlier-rejecting tools were enough. Adjustments like house biases and chewing over polling internals were not necessary. Good statistics can cut through the noise – if you have the nerve to ignore the chatter. I deeply believe this to be true for many problems in public policy and science – not just elections.
  • Our data came entirely from public sources. Pollster.com was a rich source of unbiased information.
  • We used open-source methods. All of PEC’s code is posted. Every bit of the activity was open to you, the reader. This led to wonderful benefits. Two examples are the depth and analysis added by Rick in Miami and Froggy.

Using these simple principles, we matched benchmarks such as FiveThirtyEight, Votamatic, and Pollster.com – and gave the sharpest moving snapshot of the race that you could find anywhere on the Web.

My own writing was also open-source. I wrote about what you discussed in the comment threads. Readers were a rich source of discussion and ideas. The project succeeded in large part because of your thoughtful feedback. There are too many of you to mention. This year we hit a sweet spot by having lots of readers – but not too many. Andrew and I thank you.

One more thing. Our Congress has become increasingly paralyzed in the last 20 years by Senate supermajority rules. I encourage you, especially if you donated or are a constituent of a swayable Senator, to make contact. Encourage him/her to vote for a rule change. It’s important for our democracy, no matter who controls the chamber.

In coming weeks posting will continue, though less frequently. To follow us, use the RSS feed in the top right corner, which will ping you when there’s something new. Also, please suggest topics – on any topic, whether political, economic, scientific, medical, whatever. I am interested in where this platform can go.

I end by quoting one of our most frequent commenters, Wheeler’s Cat.

Rasmussen cheating, Gallup antique methodology, non-gaussian structure of reality, asymmetrical political behavior, all fell to Dr. Wang’s pure mathematical statistics.

you were right.

but it sure was fun talking about all that stuff.

it was an epic tale of meme-warriors and Poll Jedi battling the sinister forces of disinformation and spin and “gut-feelings”.

i felt a lot of esprit de corps with my fellow commenters– we were like a rapid deployment force of truth commandos slapping down republican eumemes and pundit spin every day.

Tags: 2012 Election

124 Comments so far ↓

Leave a Comment