Review of The Signal and the Noise
In Science magazine, Ben Campbell and I have a review of Nate Silver’s book, The Signal and the Noise. Briefly…it was good for people wh...
Senate: 48 Dem | 52 Rep (range: 47-52)
Control: R+2.9% from toss-up
Generic polling: Tie 0.0%
Control: Tie 0.0%
Harris: 265 EV (239-292, R+0.3% from toss-up)
Moneyball states: President NV PA NC
Click any tracker for analytics and data
The question arises: how good is the Meta-Analysis in comparison with a more conventional measure such as an average of multiple national polls? Answer: Today, the Meta-Analysis is over four times as accurate as an average of recent national polls. Furthermore, it gives an estimate in the units that matter – electoral votes. Here’s why.
Averaging is an excellent way to see past variation in individual data points. A standard measure of how well you know the true average is the standard error of the mean (SEM), which can be thought of as a multiple-poll version of the famous Margin of Error. Today, the SEM of the last 6 polls is 2.3%, a fairly typical value.
But the Meta-Analysis uses dozens of polls at any given moment from states in contention, and over 100 polls in all… Look at today’s Meta-Analysis, which gives an EV estimate of Obama 329, McCain 209, an EV margin of 120 EV. The Popular Meta-Margin is 3.36%, which gives a linear conversion of 120/3.36=36 EV per percentage point. (Remarkably, linearity is not a bad assumption, as seen in past election data.) Our 95% confidence band is 70 EV wide. Typically, such a confidence band is about 4 times the SEM, making our equivalent SEM 18 EV. This converts to about 0.5% – less than one-fourth the SEM of the Pollster.com average.
An important caveat is individual states are polled less frequently than the nation as a whole. Therefore the Meta-Analysis responds more slowly to changes in opinion. But since it is so much more accurate, it’s still likely to be a better way to detect swings. I can’t prove that – not yet, anyway…