National polling averages are contradictory: Pollster gives Obama by +0.2%, TalkingPointsMemo gives Romney by +0.1%, and RealClearPolitics gives Obama by +3.5%. Who’s right, and why are these measures fluctuating from day to day?
Much of the discrepancy is explained by my post on the problem with averaging:
These measures bounce around far less when using medians. RCP data since July 16th give a median of Obama +4.0 +/- 1.3% (estimated SEM, n=5), consistent with the Meta-margin calculated above, which is based on state polls. And a few weeks ago, the RCP median was at Obama +2.0%, again matching the Meta-analysis.
A second answer is the use of a long time window. These other sites’ engines achieve a smoother average by looking over a longer time window. The use of medians would allow them to achieve better time-sensitivity.
Third and finally, note that
Pollster and TPM national-poll aggregators are dominated by many data points from a few polling organizations. It is a clear methodological error to rely so heavily on any one pollster. At a minimum, all data from the same pollster within a given time period should be combined to make one contributing data point. (Update: Matt Dickinson points out in comments that Pollster in fact does do this. My error!)
Update: I forgot to mention two more issues. It is not necessarily the case that national polls are a more accurate gauge of the race than state-level polls. At any given moment there are fewer pollsters operating at the national level, which means that they are subject to greater fluctuation. Also, national polls sample from all states, which are politically diverse. This presents a difficult sampling problem.