Comments on: The economics of reporting polls http://election.princeton.edu/2008/10/17/the-economics-of-reporting-polls/ A first draft of electoral history Thu, 12 Nov 2009 02:35:25 -0500 http://wordpress.org/?v=2.8.6 hourly 1 By: Sam Wang http://election.princeton.edu/2008/10/17/the-economics-of-reporting-polls/comment-page-1/#comment-1284 Sam Wang Fri, 24 Oct 2008 10:56:38 +0000 http://election.princeton.edu/?p=1917#comment-1284 <b>Jim Wheaton</b> - I would characterize that statement as statistical malpractice. Instead of a sampling error, CNN needs to place a confidence interval on the average, which is the point of reporting sampling error in a single poll. More general concepts in statistics are the standard error of the mean (approximately a 68% confidence interval), and the 95% confidence interval (similar to the famous Margin of Error). This is basic statistical good practive. On this site I am constantly reporting these kinds of quantities. In the case of a poll of national polls, if one uses 10 polls, as in my most recent post, the 95% confidence interval is Obama leading by 5 to 9 percentage points. Jim Wheaton – I would characterize that statement as statistical malpractice. Instead of a sampling error, CNN needs to place a confidence interval on the average, which is the point of reporting sampling error in a single poll. More general concepts in statistics are the standard error of the mean (approximately a 68% confidence interval), and the 95% confidence interval (similar to the famous Margin of Error). This is basic statistical good practive.

On this site I am constantly reporting these kinds of quantities. In the case of a poll of national polls, if one uses 10 polls, as in my most recent post, the 95% confidence interval is Obama leading by 5 to 9 percentage points.

]]> By: Jim Wheaton http://election.princeton.edu/2008/10/17/the-economics-of-reporting-polls/comment-page-1/#comment-1273 Jim Wheaton Thu, 23 Oct 2008 23:07:39 +0000 http://election.princeton.edu/?p=1917#comment-1273 The article below, from CNN shows a "poll of polls" but claims that it "does not have a sampling error". Is this an example of laziness on their part, or choosing polls that are statistically incompatible in some way? -Jim ---- (CNN) – As Election Day inches closer, Barack Obama continues to hold a significant lead over John McCain, according to CNN's average of several recent polls. The Illinois senator now holds an 8-point lead over McCain in the latest CNN poll of polls, 50 percent to 42 percent. That lead is 1 point larger than it was in Wednesday's poll of polls. The national general election poll of polls consists of four recent surveys: Fox/Opinion Dynamics (October 20-21), Reuters/C-SPAN/Zogby (October 20-22), Gallup (October 20-22) and Diageo/Hotline (October 20-22). The Poll of Polls does not have a sampling error. The article below, from CNN shows a “poll of polls” but claims that it “does not have a sampling error”. Is this an example of laziness on their part, or choosing polls that are statistically incompatible in some way?

-Jim —-
(CNN) – As Election Day inches closer, Barack Obama continues to hold a significant lead over John McCain, according to CNN’s average of several recent polls.

The Illinois senator now holds an 8-point lead over McCain in the latest CNN poll of polls, 50 percent to 42 percent. That lead is 1 point larger than it was in Wednesday’s poll of polls.

The national general election poll of polls consists of four recent surveys: Fox/Opinion Dynamics (October 20-21), Reuters/C-SPAN/Zogby (October 20-22), Gallup (October 20-22) and Diageo/Hotline (October 20-22). The Poll of Polls does not have a sampling error.

]]> By: Sam Wang http://election.princeton.edu/2008/10/17/the-economics-of-reporting-polls/comment-page-1/#comment-1246 Sam Wang Thu, 23 Oct 2008 12:09:39 +0000 http://election.princeton.edu/?p=1917#comment-1246 Chris has a point. To put it in a less partisan manner, there is a considerable advantage to polling being done by many organizations, each with its own view of defining priorities and likely votes. An aggregate of pollsters performs much better than any one pollster. Again, I point to the remarkable power of poll meta-analysis in 2004 and 2006 in advance prediction of outcomes. It would be hard for polling by one organization to reproduce that - or to inspire trust, as indicated by Chris. Chris has a point. To put it in a less partisan manner, there is a considerable advantage to polling being done by many organizations, each with its own view of defining priorities and likely votes. An aggregate of pollsters performs much better than any one pollster.

Again, I point to the remarkable power of poll meta-analysis in 2004 and 2006 in advance prediction of outcomes. It would be hard for polling by one organization to reproduce that – or to inspire trust, as indicated by Chris.

]]> By: Trevor http://election.princeton.edu/2008/10/17/the-economics-of-reporting-polls/comment-page-1/#comment-1244 Trevor Thu, 23 Oct 2008 06:58:26 +0000 http://election.princeton.edu/?p=1917#comment-1244 Mr. Wang: I've been enjoying your analysis since the conventions. Whenever I feel over-drama'd by cable news I come back here for a reality check. In return, I ordered a copy of your book through my local b&n today. Mr. Wang:
I’ve been enjoying your analysis since the conventions. Whenever I feel over-drama’d by cable news I come back here for a reality check.
In return, I ordered a copy of your book through my local b&n today.

]]> By: Chris http://election.princeton.edu/2008/10/17/the-economics-of-reporting-polls/comment-page-1/#comment-1231 Chris Wed, 22 Oct 2008 23:41:56 +0000 http://election.princeton.edu/?p=1917#comment-1231 David Shor: By the same logic couldn't you say that any publicly available information is a public good? (Indeed, isn't that one of the original rationales for copyrights - they're the Enclosure Acts for information?) Yet, somehow, the idea of a Ministry of Information fails to reassure me. Particularly at present - imagine what kind of heck of a job a Bush administration polling agency could do. RCP's cherry picking would be minor by comparison. David Shor: By the same logic couldn’t you say that any publicly available information is a public good? (Indeed, isn’t that one of the original rationales for copyrights – they’re the Enclosure Acts for information?)

Yet, somehow, the idea of a Ministry of Information fails to reassure me. Particularly at present – imagine what kind of heck of a job a Bush administration polling agency could do. RCP’s cherry picking would be minor by comparison.

]]> By: David Shor http://election.princeton.edu/2008/10/17/the-economics-of-reporting-polls/comment-page-1/#comment-1197 David Shor Tue, 21 Oct 2008 14:57:12 +0000 http://election.princeton.edu/?p=1917#comment-1197 I think this skirts the real economic issue here: Polls are a public good. Information generated by polls are enjoyed by everyone, but paid for by a few. By basic economics, both sampling sizes and number of polls will be under-produced. There is really a compelling case that Public Opinion Polls, like the census bureau or ACS, should be done by a government agency. I think this skirts the real economic issue here: Polls are a public good. Information generated by polls are enjoyed by everyone, but paid for by a few.

By basic economics, both sampling sizes and number of polls will be under-produced.

There is really a compelling case that Public Opinion Polls, like the census bureau or ACS, should be done by a government agency.

]]> By: Peter H http://election.princeton.edu/2008/10/17/the-economics-of-reporting-polls/comment-page-1/#comment-1191 Peter H Mon, 20 Oct 2008 17:14:15 +0000 http://election.princeton.edu/?p=1917#comment-1191 Hi Sam, Thanks for your insight. Very enlightening. How do you feel about the work being done at FiveThirtyEight.com? BTW, I'm a musician, but have been a brain hobbyist since the mid seventies. I live two blocks from Borders, and I'm on my way to pick up your book. Do you think science will every crack consciousness? I asked Pinker at Borders and he hypothesizes that it is by its very nature beyond human understanding, similar to Colin McGinn's view. Someone asked, "What about Edelman?" and Pinker just said, "Not even close." What do you think? Regards, Peter H Hi Sam,
Thanks for your insight. Very enlightening. How do you feel about the work being done at FiveThirtyEight.com?

BTW, I’m a musician, but have been a brain hobbyist since the mid seventies. I live two blocks from Borders, and I’m on my way to pick up your book. Do you think science will every crack consciousness? I asked Pinker at Borders and he hypothesizes that it is by its very nature beyond human understanding, similar to Colin McGinn’s view. Someone asked, “What about Edelman?” and Pinker just said, “Not even close.” What do you think?

Regards,
Peter H

]]> By: MattF http://election.princeton.edu/2008/10/17/the-economics-of-reporting-polls/comment-page-1/#comment-1189 MattF Mon, 20 Oct 2008 14:01:31 +0000 http://election.princeton.edu/?p=1917#comment-1189 Gallup is, I think, making a start with the RV/LV problem by giving results from a number of different models. I guess that one would like to estimate the RV/LV uncertainty in a more systematic fashion (perhaps by simulation)-- but maybe the uncertainty turns out to be unpleasantly large. One can only wonder. Gallup is, I think, making a start with the RV/LV problem by giving results from a number of different models. I guess that one would like to estimate the RV/LV uncertainty in a more systematic fashion (perhaps by simulation)– but maybe the uncertainty turns out to be unpleasantly large. One can only wonder.

]]> By: Matt McIrvin http://election.princeton.edu/2008/10/17/the-economics-of-reporting-polls/comment-page-1/#comment-1185 Matt McIrvin Sun, 19 Oct 2008 23:38:59 +0000 http://election.princeton.edu/?p=1917#comment-1185 Couldn't the difference between Gallup LV1 and LV2 be indicative of a large systematic error in all the RV polls, which would be immune to most meta-analysis? As far as I can tell it has to do with demographics: traditional "likely voters" are older and less likely to be Obama supporters. So Obama's lead actually depends heavily on youth turnout, which has a spotty enough track record to make Dem strategists very nervous. If the young ones flake out again, many of these polls could be overstating Obama's support. I think that's part of why the Obama campaign is starting to ring the alarm about complacency; their GOTV depends on energy and they want their people a little worried. Analysts like you, who say that Obama basically has it in the bag and people should consider channeling money elsewhere, worry them very much. But the thing is, they also have an incentive to act worried even if you're right! I'm not sure what to think, myself... And then there's the contrary take of some of the fivethirtyeight people, who think that an "inevitability" narrative actually does more good for Obama than harm, because people like to vote for winners, and despair (for Republicans) is more damaging than complacency (for Democrats). Could be... Couldn’t the difference between Gallup LV1 and LV2 be indicative of a large systematic error in all the RV polls, which would be immune to most meta-analysis? As far as I can tell it has to do with demographics: traditional “likely voters” are older and less likely to be Obama supporters. So Obama’s lead actually depends heavily on youth turnout, which has a spotty enough track record to make Dem strategists very nervous. If the young ones flake out again, many of these polls could be overstating Obama’s support.

I think that’s part of why the Obama campaign is starting to ring the alarm about complacency; their GOTV depends on energy and they want their people a little worried. Analysts like you, who say that Obama basically has it in the bag and people should consider channeling money elsewhere, worry them very much.

But the thing is, they also have an incentive to act worried even if you’re right! I’m not sure what to think, myself…

And then there’s the contrary take of some of the fivethirtyeight people, who think that an “inevitability” narrative actually does more good for Obama than harm, because people like to vote for winners, and despair (for Republicans) is more damaging than complacency (for Democrats). Could be…

]]> By: blislife http://election.princeton.edu/2008/10/17/the-economics-of-reporting-polls/comment-page-1/#comment-1180 blislife Sun, 19 Oct 2008 06:53:15 +0000 http://election.princeton.edu/?p=1917#comment-1180 Say, Sam, I assume you or others on this blog have heard of the brilliant polling expert, Alan F. Kay. (http://www.alanfkay.com). I notice his articles stop in 2006. Has anything happened to him? His analysis of the polling world and his social solutions offer some of the best thinking around. Anyone with any knowledge of Mr. Kay: I'd love an update. Say, Sam, I assume you or others on this blog have heard of the brilliant polling expert, Alan F. Kay. (http://www.alanfkay.com). I notice his articles stop in 2006. Has anything happened to him? His analysis of the polling world and his social solutions offer some of the best thinking around. Anyone with any knowledge of Mr. Kay: I’d love an update.

]]>