Princeton Election Consortium

A first draft of electoral history. Since 2004

We’re back!

December 5th, 2012, 9:40am by Sam Wang


After a rest, I cautiously re-enter the fray. Analysis of polls is far less interesting when there’s no horserace. Let’s just say that I don’t expect NPR to drop by today.

Other themes from this summer remain important, including filibuster reform and gerrymandering. Gerrymandering lends itself well to what we do here. Stay tuned!

Filibuster reform: As I wrote in October, Senate Majority Leader Harry Reid is working on filibuster reform. This critical change will partially repair damage done to our legislative process over the last 20 years. For a primer on why it’s important, see James Fallows. Ezra Klein thinks the rules changes are incremental. But forcing Senators to actually speak on the floor puts them in the public eye. That makes them put their mouth where their money is. Don’t underestimate it.

Senator-elect Joe Donnelly (D-IN) is on the fence. If you are a constituent, or if you donated to his campaign, write to him at his current House site (update: to get the form to work, enter an Indianapolis ZIP code, 46202). At least 455 of you donated via PEC’s ActBlue page. For a further rundown of undecided Democratic Senators, see this piece by Kevin Drum. Lots of targets for persuasion there.

The Signal And The Noise: I’m currently reading this and working on a review. I am impressed at the sight of statistics on the best-seller list – a big win there. However, as a scientist, I’m disappointed to see prediction a bit divorced from physical law. Climate scientist Michael Mann has pointed this out. Also, there’s not enough math for me. But then again there never is.

I am sure many of you have read the book. What did you think? Update: here’s a rather negative, but factually accurate review. And Paul Papanek puts his finger on the problem with the science examples.

Tags: Politics

77 Comments so far ↓

  • bks

    Haven’t read the book, but aren’t the IPCC reports examples of predictions that did *not* fail?

    Here’s an interesting post-election poll from PPP:

    “49% of GOP voters nationally say they think that ACORN stole the [2012] election for President Obama. We found that 52% of Republicans thought that ACORN stole the 2008 election for Obama, so this is a modest decline, but perhaps smaller than might have been expected given that ACORN doesn’t exist anymore.”

    http://www.publicpolicypolling.com/main/2012/12/republicans-not-handling-election-results-well.html

    –bks

    • Matt McIrvin

      Note, the only options they offered were “Obama was elected fair and square” and “ACORN stole it.”

      Elsewhere I described PPP’s MO in cases like this as “Socratic trolling.” It’s amusing, but I don’t think we can take it as an accurate indicator of previously held beliefs.

    • bks

      There was a third option of “Not sure” for those who didn’t want to fall into PPP’s clever trap, Matt.

      –bks

    • Olav Grinde

      The PPP poll is most revealing about what Republicans think, feel and “know” — and it seems very much in line with previous polls that measure GOP views on Socialist, Kenyan-born hoodlum Presiden Barack Hussein Obama.

      Groan!

      My wife just completed a seemingly appropriate Christmas artwork celebrating what we “know” about the age of the Earth and when the dinosaurs lived.

      http://shelahh.files.wordpress.com/2012/12/babyjesusblessesthedinosaurs_600px1.jpg?w=600

  • Dave Barnes

    I am about 40% in on the iPad version.
    Disappointed in the low quality of the illustrations.
    Disappointed that there is no color when the iPad supports “millions of colors”.
    Easy reading even for the non-statistics geek (I say this even though I have 2 engineering degrees).

  • Jay Bryant

    Good to have you active again, Sam, though I very much understand taking a long rest.

  • Joel

    I’m thinking of buying the book for my brother, climate change warts and all (what’s with NYT writers and their blind/contrarian spot for climate change?)

  • Broadcast James

    I think it would be helpful to explain to Democrats why filibuster reform won’t make it too easy for a future Republican majority to quickly undo hard-fought progressive victories. In example, the filibuster seems like the best defense against the overturning of the Affordable Care Act.

    • Sam Wang

      Under the proposed reforms, a filibuster to guard against overturning ACA would involve Democratic Senators taking to the floor for long periods. I believe they would welcome that opportunity to talk about the details of that legislation, which are popular.

  • mediaglyphic

    would love to see a review of the book. Haven’t read it yet. I always find the assumptions behind the model to be more interesting than the results of the model.

  • Jim McMartin, Ph.D.

    I’m now into the second chapter of Nate’s book. Am impressed with the quantity of cited references so far. Writing is clear and his analysis of the financial meltdown in 2007-2008 seems spot on.

  • JaredL

    Good to see there will be new posts, Sam.

    I would like to see your perspective on The Signal and the Noise. I’m about halfway through and enjoying it so far but I can see the criticism that there isn’t enough math. I guess the last part has more of that, though.

    “I’m currently reading this and working on a review. I am impressed at the sight of statistics on the best-seller list – a big win there. However, as a scientist, I’m disappointed to see prediction a bit divorced from physical law.”

    This appears to be quite intentional on his part, and I think there’s an interesting debate there. If for no other reason, you should elaborate on this since he brings you, and this criticism, up in the book. From page 66 (parenthetical is in text):

    “The critiques usually rely, implicitly or explicitly, on the notion that politics is analogous to something like physics or biology, abiding by fundamentals laws that are intrinsically knowable and predictable (One of my most frequent critics is a professor of neuroscience at Princeton.*) Under those circumstances, new information doesn’t matter very much; elections should follow a predictable orbit, like a comet hurtling toward Earth.”

    * endnote references this post: http://election.princeton.edu/2008/08/04/on-a-flaw-in-fivethirtyeightcom/

    I find it interesting he brings you up because you are the opposite of the villains he’s talking about in that chapter, and even that section. Your model took in new information and the numbers it spit out changed along with it. Your graphs, probably the main feature of the site, showed precisely how the forecasts changed over time.

    • Sam Wang

      The point he makes there is a misunderstanding of what scientists do. In fact Bayesian approaches figure strongly in the scientific method, especially in my field, neuroscience.

      Frankly, I am finding his intellectual case (Bayes is good, hedgehogs are good, foxes are good) to be a bit incoherent. Just to pick an example, he seems to think that Ronald Fisher, a foundational statistician, argued wrongly that smoking did not cause cancer because he was not a Bayesian, i.e. he was a “frequentist.” As far as I can tell, the real reasons were that (a) Fisher purposely ignored large bodies of epidemiological and biological evidence, and (b) Fisher was a smoker. So it’s more a case of motivated reasoning. See http://www.epidemiology.ch/history/PDF%20bg/Stolley%20PD%201991%20when%20genius%20errs%20-%20RA%20fisher%20and%20the%20lung%20cancer.pdf

      I’m currently digging into a claim about self-canceling predictions (Chapter 7), in which he asserts that mobile navigation software undermines its own utility by filling up “fast” routes. He says there’s empirical evidence…but the footnotes lead to simulations. I am unable to find any actual evidence for this suggested phenomenon in the research literature, at least as of 2012.

      That’s just a snapshot of where I am now. I have to sort through which problems are important, and which are just details.

    • Jacob H.

      I thought the message of 2012 was…
      a) Fundamentals based forecasting works (Drew Linzer, John Sides)
      b) Pure polling based forecasting works (Sam Wang)
      c) A mix of fundamentals and polling-based forecasting works (Nate Silver)
      d) Punditry doesn’t work (Morning Joe)
      e) Motivated reasoning doesn’t work (Romney campaign)

      As for the IPCC, well, it works, but for different reasons. Arrhenius calculated the expected warming for a doubling of atmospheric CO2 over a century ago, and so far, we haven’t shown him to be wrong. The top-line level of global warming (rather than the localized effects) is a matter of simple and well-established physical law, rather than complex modeling, advanced computation, or contraversial frameworks.

    • Matt McIrvin

      in which he asserts that mobile navigation software undermines its own utility by filling up “fast” routes.

      That is an interesting claim, but it seems to me to be a potentially self-fixing problem. The better mobile navigation software is already actively responsive to traffic reports. Wouldn’t the net effect, given sufficiently good data, be to distribute the traffic over a wider variety of routes, producing better load-balancing than if everyone just takes the interstate?

  • MAT

    I felt that “The Signal and the Noise” performs a valuable service, much like Richard Muller’s “Physics for Future Presidents”, in that it presents a strong defense of Baysian prediction techniques, coupled with a realistic look at the limitations of the current state of practice. This isn’t a book aimed at Dr Wang. The avoidance of anything other than basic math seems very intentional. Silver wants his readers to ‘get it’, without drowning them in detail on how a model is built – it’s for those of us who aren’t walking around with STEM PhD’s. I found the explanation of Bayes formula and the need for priors to be one of the best and most intuitive I’ve read.

    If nothing else, educating everyone on just how that 30% chance of rain forecast is made is a valuable service indeed….

    “Physics for Future Presidents” explains the implications of physics as it relates to policy and everyday life – without requiring an understanding of the math behind it. “The Signal and the Noise” does the same for probability and predictions.

    • Some Body

      Haven’t read the book, but I think your main point holds for the 538 website as well: Silver is a journalist and a popularizer, not a scientist, and he’s writing for the broader public. I think having this in mind can explain quite a bit also about his choices in assessing probabilities of political results (his preference for erring on the side of greater uncertainty in particular).

    • Sam Wang

      My first take on the climate change chapter was that it wasn’t so bad as Mann made out. The basic issue predictionwise is that the predictions are coming true, but the exact amount is a fairly wide range. Given the profile and importance of the question, more clarity would have been welcome on what is true or false at a scientific level.

    • Richard

      I agree with MAT. For a book intended for the general public, it is very good. I have a PhD in physics, but I still really enjoy reading this genre of books, since I believe communication by scientists to the general public is so very important and I like to find examples of authors who do this well. I thought Nate did a very good job of presenting Bayesian statistics, although I thought he could have given a better “derivation” of Bayes formula instead just writing it down. But most of all, what comes through in the book is Nate’s quirky personality and his own eclectic set of interests like poker, sports betting, baseball performance, and political races. It was fun to see these topics juxtaposed with weather, climate and earthquakes. I know there is a bit of rivalry between Nate and Sam, but both are on the side of fact based analysis and rational use of statistical methods. So enjoy the Signal and Noise for what it is: a very good, though not great, book aimed at the general public.

    • urbantravels

      I’ll have to come back to the book another time. I really wanted to read it in all the post-election excitement, but I couldn’t get past the prefatory “historical” material, which was just staggeringly bad. Despite (or perhaps because of) the heavy sourcing, it committed an insupportably broad or off-the-mark generalization in almost every sentence. It wasn’t any good as a history of information, and I think it really ought to have started with the history of human understanding of risk and odds, which goes back a few thousand years prior to Gutenberg. At the very least, he oughtn’t to have skipped the classical period.

      The badness of this section may have nothing to do with the merits of the rest of the book; if that’s the case, it really ought to have been omitted. Did anyone who has read the whole book feel that the “historical context” portion was important and supportive of the overall arguments?

  • Gerald Tuggle

    I have not read the book but I did read Michael Mann’s commentary. He says, “Nate conflates problems of prediction in the realm of human behavior — where there are no fundamental governing ‘laws’ and any “predictions” are potentially laden with subjective and untestable assumptions — with problems such as climate change, which are governed by laws of physics, like the greenhouse effect, that are true whether or not you choose to believe them.”

    It is disappointing when a high-profile person (especially now) lends credence to the deniers by allowing a false equivalency between scientists and industry attack dogs. His expertise in the science of politics may give him the bonafides to write about the politics of science but he should clearly not muddy the water in the science of climate change.

    • MAT

      @Gerald

      I did not get the impression that Silver did a false equivalency on the climate issue, if anything, he does the reverse. What he does do is point out the many different difficulties building predictive models, including the impact that chaos theory can have. He backs this up with numerous examples of both successful and failing predictive modeling efforts. The overall theme of the book is to encourage individuals to approach things probablistically, rather than with certainity, in all aspects of life.

      With all due respect, I’m of the strong opinion that attacking the contents of a book without actually reading it first, is, well, wrong.

  • jharp

    I am a constituent of Senator-elect Joe Donnelly (D-IN).

    And I will contact him tomorrow.

    • Eric Walker

      Note that Donnelly’s current web page, as linked in the article, is useless if one does not live in Indiana: it literally will not accept any address (required datum) outside the state. I am outside the state, but donated, and would like to be heard. I guess I’ll have to telephone.

    • pechmerle

      Note to Eric Walker:

      This is typical for Congressional websites. But you can work around it very easily: just insert an Indianapolis zip code as part of your address and your message will go through. (I’ve used this trick myself successfully in the past.)

  • Some Body

    To do a shameless change of topic (and with all the caveats of commenting on single polls), but had anybody else noticed this:

    http://www.rasmussenreports.com/public_content/politics/mood_of_america/generic_congressional_ballot

    Given the identity of the pollster, and the result itself, and, finally, the timing, I’m wondering if this poll is going to produce a reaction of one sort or another on Capitol Hill…

    • pechmerle

      The Republicans on the Hill are already well aware of the sentiment shift reflected in this poll. Many of them understand perfectly well that the generic spending cuts they advocate are deeply unpopular when it comes to specific cuts that will affect their own districts.

      The most dramatic example is the automatic cuts on defense spending that going over the fiscal cliff (bless you Nancy Pelosi for that very astute bit of hard bargaining) would impose. That hits hard at conservative Republican districts in the South, as well as Southern California, that have either military installations or defense industry companies. I’m quite sure their business constituents have been letting them know how unthinkable (to them) such cuts are.

    • Jacob H.

      I think it’s pretty much inevitable that the president’s party becomes more popular following an election in which he is seen to have prevailed: bandwagon effect and all that.

  • MAT

    There is a pretty good conversation going on over at Jonathan Bernstein’s blog on the impact (or not) of forcing a talking filibuster. Much like PEC, the comments are well worth reading:

    http://plainblogaboutpolitics.blogspot.com/2012/12/what-talking-filibuster-reformers-are.html

  • Jonathan

    As an economist with some environmental training, I’m a staunch believer in the anthropogenic argument. I haven’t read silver yet, but It’s a simple story that doesn’t need a bunch of graphs (except for dramatic effect). For 10k years co2 never changed by more than 30 ppm in any 1k year window and rarely breached 280 ppm. Since indust revol era, co2 has climbed to 385 ppm. The science is straightforward: more co2 in at,osphere means more heat absorption and retention. Surface temp data are consistent with this conclusion.

    I think the real concern lies in estimatingn the impact that rising temps have on global economy vs costs of abating gigs. WeiTzman makes a strong case that the tail uncertainty over the catastrophic region of the support ought to give way to more aggressive policy. E.g.
    Reep.oxfordjournals.org/content/5/2/275.abstract

  • Olav Grinde

    Mother Jones has a very interesting article today about efforts by some in the Republican Party to change how Electoral Votes are allocated.

    http://www.motherjones.com/politics/2012/12/after-romney-loss-gop-plan-recovery-change-rules

    If successful, an allocation more in line with Congressional districts might well result in gerrymandering also deciding future Presidential Elections.

    This is especially the case if such legislation is passed in “Blue” states or states that barely swung Blue, while Red states remain winner-take-all…

  • Andrew

    “If successful, an allocation more in line with Congressional districts might well result in gerrymandering also deciding future Presidential Elections.”

    Republican congressional candidates got a majority of the statewide vote in FL, VA, OH, and CO, and just barely lost PA, WI, and IA.

    All a Republican for President needs to do to win is figure out how to become as popular with voters as the Republicans in the House. In many districts, the House Republicans ran 10-15% ahead of Romney.

    Gerrymandering has nothing to do with it.

    • Sam Wang

      This is probably not correct. You are cherrypicking. Democrats got a majority of the Congressional vote in NC, yet the delegation is now 9 R, 4 D. This is omitted from your list. I suggest that you redo the calculation for all states.

      It is true that Congressional candidates ran about 3% ahead of Romney at a national level. National Congressional and Presidential popular votes track one another in on-years. In such a situation, redistricting has a strong and potentially anti-democratic influence on the outcome. This “reform,” places state legislatures in partial control of Presidential elections. It’s a really bad idea – the opposite of what we need right now.

      In short, our democracy has problems at the moment, but the Electoral College isn’t one of them. Drop it, guys.

    • Olav Grinde

      Well, it would be interesting to see the math. What would be the electoral count (especially in the swing states that the GOP hyper-gerrymandered), if counting was done according to the proposed Pennsylvania rules? I.e. by congressional district, with a bonus for the state’s popular vote.

  • Dr. Xylem Galadhon

    Sam- at your ‘behest’, i got motivated, and filled out the form on Sen.-elect Donnelly’s page. This issue *MATTERS*, bigtime!!!

    Glad you continue to write–
    Dr. G.

  • E L

    ActBlue has a way to contribute to all the Senate sponsors of filibuster reform with one donation. I contributed about a week ago.

    • pechmerle

      EL, do you have a link to that page? I’m not finding it on Act Blue’s website.

      Thanks.

  • E L

    P. S. I’m very happy Sam and Andrew have chosen to keep PEC’s web site active. Thank you both.

  • Amitabh Lath

    Hi all, nice to read from all of you, I missed everyone.

    As for the “Signal and the Noise” I just read the look inside bits that Amazon allows, and it looks to be a bunch of cute anecdotes in the Freakanomics vein. As Sam said, not enough math.

    Frankly, I am a bit sick of all the Nate worship. He took a simple idea and made it look complicated. And he didn’t even plot uncertainties until late in the game.

    I remember sitting through an orientation about retirement planning when I first got this job. A large part of it was a big paean to the power of compound interest, really boring for those of us who learned about exponentials in high school.

    • Hayford Peirce

      Just because *you* know a bunch of things doesn’t mean that most *other* people do. A lot of people talk about stocks and income and retirement planning. For years I’ve been telling them about dividend-paying stocks that increase their dividends every year — your “compound interest” in a sense. They nod, and look thoughtful, and then forget whatever I’ve just said. (Example: a basket of dividend stocks that I first charted on a 30-year spreadsheet, and actually owned, back in 1995, was paying me $13,000 annually at the time — this last year the same basket paid me $76,000. And I was spending the dividends, NOT reinvesting them. Simple, but very few people do this.)

      In the same way, Bill James, the baseball stats guru who was certainly an inspiration to Nate Silver, years ago looked at a question that had been argued about for 80 years — who is the greatest center fielder, or first baseman, or pitcher, or just all-round player?

      He considered it carefully and came up with the first answer that had ever made any sense: You can’t compare apples and oranges. You have a pitcher like Warren Spahn who had a very BROAD career of greatness, and then you have a pitcher like Sandy Koufax who had a very TALL career of greatness. So that Spahn is a much greater pitcher in CAREER value (lots of 20-game seasons, but no particular year in which he was clearly the best pitcher in baseball), but Koufak was a much great pitcher in PEAK value (over, say, his three best years, when he was clearly the best pitcher in baseball).

      Simple, eh? But no one else had ever looked at it this way until about 1985. (By this reckoning, Mays was better than Mantle over his career, but Mantle was a better player at his absolute peak.)

      That’s sort of what Silver has done to this whole business — he has codified and popularized what maybe you and Sam and six other people already knew, or at least intuited. But that millions of others didn’t know.

      Including, it would appear, Mitt Romney, his advisers, George Will, Karl Rove, and the good folks at Fox….

    • Sam Wang

      I’m with Amit on this one.

      Poll aggregation using statistical tools is not a hard problem, but Nate has made it look hard and/or mysterious. It might be in his nature – he’s basically a hobbyist grown large, not a teacher. Or maybe reporters focus on personalities, which is how most readers think. Obviously I’m pleased about the attention the whole activity has gotten, which is basically thanks to him – he did work at it fairly hard. It’s a pity that it’s shaken out in a way that doesn’t improve awareness of reasoning about data.

    • Amitabh Lath

      Hayford, you are right, perhaps we underestimate how confounding simple (to us) mathematics can be to the general public. And it does not help that we spend our days in our academic bubble where everyone knows calculus.

      I apologize for sounding condescending.

      My fear is that instead of getting people to learn more about probability and statistics, books/sites like Nate Silver’s cause people to say “poll aggregation is difficult, I’ll let the experts do it, no way someone like me could begin to understand…”

      Given how many innumerate people began to appreciate polling math, it would be nice if we could draw some fraction into a proper intro seminar on probability, rather than “look at all the cool things experts can do with funky math tools”.

      Sam’s approach is quite different. The code and data files are all published, and the technique reasonably easy to grasp if you make the effort.
      It’s a powerful idea, using the median. I think I could explain it to a humanities major.

    • Wheeles cat

      Oh Amit. There are so few of us on the barricades anymore. Perhaps we should just go dark and wait out the new Age.

  • mediaglyphic

    I think Nate Silver main genius is marketing and timing. Often the material spoils of an idea go to the best marketers.

    I haven’t read “Signal and Noise” but in the past i have read the adage that the best empirical forecasters cannot tell one how they did it. The more complex the issue, the more variables there are. Does Silvers book deal with complexity theory at all (mathematically or in prose/poetry)?

    • Kevin Kesseler

      “the best empirical forecasters cannot tell one how they did it.”

      I couldn’t disagree more. As a professional modeler, I think that one of the most significant ways in which Sam is a better forecaster than Nate is that he can and does explain exactly how he did it. If you can’t explain it well enough for others to repeat it then it isn’t really scientific. Sam has laid out his methodology thoroughly enough for anyone to reproduce his results and while Nate is not as transparent, it wouldn’t be difficult to make a model similar to his and I’m sure that Nate himself could explain exactly how to do it. Do you have an example of a political forecaster better than Nate or Sam that cannot explain his methodology?

      I don’t know if Nate’s book deals with complexity theory, but the fact that Nate created his electoral model implies that he knows something about dealing with complexity.

    • mediaglyphic

      Kevin,
      I hear you that you believe otherwise, but what does the evidence say? Its my understanding that the best forecasters (empirically speaking), are not able to explain the calculus behind the forecasts. Do you have evidence otherwise?

    • Hayford Peirce

      That’s twice you’ve said that now, Mediag, but it’s about like me saying twice that the Moon is made of green cheese. Repeating it doesn’t necessarily make it true.

      1.) *Which* forecasters are you talking about? And forecasting *what*? SuperBowl outcomes? The weather tomorrow in Tucson, Arizona? How long Assad will remain in power? Who will win the 3rd Congressional District in New York in 2014? What the Dow Jones will do next week?

      2.) And *who* is saying that “the best empirical forecasters cannot tell one how they did it”? Paul Krugman? Duke in Doonesbury? A professor of forecasting at Harvard? One of the witches in Macbeth? Warren Buffett?

      Unless you can answer those questions specifically, I don’t think we can take you seriously.

    • Kevin Kesseler

      mediaglyphic,

      I’m not sure what you mean by “the calculus behind the forecasts”. What “calculus” behind Sam’s forecasts, for instance, is he unable to explain? To my mind, he has explained exactly what he’s done to the point where I could reproduce his analysis exactly if I wanted to — is there some higher standard he failed to meet in regards to conveying an understanding of his methodology and results?

      Or, to take a different example, what is inexplicable about how weather forecasting works? In the case of weather, who is the forecaster anyway?
      The guy on your TV who tells you it’s going to rain today?
      Or the person who developed the computer software that he used to make his prediction (or the software itself)?
      Or is it the one who developed the forecasting methodology that the software implements?

      I think that you need to clarify what it is that you are suggesting before you ask for evidence of whether or not you are correct. I’m suggesting that we look to Nate and Sam for empirical evidence regarding forecasters — do they provide examples which are consistent with your hypothesis or not? Why?

    • mediaglyphic

      @kevin,
      I am not criticizing Sam or Nate Silvers methodology. I am trying to get at the bigger picture of forecasting. Both PECand 538 use polls as the basis. 538 is more complicated than PEC and doesn’t do as well. If we look at more complex multistate forecasting (stock portfolio or geopolitical events) We can’t use polls (though polls work not badly for individual stocks).

      I do need to find my source for “what makes a forecaster” and trust me i am looking through old notes.

    • Kevin Kesseler

      mediaglyphic,

      You asked for empirical evidence of forecasters who understood “the calculus behind the forecasts” — I’m asking if you think Nate and Sam, as forecasters, have this understanding and, if not, what it is about their understanding of their forecasts that you feel is lacking. In other words, how do the obvious examples of forecasters in this discussion fit your hypothesis?

    • mediaglyphic

      Kevin,
      i do think that both PEC and 538 are forecasters. I do think its only one kind of forecasting and one for which polls (the main input) have usefull information. There are a host of other forecasting problems which don’;t. I haven;t read Nate’s book, but the title suggests that it addresses the generalized problem. I am wondering if anyone here knows of an empirical study that looks at a variety of forecasting problems and comes up with conclusions.

  • mediaglyphic

    Hayford,
    i guess you don’t have empirical data either.

    I read this a while ago in a class i took at Penn, can’t seem to find the notes. I am not vouching for this, just asking if anyone on this board has looked at what makes a good forecaster.

    Am very open to new information, if you have some. If you don’t thats ok als0 (as you noted i don’t have any citations either).

    I was a wall street analyst for about 10 years and was ranked well. My very anecdotal observation was that *gut* was often better than models, and that simpler models were better than complicated ones.

    But i don’t have any conclusive proof of this.

    by the way who is the WE that you are speaking of?

    • Amitabh Lath

      For a random system, “gut” will get you the right answer 50% of the time. In telling their anecdotes, most people will round it up.

      It’s called confirmation bias.

    • mediaglyphic

      Amit, gut will only get you 50% correct if there are 2 states to forecast. For an N state system, random would be 1/n.

      I think the logic behind GUT is that for complex systems there are a lot of variables and often experts (good forecasters) model these internally. Of course i have no studies to site, just a vague reference to a forecasting lecture i attended more than 20 years ago!!

  • Wheelers cat

    I guess it’s true that you always become what you most despise. Nate has become a pundit. I’m reading Sean’s new book instead. Amit, did you get a share of the prize for the LHSC researchers?

    • Amitabh Lath

      Meidaglyphic: if there are N systems, and the probability of getting any one correct by random (election win, stock price rise, coin toss showing heads, etc) is 0.5 then the probability of getting all N correct is of course 0.5^N.

      (assuming no correlation between systems)

      For stock markets, one has to define win/loss properly. In a rising market, a chimp throwing darts will “win” if you define win as just coming out ahead in money. You need to normalize to some broad based index, and then half the guts will be above, and half below.

    • Amitabh Lath

      Wcat, nice to have you back. No cash infusion for experimental physics at the LHC, I’m afraid. I do know some of the theorists who got some big cash, so maybe we’ll get a nice dinner at a fancy restaurant.

      If I could choose my prize, I would want a nice unexpected resonance bump in the data somewhere.

    • mediaglyphic

      Amit, i am saying N states systems (the number of states is n) not N Systems. i.e.. the outcome could be from 1 to n. Forecast the outcome.

  • Ben

    I guess this is just an open question to the group – if you could recommend some books that are slightly more advanced in the area of statistics/probability than, say, Silver’s recent book what would they be?

  • Richard

    I want to argue for a moderate position both on the topic of Nate and his book. Nate is neither a demigod nor just a journalist. He is engaged in scientific modeling of opinion data, which is a topic of research in the field of political science. His efforts are analogous to the research of Drew Linzer, Sam and others. In the case of Drew, he is “officially” a political scientist and his modeling of opinion data is central to his academic research. Sam’s foray into political science is outside his main area of research, but it doesn’t mean it isn’t a scientific endeavor. Publishing results in peer-reviewed academic journals, although often used as a standard for whether research is scientific, is not a requirement. I am not sure if Sam has published his political science research in academic journals (I seem to recall him saying he was planning to), but the work is still legitimate political science, just as Nate’s work is.
    Nate did a good job modeling opinion data for this most recent election. I prefer Sam’s model to Nate’s or Drew’s or Simon Jackman’s, because I found Sam’s model to be the most elegant, transparent and informative. But I don’t discount the contribution of the other modelers. If one looks at the Applied Rationality and Margin of Error websites that Sam was kind enough to link to (http://election.princeton.edu/2012/11/13/an-open-source-thank-you/#more-8926), one can see that Nate performed well in comparison to the other modelers. He was on top in some benchmark categories, just as Sam and Drew were on top in other categories.
    And I do think what Nate, Sam and Drew did was challenging and deserves kudos. We are not talking Nobel prizes here, but it is still solid work. I also think what Nate did was hard in another very important sense – he was a lightening rod for criticism from the poll deniers; he held up well to a shitload of attacks (see Sam’s “dog pile on the rabbit” post). Nate’s insightful commentary takes a lot of work and is both valuable and enjoyable to read. Of course, others might not like Nate’s commentary or prefer Sam’s or Drew’s. I actually enjoyed them all and learned different things from each of them.
    As to Nate’s book, it is in the genre of popularizing science and a good read. It certainly deserves criticism in some areas and I thought Michael Mann pointed out some important shortcomings. Still, it is pretty damn good in a tough genre. It compares well to books like Strogatz’ Sync, Watts’ Six Degrees or Barabasi’s Linked and there are plenty of details in those books open to criticism. It is incredibly hard to write to a popular audience and I found Nate’s book engaging and fun and imperfect.

    • Wheelers cat

      Point being, Nate has become a pundit. He pandered to conservatives for audience share which is why they turned on him with a vengeance at the end. He added artificial uncertainty for pageclicks.
      /sadface

    • Amitabh Lath

      Richard, I have thought about your argument.

      While I agree that Nate Silver was a lightning rod for conservative critics, that was due to a) his prominence as a NYT blogger and b) the opaqueness of his methods.

      The fact that Nate had a black box with all sorts of questionable inputs makes his setup qualitatively different from Sam’s. Frankly, does GDP or unemployment going up a fraction of a percent really make any difference? I know Nate had some scatter plots showing correlation w/ election outcomes, but did he ever show that they added information (ie, were not correlated with polls?)

      Sam, on the other hand, had only polls as inputs. You could quibble with the time window for dropping old polls, or one or two other things, but basically his algorithm can be sketched out on the proverbial back of the envelope. And he published all his code. Go take a look. Even if you don’t know MATLAB, you can see the basic flow and how compact it is. You could run it on an old classic HP programmable calculator.

      Allow me an analogy with the LHC higgs search:

      The results presented in July 2012 were straightforward. The “money plots” were simple invariant mass plots. A physicist from the 1940′s would have understood what was being shown.

      Some years back, there was a debate if the higgs search should be done via multivariate techniques. There are powerful neural-net (NN) codes available. You train the NN to recognize signal vs. background using whatever criteria you think useful, and then you feed it the data and out pops your answer.

      You do not get a nice mass plot. You get a yes/no answer about the existence of signal, and a p-value. Of course, the sensitivity of an NN analysis is higher than the old-fashioned method.

      But the community knew that an extraordinary discovery would need a highly defensible analysis. Nothing is more defensible than an old-fashioned bump in a mass plot.

      Similarly, Sam’s calculation is simple, defensible and transparent. His response to complaints could be: here’s the data, here’s the algorithm, you do the math. While you could argue with individual polls, it is understood that the median is quite robust against outliers.

      When you construct a measurement, you have to think about how you would handle unexpected results. Black boxes are not good at that.

  • MAT

    OK, here’s some bait for a conversation – other than a combination of historal and demographic information (or maybe using just that), what information could be used to build a predictive model of how someone could be predicted to vote. In other words, could we of the PEC commentariat figure out how to open source something like the predictive model the Obama campaign used?

    Dr Wang’s methods work wonderfully on a national level when there is a rich level of polling. But what about for house elections, or better yet, state level elections? What can be used to identify the persuadable voters?

    • bks

      I think the lesson is that it’s all about persuading your voters to get out and vote, while not energizing those who would vote for your opponent. The GOP tried voter suppression:
      http://www.palmbeachpost.com/news/news/state-regional-govt-politics/early-voting-curbs-called-power-play/nTFDy/
      but that seems to have backfired, in that once the target voters became aware of the effort, it gave them motivation to turn out in large numbers. The success of the Obama campaign was in having lots and lots of volunteers and a coherent plan for getting known Obama voters to the polls. Same as it ever was.

      It’s very rare that questions about human behavior can be susceptible to the precision attained by Sam and Nate Silver. It’s one thing to predict who will win the Presidential election, which is a highly constrained process with a set of rules and procedures and a well-defined outcome, versus, say, predicting the outcome of giving weapons to rebels in Syria.

      –bks

    • Wheelers cat

      Oh bullshytt bks. How did OFA find persuadable voters? The Dream Team is how. Behavioral scientists. The bio basis off behavior.
      Political science is to neuropolitics as haruspicy is to forecasting.
      Red/blue genetics, cognitive genomics…that is the future. Like Dr. Wang says, brain science.

  • Wheelers cat

    And I can totes predict the outcome of intervening in Syria on the side of the rebels. Same as in Libya. Rebels win. My hypoth. Lets call it Arab spring game theory. IFF US support and/or conscript military == rebels win. Qed

  • Wheelers cat

    Conscript military is both neccessary and sufficient.
    Like Tahir.
    Thus only way oppo in Egypt can win is US intervention.
    So…quant. suf.

  • Wheelers cat

    Here’s a question for the commentariat . What is more strongly predictive of homo sap behavior– game theory or DSP?

    • MAT

      Wcat – sorry to sound obtuse, but in my world, DSP means digital signal processing. I’m guessing that isn’t what you meant. Can you expand a little bit on your question?

      I’ve just started diving into game theory. Cool stuff, wish I’d done this years ago….

    • Wheelers cat

      Yup DSP is what I meant. Linear signal processing of Gaussian distribution, mean mode median variance skew kurtosis. That is the current state of the art in predicting human behavior.
      What if the underlying structure of carbon based reality is non-Gaussian ? What if the CLT is just serendipity? What if symmetry is a lie?

  • Tom

    Just ran across this interesting article today entitled:
    Sociophysicists Discover Universal Pattern of Voting Behaviour

    The same voting patterns crop up in every country that shares a particular type of electoral system, say sociophysicists

    http://www.technologyreview.com/view/508676/sociophysicists-discover-universal-pattern-of-voting-behaviour/

  • pechmerle

    FYI, Congressman – soon – to -be Senator Donnelly’s online contact form no longer works (Indianapolis zip code or not). He has had to vacate his House office, and his Senate site isn’t up yet. But there is still a phone number at the web site, so by all means call him re filibuster reform.

  • skmind

    I want to see Mitch drone on and on explaining why he is filibustering his own bill.

    I bet C-SPAN ratings will spike with the reformed rules.

  • Wheelers cat

    Does anyone have a good source for spree killing incidents cross reffer’d by country and age of victims?

Leave a Comment