In search of the nerd/poet gene
Today, I ask for your help with my laboratory research. It will only take a few minutes. The concept of a “physics gene” or “poetry...
Review of The Signal and the Noise
In Science magazine, Ben Campbell and I have a review of Nate Silver’s book, The Signal and the Noise. Briefly…it was good for people who don’t know any math or science, and was best when he recounted his own exploits in poker. But there were some flaws, for instance on the use of statistics in science. And on climate change…let’s leave that alone.
You can read the whole review here! (direct link to PDF)
Topics:
Great title, great article. I was an otaku of Nate Silver 2008– Nate 2012 was a ginormous letdown.
Nate seems to have been corrupted by market forces. I wonder if he is metamorphizing into a pundit or a conservative? Or both? His treatment of global warming in the book smacks of asymmetrical ideology and conservative eumemes….and I simply cannot forgive him for pulling his oversampling post on orders from someone or something.
Do we always become what we most despise?
Thing is….bayesians were wrong. The Bayesians, chief among them Nate himself, clung to apriori econometric data like Trofim Lysenko clinging to Lamarck.
Thanks for the shoutout to experimental particle physics.
As you can imagine, statistical issues are hotly debated in our collaborations. One of the points of contention is that a Bayesian prior allows you to confirm or deny that particular model. And only that model. Frequentists still have their adherents.
Most of us try to stay out of these debates, which can get heated.
Particle behave symmetrically, right Amit?
Humans don’t. That’s why someday the fractalists will sneer at the Bayesians like the Bayesians are currently sneering at the frequentists.
Apriori can’t model Black Swan events or indeed, new events.
Is there chaotic behavior in particle physics?
Thanks for the link to the review. All this talk of Bayesians vs. Frequentists is interesting. I still remember studying Fuzzy Sets and Dempster in grad school and can only vaguely remember these arguments.
Is there a good generalized book on forecasting. I haven’t read Silvers book yet, but it sounds somewhat narrow in its scope. Would like to get an overview of the state of the art (which might be too broad a topic for one book)
mediaglyphic, I presume you have seen this XKCD comic?
http://xkcd.com/1132/
Take a look at the short writeup here:
http://pdg.lbl.gov/2012/reviews/rpp2012-rev-statistics.pdf
The Particle Data Group is the final arbiter in most things in our field.
From the writeup (by G. Cowan):
Frequentist statistics provides the usual tools for reporting the outcome of an experiment objectively, without needing to incorporate prior beliefs concerning the parameter being measured or the theory being tested.
Bayesian methods allow for a natural way to input additional information, which in general may be subjective; in fact they require the prior p.d.f. as
input for the parameters, i.e., the degree of belief about the parameters’ values before carrying out the measurement.
…
Bayesian techniques are often used to treat systematic uncertainties, where the author’s beliefs about, say, the accuracy of the measuring device may enter. Bayesian statistics also provides a useful framework for discussing the validity of different theoretical
interpretations of the data.
Amit,
thanks for the links, i hadn’t seen the comic before very funny.
Also thanks for posting the paper. I am going through it slowly and am sure i will have more questions!!
What exactly did Bayesian analysis contribute to predicting the outcome of the election? Economic indicators turned out to be just noise. Aggregation removed bias and acted as cheater detection. And I maintain the polling was just snapshotting the composition of the electorate.
What were priors good for? Economic priors turned out to be WRONG. Only a few people even discussed the fact that Romney needed +60% of the white vote in a 72% white electorate.
He got 59% and lost.
Did Linzer use economic priors? Maybe he could do a guest post and defend rev bayes. I don’t see any value added myself.
I used a Bayesian formula to include national polls as part of the popular vote estimator. Properly applied, one includes the degree of certainty that an estimate contributes. That’s even true of economic indicators. It’s just that state polls are most accurate, historically. That’s all.
One could argue that be taking the median of polls rather than the mean, Sam was reducing the effect of outlier polls, and thereby introducing a (Bayesian) prior on probability distribution.
Linzer used a prior that included 2nd quarter GDP growth, but also approval ratings and a measure of tenure.
During Romney’s spike in October, the priors at Votamatic kept Obama’s chances up and might have helped Linzer predict the outcome early on. Linzer did manage to nail the outcome around July, but so did PEC.
Amit, don’t you mean Huber not Bayes? The Father of Robust statistics?
Thnx Nadia. But Linzer DIDN’T use a lot of economic priors. I wonder how he chose GDP. A reason the GOP failed so badly at prediction was that they relied heavily on economic priors. No president ever won reelection in such a dire economy. The GOP was ‘Spoofed by Bayes.’
Classical statistics relies heavily on symmetry.
But humans are not particles. As it turns out, we are quite shockingly asymmetrical in our ideology and political behavior.
We seem to be mixing up model building with Bayesian/Frequentist inference.
Bayes does not prescribe how you should build your model, that’s political science.
Bayesian inference does come in when you want a p-value (posterior) from your model’s test statistic distribution. Bayes requires a prior for the probability distribution of your inputs.
One could have had all sorts of economic data, approval ratings, world series winners, phase of moon, etc inputs to the model, and yet calculate p-value using frequentist inference.
Caveat: We are all Bayesian. There is no way not to let ones prior beliefs color how one constructs a likelihood. Every hypothesis I make comes with my opinion of how likely it is. Bayesian construction makes this explicit.
Amit, you are still relying on the assumption that the underlying structure of reality is Gaussian, which is the basis of modern statistical analysis. But what if its not?
Caveat: I think it prolly is for particle physics.
I’m not so sure about human reality.
What amazed and dismayed me about Signal & Noise was the shoddiness of the writing and frequent bouts of disorganization and repetition. In one memorable phrase, Silver says his job is to “weed out root causes.” Sam, you should do a book on this, and do it right!
Yes, the book could have done with a firm editing, and a critical eye to parts of it, especially regarding scientific subjects.
A book done right…that would be most effective if it had excellent, vivid, friendly illustrations, drawn by a master of the technical cartooning craft. Hmmm….
Now that I’ve pushed myself to read further in (and it did take pushing!), I like the book far better. As you said in your review, where he’s on firm ground with the content, the presentation is strong. There’s plenty worthwhile to read there. And, let’s face it, the guy deserves creds because he, like you, has done a great service to the country by making quantitative sense of information too often misused and distorted by partisan blowhards.
Those positives are quite true. I think it depends on whether we are grading on a curve that includes Ronald Fisher, or one that includes Peggy Noonan. Other than my own review, I enjoyed Leonard Mlodinow’s critique.
Nate Silver might be coming around to your kind of thinking, Sam. Here is a snippet from his recent column:
“In fact, I have grown wary that methods that seek to account for a more complex array of factors are picking up on a lot of spurious correlations and identifying more noise than signal. ”
You don’t say.
What did Bayes deliver in the election prediction? My point is that state poll aggregation delivered the same results. Bayesian priors were just noise as far as I saw empirically. I’d say Linzer’ s result was coincidence, not correlation.
I actually did better than Nate or Dr Wang if you recall. I predicted 332 for the electoral college based on state polling and game theory. And I predicted 4.2 for the popular vote.
Humans use Bayesian analysis naturally, this is true and ALSO part of the problem. Why repubs got spoofed I think.
It’s like rewiring a conventional human brain to understand q-physics. We expect priors to predict.
But what if they don’t? At least, not anymore they don’t. The speed up of memetic transmission and mutation delivered by social media and the Internet is why I think asymmetry is ascendant.
And Nate is going to have to take a knee to Taleb at some point. We all will. That’s why he’s softening on Bayes .
Are we going to be able to hear about Dr. Wang’s views on the neuroscience investment Obama reportedly wants to make? It’s gotten mixed reviews, but the detractors seem more concerned that it will take a bit out of other research funding than objections to the idea of a big project. Gary Marcus had some interesting thoughts about how to allocate the funding.
Sam, I am curious what you would respond to Nate’s recent column in which he pretty much downplays the effects of gerrymandering?
“Democrats are quick to attribute the Republican advantage in the House to gerrymandering. This is certainly a part of the story. (…) However, much or most of the Republican advantage in the House results from geography… “
That is false. I guess I have to write more on this.
Dr. Wang,
Yes more needs to be voiced about the illegitimacy, of the republican congress. Nothing is being said about the fact the revenues/GDP are low (i recntly heard Jeremy Grantham a very succesfull boston fund manager speak about this). The narrative that an illegitimate congress is pushing an agenda that only helps their paymasters needs to be put forth.
Step 1 in putting forth the narrative is to clearly establish the illegitmacy of congress. As readers of this blog we need to point more opinion leaders your way.
This might amuse you
Lol, that is epic. Can you extend it with by adding Nassim Nicholas Taleb and this to your discourses? http://arxiv.org/abs/1212.0953
Lol at all Dr.Wangs Intrade trolls.
Intrade is no more.
I know this is outside your wheelhouse, Sam, but wondering about your thoughts on sequestration. It obviously has effects on your #1 job (and mine, too).