Review of The Signal and the Noise
In Science magazine, Ben Campbell and I have a review of Nate Silver’s book, The Signal and the Noise. Briefly…it was good for people wh...
Senate: 48 Dem | 52 Rep (range: 47-52)
Control: R+2.9% from toss-up
Generic polling: Tie 0.0%
Control: Tie 0.0%
Harris: 265 EV (239-292, R+0.3% from toss-up)
Moneyball states: President NV PA NC
Click any tracker for analytics and data
Today, I ask for your help with my laboratory research. It will only take a few minutes.
The concept of a “physics gene” or “poetry gene” might not be a joke. Intellectual predispositions are likely to be driven in part by genes – including those that drive autism, mood disorders, and substance abuse.
My students and I are building a tool to measure intellectual phenotypes. To participate, take our brief survey. Continue here…
First, I am required to get your informed consent, which is required by Princeton University for any person involved in a research study conducted by investigators at the University. Our study has been approved by the University’s Institutional Review Board for Human Subjects.
I understand that I will be asked to complete a questionnaire about my intellectual interests and family history of neuropsychiatric disorders. The researchers will use this information in the development of a standardized inventory of intellectual interests. I understand that I will not be directly compensated for this study, and that my answers will be made anonymous.
If you acknowledge the above, then please proceed to take the survey here.
Now that you’re back…are you curious about the rationale? Basically, it’s this. Many neuropsychiatric disorders are caused in part by combinations of genes. We suspect that diversity of normal minds is also genetically driven – that the blank slate we are ostensibly born with is not actually blank.
Once we have a tool to classify people’s interests, we can do a lot with genetic information (not yours!). Believe it or not, such a survey-based tool does not exist, as far as we know.
I’ll gladly discuss it further in comments. If you are impatient… To learn more, read this, this, and this.
Did the survey, and have some feedback:
1. All the questions on psychiatric and other conditions ask about whether people have been diagnosed with this or that. Considering how little awareness there was only some decades ago of many of the conditions mentioned (ADHD is the most obvious example on the list, but others too), you’ll be missing quite a bit of information there (which maybe you don’t need; I don’t know). People my parents’ age, even my own age, could easily have had ADHD and some of the other things out there without ever being diagnosed.
Also, people suffering from mental illness in the narrow sense often refuse to be diagnosed at all (I have a rather tragic story in my immediate family to tell about that), but then you didn’t list that sort of conditions at all, so maybe you’re not interested in them.
As for classifying levels of interest, well, having finished a Ph.D. in philosophy and having spent practically all of my adult life studying and researching in the humanities (and being very enthusiastic about it), I got an Interest in the Humanities score of 8, just one point above the average. That, in truth, doesn’t sound very likely to me.
I agree about the diagnoses (and left a far-too-long comment touching on that), though I also wanted to mention a few other reasons I know “undiagnosed” or untreated (i.e. people who acknowledge they’d probably be candidates for psychological help or who have a group of friends/family who independently reach that conclusion):
1. fear the impact of medication/counseling on their creativity/personality/core-self
2. look at the studies on the meds and see that it’s a barely-scientific crap-shoot where you’re best off with the placebo
3. have seen how people with mental health problems are treated when they seek help (involuntary commitment, lack of empathy, stigma, forced medication)
4. face stigma both from self and from others, including multiple people (with ADHD) who would go off prescribed medication while looking for jobs due to potential drug tests.
5. have been in years and years of counseling without benefit
6. don’t see mental illness as interfering with their life (even when it is clear to all around that it is)
7. see their mental illness as a logical response to an objective analysis of circumstances (for example, very depressed people who see their state as rational)
8. see their mental illness as proof of a specific status or inclusion in a specific group and see treatment as lessening these
9. lack of health insurance; fear diagnosis making health insurance harder to obtain; are unable to pay for either medication or therapy
10. tend to avoid medical treatment generally; are the “long-suffering” or stoic
11. believe it’s circumstantial or external
I thought the questions were oddly too narrow in many cases- science being defined mostly as computers or math, humanities being seemingly mostly philosophy or art or narrow questions: do you want to learn a foreign language- no, and not because languages are not interesting- linguistics is fascinating. I suggested that some questions be prefaced by if you had extra time on your hands, would you enjoy…x or y. Anyhow, my pal was a Classics and Biology double major, but is now a scientist, so put down scientist as major, and then was rewarded with a -1 for Humanities and only a 6 for science. I will be honest, I saw the results and then though maybe the point of the survey is to challenge people with quantification about their interests that is wrong and monitor their response to the assessment. I wonder if the most important answers are the comment field where people give all this evidence as to why the survey is wrong. They love to mess you about with these things!
I think this is really cutting edge. And timely.
I’m assuming you’re also accessing the phylogenomics & cognitive genomics (BGI) databases. As a high IQ aspie gamer I’m already being tarred with the Adam Lanza brush, even though there’s no correlation between either aspergers or gaming with violence.
So for the promise of anonymity my stats are yours.
For whatever a sample of one is worth. 🙂
Thank you for your participation. The clickthrough rate on this post is phenomenal at the moment: 42%. And we have >1000 answers by all routes combined (PEC, Facebook, Twitter). Keep it up!
Some notes. First, to remind you all: this survey is anonymous, and no identifying information is retained. When providing us with feedback, we encourage you to keep your comments general.
The score at the end is our first pass, and is largely for entertainment – a little Cosmo-quiz-style score as a small reward. It is by no means definitive. We will apply machine learning and/or clustering algorithms to look for patterns that correspond to known intellectual phenotypes (college major). In such an analysis, an answer on even one question might be informative.
If you have questions or suggestions, you may also contact us directly at the address(es) given in the survey itself.
You should not mix “like” and “dislike” in the questions. You will get a lot of respondents who get the scale reversed. I used to write software to create and analyze surveys and I know about this problem and yet I still had to switch one of my answers after double checking.
–bks
bks: Occasional reversal of the like/don’t like scale was done on purpose. It has to do with scale calibration. The unavoidable tradeoff is occasional confusion.
Interesting survey, though it seems very unlikely based on my answers to the survey that my favorite law school class was a seminar on Hegel and the philosophy of law.
Will you use Amazon Mechanical Turk to broaden your survey base? The Monkey Cage recently had an interesting post on AMT and research: http://themonkeycage.org/blog/2012/12/19/how-representative-are-amazon-mechanical-turk-workers/
Took the quiz. One thing that I thought was missing was a button for people who actually do fictional writing as an occupation. You had one for graphics design/artist…but not one for people who write full time…Not all writers do their work as a part time job.
Seems it would be a useful thing to have in a study that’s looking for traits in this type of study.
Just did the survey. By the way, hard science fiction still counts as a novel, right?
Also, my humanities distribution in college was poetry, which required less typing than essay writing.
I’m so, well, average 🙂
It just goes to show how crude our initial scoring system is. But with so many answers, we can do something more refined. Thank you!
Oh MAT.
None of Dr. Wangs commenters are ‘average’.
Just opening your mouth here is a pretty fierce selection gradient.
Not getting moderated or deleted is another one.
I’m just a little above average in all three categories. I feel like I should have grown up in Lake Wobegon.
Just took the survey. As a physicist, I am obviously very interested in science. I therefore found it rather odd that my interest in science rates at only 2, below the average score of 4. I don’t have the impression that my answers reflected that…
Pat, real scientists answer the questions honestly. For example, no one really thinks about tough problems for hours on end. We think about tough problems for minutes on end, then think about something else, then return to the problem. Also, the quiz is problematic for people over, say, 50 (i.e. me) because the answers are very different now than then were when I was 25. I used to read fiction voraciously. Now I wouldn’t think of opening a novel (I can barely keep up with the journals) and l no longer do puzzles though I used to seek them out.
–bks
Some of us read unskewed polls and other such sites during the election cycle for entertainment purposes. Where do they fit in? 😉
Unskewed Polls? Poetry perhaps, possibly also a kind of puzzle (not logical, though), but most likely it would be classified as drama (specifically, the Theater of the Absurd). Alas, I recall no questions about drama in Sam’s survey.
I think you have some pretty severe selection problems given your readership, especially after the election.
I have ADHD. The hyperactive symptoms are present but the inattentive ones are quite strong. I put 0 for diagnosed family members, which is factually accurate, but my dad pretty clearly has ADHD as well.
“Scored” a 9/6/15, which seems about right, though I would have guessed the first number would be higher based on the description. I’m extremely curious and nerdy and, like many with ADHD, I tend to become obsessed with new topics and fields but then my interest fades quite a bit at or just past an intermediate level.
on the lists of health and psychiatric issues, i think you should have included psychopathy and schizophrenia. both are easily common enough to give you statistically significant samples. also, on the jobs list you did not include retired. in my case, i had to write in ‘retired/disabled’ because that is my current status. perhaps i am just overliteral, but it might have been more informative if i had been able to enter my former occupation.
anatman: Schizophrenia, which is among the most common of neuropsychiatric disorders, is on the list.
I scored 9 on science, 17 in humanities and 13 on, what was it, intellectual something. It is nice to see my love of the humanities reflected (I love poetry, play the cello, the guitar and sing, I speak three languages and I read a lot of fiction), however I am a pretty successful scientist (I have a PhD in chemistry, wrote a textbook and I run the X-ray Diffraction Facility for MIT) with well over 100 publications and an H-score of 23. Go figure…
Mueller: It is very important for our training set to have outliers. Thank you.
By the way, the very fact that you know your h-index is telling.
Surprised you don’t collect age, gender, and income (broadly) as the most important confounders, sources of bias, and likely stratifers.
Why aren’t there any questions about gaming or music?
congrats on the wonky award at wapo!
Completed, and shared. Hoping that it helps broaden the pool.
I have written a lot of fiction and sold most of it professionally, so I could be called a professional writer, although not one who made a living off it and nothing else. This category of occupation is not on your list.
I live off investments that I make in the stock market. There is no category here either.
Finally, as someone else mentioned in a Comment, time changes our attitudes. Twenty-five years ago, or forty, I would have answered that I read fiction voraciously, ie, at least one book a day. Today, I hardly read one novel a year. And if I do, it is probably a novel that I have previously read five times….
I love your site, Dr. Wang.
Interestingly, I scored below average in the humanities category. I think this due to liking to do things rather then watching others do things. We love making art around my house – painting, photography, print making, making up stories, reading poetry to each other, etc. but I hate going to museums, galleries, readings, etc. Even when I was a BFA student I hated critique and after a semester of MFA work, I was close to throttling my fellow artists. I spent a career at the back end of the photography and printing business – helping visual artists and commercial photographers look their best, as opposed to showing my own work. 35 years on, I still love taking pictures (currently I’m doing wet plate and salted print photography), but hate talking about the pictures. I do love to talk about process, though! 🙂
Designing such a survey is an interesting exercise. I wonder about the validity of self-evaluated levels of interest; for example, I’m very fond of music, I answered that with a 5 (or whatever the far right button was). My friend Susan is very fond of music, she also would answer that with a 5. But Susan has made a good bit of her income as a professional musician. I assure you no one has ever paid me to whistle, which is all I can do.
Economists have the idea of the Doctrine of Revealed Preferences. Assuming you trust people to answer honestly, it might be useful in the future to ask questions about how we *spend* scarce resources such as money or time (on music, fiction, puzzles, art, math, theater, etc) and how we *make* money (same list; write fiction? Sell art? Perform music? Research science?). That would differentiate music 5’s of my sort from music 5’s of Susan’s sort.
Is there some reason that my earlier comment hasn’t been moderated?
Probably.
I was surprised at the lack of OCD on the list of disorders. I was also surprised at the lack of questions about more removed relatives, as well as the lack of questions about the size of the family. I have a very small immediate family but large extended one – with one more degree of removal the number of disorders goes way up.
Thank you regarding OCD.
At farther removes, the fraction of shared DNA decreases from one-half. From a biological standpoint that does not help us.