Princeton Election Consortium

A first draft of electoral history. Since 2004

Weaknesses of InTrade – and opportunities

July 11th, 2012, 9:29am by Sam Wang


Failures of InTrade to accurately estimate probabilities are frequent. The reasons are obvious, but perhaps unappreciated by market-worshippers. Some of those reasons lead to inefficiencies…which could be exploited.

InTrade uses group betting behavior to come up with a wisdom-of-crowds estimate of various probabilities: sporting event outcomes, political races, and all kinds of events. Recently the market completely failed to predict SCOTUS Chief Justice John Roberts’s ACA ruling. This example reveals one big reason for market failure: when bettors lack  relevant expertise and/or information. In this case, a reasonable probability for upholding the ACA might have been drawn from Constituional law scholars: 8/21 = 38%.

Even when lots of data are available, such as political polls,  InTrade can still fail. One simple reason is bias: InTrade bettors appear to skew Republican. This could explain why there is such a mismatch between the poll-based Obama win probability (>99% for an election today, probably >80% in November) and the InTrade price (equivalent to a probability of about 0.56). This could be excused on the grounds that the election is far off, and there is uncertainty as to what will happen in the next 4 months. However, there is a third flaw.

As I’ve written before, InTrade bettors are habitually underconfident in the face of polling data, even on the eve of an election. Even a 10-point lead in a race is insufficient to drive a market-based probability estimate above 80%. This is perplexing since such a lead is basically a sure thing.

I am currently analyzing some of the biggest weaknesses. A second point is how to allocate resources. For that the Kelly criterion seems to be useful.

More later.

Tags: 2012 Election · Uncategorized

8 Comments so far ↓

  • Sean

    For much of ’08, Obama’s numbers on Intrade were similar to what they are today. In fact, I Internet Archived the site and found, on July 12th, he was at 57.5% probability of winning the presidency. So, if you take their current numbers into account (55.7), he’s essentially only two-points behind his place four years ago.

    I think what skews it more is the human element. People are going to have biases, but also are going to be influenced by the overall noise of the media, which isn’t the case when you’re dealing with a formula to come up with a certain probability.

    It wasn’t until late in the campaign that Obama’s numbers pushed into the 70s, even though I’m assuming a great deal of formulas gave Obama a far greater chance of winning the election.

    But it’s not a surprise. People are going to bet based on outcomes that don’t necessarily impact an entire arc of the campaign.

    Back at the beginning of summer, Obama’s numbers consistently hit the 58-60% range and then plummeted in early June on the release of the dismal May jobs report. His number sank to 52%, his lowest point on Intrade since mid-January.

    No one knows what impact jobs reports will have on Obama’s reelection chances and while we are left to speculate if his numbers will dip on bad jobs reports and rise on good ones, in the end, it really has proven to be of little impact. However, back in June, the numbers cast a narrative out there that the economy was not only stagnating, but maybe getting worse.

    However, that dip seen at the beginning of June was solely on speculation that it would negatively impact his reelection chances. In reality, on June 2nd, the day after the jobs report came out, Obama was + 1.6 on RCP’s polling average. Today, over a month and two job reports later, Obama is +1.9 (as of this post) on their average.

    As it became clear Obama’s numbers weren’t taking near the hit as expected, Obama’s stock on Intrade continued to improve. It finally pushed beyond 54% around the time the Supreme Court issued its ruling on Obamacare.

    It’s stayed between 55-56% since.

    So, the market is driven mostly by speculation and in-the-now events.

    But since they’re human, and only going on human instinct, of course it’s not a surprise that they would be overly cautious on some things.

    AS for the Supreme Court, again, you have Intrade going with conventional wisdom. The problem with the healthcare ruling was that no one knew, either way, what direction the court was going to go. It’s not a presidential election where you’ve got a dozen or so polls to indicate where things might be going. That whole thing was solely opinion based and that opinion was driven by the idea that since the mandate was ripped to shreds during the arguments, the court would eventually strike it down. They didn’t, of course, but it was a decision that even surprised a great deal of legal experts (though, I should note that some, including SCOTUSBlog, got it right).

    Intrade is like that Gallup poll asking Americans who they think will win the presidency. Nothing more. But, more times than not, I’m guessing the candidate Americans think will be their next president ends up actually being their next president.

  • Sam Wang

    Sean, that is all very interesting. Thank you.

    In the 2008 election, the only moment when the race was actually a toss-up was immediately after the Republican National Convention, when Sarah Palin had just been added to the ticket.

    If you care to, I’d love to see the InTrade history. It would be worth graphing against our probability and EV estimator measures from that time.

    Regarding the Affordable Care Act decision, certainly Lyle Denniston and others at SCOTUSblog were insightful before, during, and after the event. Denniston is someone to listen to in the future – perhaps even more so than Jeffrey Toobin. The problem is that it’s hard to know which constitutional expert to pay attention to. Therefore my estimate of 8/21.

  • ron bernstein

    “Failures of InTrade to accurately estimate probabilities are frequent. “….

    I am not sure you are describing an intrade “failure”

    see this: http://rcavanaugh.com/2012/07/how-not-to-interpret-intrade-results/

  • LondonYoung

    Wow, Kelly.
    If one knows with certainty the probability distribution of an uncertain event, and there is no Knightian uncertainty, then you are fine. But the real world is filled with Knightian uncertainty.
    The world has no friction-less ropes and no mass-less pulleys.
    Polling organizations pay a lot of attention to getting their polls right the day before the election (lest Nate Silver berate them for a republican bias) but what care they of their numbers 4 months out?
    When you look into your heart, what odds would you really lay on Obama’s November victory? Far from intrade’s?

  • Sam Wang

    It is true that polling organizations only have to be correct in the days before the election. However, they can be compared with one another.

    Gut feeling? I think the only good predictive data are where polls have been this season, to date. In past Presidential elections the SD has been 15-20 EV. Based on that (and I’ll formalize this in coming weeks) I think 4-1 in Obama’s favor.

  • LondonYoung

    Yeah, I can’t argue with your reasoning on how polls at this point (July) have typically done vs. the results in the election. Intrade is at 3:2 and it feels a bit light to me for the president.

    But I cannot hide my frustration that, for business and personal planning, I need to know the results as long in advance as possible. It is of approximately zero value to me to be told who will win on election eve because in just one day after that I will know for sure. There are big policy differences to be decided in this election, and being positioned the wrong way will be very costly indeed …

    • Sam Wang

      For purposes of true prediction, calculate the probability as follows. Take the EV history (or Meta-Margin) for July 2012 and average it. This gives about 310 EV (MM = Obama +3.0%). Then use the SD of the EV history from the 2004 and 2008 elections to create a prior distribution for 2012, assuming the median is shifted, but the shape of the distribution is the same. Then sample this for a November win probability. Based on this procedure the Obama win probability would be >95%. I’ll give a precise estimate soon.

      A caveat is in order. Is 2012 more like 2004 or 2008? In 2004, a close re-election fight, a histogram of the EV estimator was approximately Gaussian. In 2008, with two new candidates, the distribution had a long tail on the Democratic side. Economic collapse was probably an underlying cause. How does one estimate the long tail? In 2012, an October Palin-sized event would swing the election toward Romney — or increase Obama’s lead. Before I go live, I’d be interested in your take on how to estimate that probability.

      As for business/personal planning, I believe the real unknown is who will control the Senate (slight advantage to Democrats) and House (slight advantage to GOP). So…the odds currently favor divided government in 2013.

      Again, all thoughts are welcome. Once I get done with some urgent lab business I will turn to this.

  • LondonYoung

    Well, given the amount of time you have thought about this compared to me, I think I have “little to add” but “much to ask”. But you do jump very quickly to exactly what I most wanna know – odds of divided govt.

    Let’s say I believe that all of house, senate and presidential results are correlated in the final outcome vs. current indicators. For example, if the dems take the house, LondonYoung is close to certain they keep the senate and the White House. So, if (as you say) the GOP has a slight advantage in the house (55% win probability?), then I might say the odds of a dem sweep are not far short of 45% (=100-55). Is this fair?

    I guess information like “what is the correlation between the time evolution of polls in Ohio and Florida and Virginia?” means a lot to me. If the states are independent, then Romney is toast. I *do not* understand how you or others model this. (yes, yes, because I am lazy)

    However, I cannot emphasize enough just how little use I have for information about “if the election were held today” beyond how it helps me to estimate the actual election.