Live thread on Election 2022
Open thread for those who celebrate. 11:03pm: New Hampshire Senate's called for Maggie Hassan (D-incumbent). Her margin appears to be somethin...
Analyzing Iran 2009: Part 1, Pre-election polls
Updated with a third possible reason for the discrepancy. -Sam
At a minimum, hundreds of statistically minded people are poring over Iranian election data (see my previous post and these posts). Sorry for my slow start…but let’s roll.
Three general categories of data are currently available for validating the Iranian election: (1) pre-election polls, (2) statistical methods for analyzing standalone voting data, and (3) statistical comparisons with past elections. Categories (2) and (3) are out on the web already, and I’ll comment soon on those efforts.
First, let’s look at some polls. A simple look at pre-election polls leads to the following assessment: National Iranian polls were highly variable and of suspect quality. But within Tehran, polls were more uniform and allow a comparison. Six Tehran polls gave a median lead for Moussavi by 4%. This differs notably from the official tally for the city, Ahmadinejad by 12%. The 16-point discrepancy suggests an anomaly in Tehran and opens the question of whether fraud occurred here – and elsewhere. However, it is also important to note several caveats, including polling uncertainty and possible shifts in opinion following the Ahmadinejad-Moussavi debate on June 3rd.
More after the jump.
In the US Presidential elections of 2000, 2004, and 2008, election-eve opinion polls were remarkably accurate predictors of the electoral outcome. In each case, aggregated, unadjusted polling data successfully identify key states (Florida, Ohio, and Pennsylvania) and came quite close to the final outcome. It’s a testament to the power of polls.
The Iranian election presents a harder case. Polling is sparse, professional standards of reporting polls are absent, and respondents are potentially unwilling to answer questions or hard to reach. Still, let’s look at the publicly available polls.
National polls are all over the place, even if we only take data after the Ahmadinejad-Moussavi debate on June 3rd, potentially a major decision point for Iranians. One post-June 3rd poll shows Ahmadinejad +16% (47% to 31%). The other shows Moussavi +32% (23% to 54-57%). Earlier polls range from Ahmadinejad +33% to Moussavi +30%. These data are so variable that they are unusable.
Polls within Tehran may be a better source of data. This is plausible because urban areas are easier to sample. The last three Tehran surveys (before June 8th, before June 7th, and June 3rd) show Moussavi +4%, Ahmadinejad +8%, and Moussavi +17%. Before that, three surveys showed Moussavi +12%, +4%, and +2% (May 27, May 26, May 14). The averages are
Last 3 polls: Moussavi +4 +/- 7% (median +/- MAD-based SEM).
All 6 polls: Moussavi +4 +/- 4%.
The announced official result was Ahmadinejad +12% (51.6% to 39.4%), a discrepancy of 16 points. When all 6 polls are used, this discrepancy is highly significant (p=0.003).
For now, my interpretation is that the official returns in Tehran are unbelievable. However, I can think of two three alternate explanations.
(1) Ahmadinejad really mopped the floor with Moussavi in the debate. The experience in U.S. elections is that debates provide a side-by-side comparison that can shift opinion substantially (for a famous example see Carter-Reagan 1980). In the case of Iran 2009 there are only 2 or 3 post-debate polls. A comparison using just 3 polls does not quite reach statistical significance (p=0.07).
(2) Tehran polls have a systematic overall pro-Moussavi bias that prevents a direct comparison with vote counts. For example, as David Shor points out in comments, polls might have been restricted to the actual city of Tehran, which is not all of Tehran province.
(3) Last-minute minor party candidates broke in favor of Ahmadinejad. There is plenty of precedent for third-party (and in this case, fourth-party) candidates to revert to one of the major candidates. U.S. readers, think of Nader supporters in the last two elections, who underperformed the opinion polls. And then there are undecided voters, who usually break against the incumbent in the U.S. but it’s not clear what would happen here.
I should emphasize that Tehran is not representative of the entire nation. It is notably more pro-Moussavi, which can account in part for the public anger there. In fact, if the 16-point discrepancy were corrected nationwide it would still not be enough to alter the overall outcome.
Iranians and other knowledgeable observers, please comment.
Topics:
It could be that we are dealing with different definitions of Tehran. The polls could be referring to Tehran, Tehran(تهران ), where Mousavi supposedly got 52.65% of the vote. Kind of like the difference between Tokyo(A large province that includes far off islands 1000 km away), and central Tokyo(the 23 wards that used to make up the city of Tokyo).
If they were just polling Tehran, Tehran, that would explain the numerical discrepancy. Luckily, most Iranian polls were done by think-tanks that released hefty PDFs that might explain these things. If not, a quick email might sort things out.
Not to spam, but I’ve posted some commentary on this post at http://stochasticdemocracy.blogspot.com/2009/06/more-iran-coverage.html .
If I can ask a dumber statistical question, how would one construct confidence intervals using MAD-SEM assuming normally distributed data?
I’d assume that a 95% confidence interval would be 1.96*σ~1.96*(1.48*MAD), but it’s early in this time zone, and the intervals seem too wide to pass a sanity test.
Thanks very much for your note. Of course I’m not a student of Statisitcs, but I want to mention another point which shows why official results in Tehran are unbelievable. In Tehran Monday’s protest, there were at least 1,500,000 people in streets. (Various agencies estimated the population between 1.5m to 3.5m). I should mention that many people did not come to Azadi st. because they were afraid of police and the MOI emphasized that we are not permitted to come to streets. On the other hand, the announced that Mousavi has 2,100,000 votes in Tehran. It’s really funny! When at least 1.5m people came to streets, there is no doubt that he had much more than 2.1m votes. Don’t forget that they killed 8 people in Azadi because they did not want to see these huge population again.
I should also include two points.
Firstly, would you please tell me your idea about this pre election polling?
http://www.terrorfreetomorrow.org/upimagestft/TFT%20Iran%20Survey%20Report%200609.pdf
secondly, it’s very important to notice that most polling expect 28m to 35m of people to vote in recent election. However, around 40m vote in this election which, in my viewpoint, was a strong reason why people vote for mousavi (It’s clear that when we want a reformer, we vote more). I cannot believe that 26 percent more than 2005 election vote, because they want Ahmadinejaad.
Mohammad (3:41 am), that’s an interesting point. But to be honest, I don’t necessarily think your observation proves the point. One problem is that a crowd that large is extremely hard to estimate. If Tehran was approximately split between Ahmadinejad and Moussavi, Moussavi supporters would feel numerous – and therefore suspect fraud. However, I agree in general that the protests will end up telling a story that these polls do not.
Mohammad (4:05 am), That appears to be a good survey; its numbers for issues such as nuclear weapons and Hamas/Hezbollah are illuminating for U.S. readers. Its bar graphs have errors so they should be ignored.
But for our purposes here there is a problem. It reports many undecided voters in the presidential race. In US polling, undecideds tend to vote against the incumbent (Ahmadinejad in this case). Combined with the survey it would suggest a fairly close race. But this is not guaranteed.
In regard to turnout, I note that only about 21m voted in 2005. What was the general feeling that year?
David, follow the link to find a formula, SD_estimated = (median absolute deviation)*1.483. After that I believe SEM can be calculated in the usual way. In this case the n=3 confidence interval is large because of the one outlier showing Ahmadinejad ahead.
[…] Three general categories of data are currently available for validating the Iranian election: (1) pre-election polls, (2) statistical methods for analyzing standalone voting data, and (3) statistical comparisons with past elections. Categories (2) and (3) are out on the web already, and I’ll comment Read more at http://election.princeton.edu/2009/06/18/analyzing-iran-2009-part-1-pre-election-polls/ […]
[…] Three general categories of data are currently available for validating the Iranian election: (1) pre-election polls, (2) statistical methods for analyzing standalone voting data, and (3) statistical comparisons with past elections. Categories (2) and (3) are out on the web already, and I’ll comment Read more at http://election.princeton.edu/2009/06/18/analyzing-iran-2009-part-1-pre-election-polls/ […]
Firstly, we should notice that in 1.5 – 3.5m protest of Azadi st, people were really afraid of being arrested or killed. The police – two hours before the protest – came to TV and asked everyone stay in their home! Hence, 1.5m people proves that mousavi had much more vote in Tehran (he claimed 4m which is rational I think!)
On the other hand, two years ago, people were not satisfied with reformers (Khatami was the president those years and people were angry about some of his actions) Hence, when Hashemi Rafsanjani (a reformist) was against Ahmadinejaad, people did not tend to vote, and another important fact is that Hashemi was not popular in Iran and a lot of people vote for Ahmadinejaad because they hated Hashemi!
[…] The Iranian election presents a harder case. Polling is sparse, professional standards of reporting polls are absent, and respondents are potentiallRead more at http://election.princeton.edu/2009/06/18/analyzing-iran-2009-part-1-pre-election-polls/ […]
[…] vorherige Umfrage finden. Zum anderen gibt es eine interessante Untersuchung der Princeton Universität just zu diesen Umfragen, die gehörige Zweifel anmelden. Perthes indessen hat sie 1:1 für seine […]
Iran should be examined before I actually countries in the world is China. and the fast-growing economy after the revolution mao evolving structure with the whole world began to keep their country under the economic mortgages. hand and the cheap production of the high amount of American securities offered to the world market with their products, especially in developing countries that significantly threaten their industry.
It is true supporters of both Ahmadinejad and Mousavi were demonstrating actively in downtown Tehran before and after the election, to say the people demonstrating after the election were (all) Mousavi supporters would be very unlikely. That 2009 was Irans first Presidential Debate and first electronic uploading of election results is significant and important, that a very large number voted also significant. That provision of monitors for each candidate at voting stations is significant, being that no voting irregularity would have happened. That perhaps 200,000 college aged kids were hired to enter the vote-station tally into the mainframe is significant, as these would likely be from wealthier families, these being predominantly Mousavi supporters would not be the point voting would be rigged for Ahmadinejad. As already mentioned, Ahmadinejad did significantly better during the Presidential Debates. Mousavi rejected request to submit writen charges and any proof of vote rigging he implied he had spoils his immediate charges before full taslly was in. Rigging of voting would have been almost impossible without many witnesses considering the involved security messures used at vote stations, had vote fixing occured in such large numbers as charged, many witnesses would have noticed and come forward. Learning of Mousavi as Foreign Minister involved with terrorists, leaves me doubting most of this situation since the US-Gov being heavily involved with questionable agencies such as NED proven to have been active in Iran.