The mailbag
Gerrymandering makes for interesting mail! Here are some excerpts from activists, a journalist, political scientist, and a few redistricters. ...
Gerrymandering theorems
To me, today’s news that Gallup is sitting out the primaries, and maybe even the general election, is not all that notable. The primaries are a hard-to-poll question; any race with more than two candidates seems to have issues (see the UK and Israel as examples). And the general election? It’s such a well-populated space, so there’s not that much publicity value in polling it. Besides, Gallup says they’re tweaking their methods, which is a good thing for them to be doing after their performance in 2012.
Also, I am considerably more preoccupied with gerrymandering and redistricting. I am developing statistical approaches for detecting a partisan gerrymander that can be used as a standard to be used by federal courts. This builds on work I published in the New York Times and here at PEC. I have done new calculations demonstrating that in the House, the likely total nationwide effect of gerrymandering is larger than the effect of other factors, including the clustering of Democrats in populated areas. So if a standard (mine, or anyone’s) is adopted by courts, the resulting reform can have a rather large effect on fairness of representation.
For law aficionados, the goal is to come up with a manageable standard for partisan gerrymandering, which is a justiciable question due to Davis v. Bandemer in 1986, followed by Vieth v. Jubelirer in 2004. I will be writing about the math of this question in the weeks and months ahead.
In addition to the legal battle, as an offshoot this project leads to a need for what I’ll call “gerrymandering theorems”: ways to derive the relationship between voting and seats from a small number of assumptions. For that, I’m looking for someone who is good with probability distributions, the Central Limit Theorem, and stuff like that. That person would ideally be not far from Princeton. Any takers?
Update: I have now spelled out the problem in the comment section. If I don’t find a proof, I will just end up graphing the outputs of the simulations I did previously. These days people don’t care so much about rigor. It just feels like it should have a closed-form solution, and I hate the thought of somebody pointing that out in the future!
I certainly know the stuff, but nearby only by airplane metric!
This is an EXCELLENT idea! Courts have needed better metrics. I foresee issues in terms of size of sample: if Colorado has 8 districts, it’s hard to get a neat statistical fit.
This has a legal “impact” nature to it: as you know (and I assume your motive here), courts have essentially waved away partisanship as a measure. Providing a mathematical metric would bring the issue further to the fore in courtrooms, by increasing the profile of the issue, since judges are just as prone as everyone else to paying attention to issues that have impact and glossing over issues that they can ignore.
A good starting place might be trying to find out what has been done before, other than ratios of partisan registration.
Thanks. Indeed that is the point. Specifically, the goal is to come up with what has been termed a “manageable standard” in Vieth v. Jubelirer. Generally, I am familiar with both the legal precedents and the prior mathematical work. I am now at the point of wanting to convert my analysis into amicus briefs and, I hope, law reviews.
Here I’m looking a step ahead to another branch of the project – basically a math project. It’s a bit far afield of the important point of fixing a broken aspect of our government – as you point out.
Awesome project! I don’t think you’ll have a hard time finding ideal help. I would also love to contribute any way that I can (given my limited time schedule).
(p.s. it looks like the link to your 2013 work is broken)
Ok, sounds interesting. What do you need? (Don’t want to learn Matlab but I have picked up python in the last few years).
Amit, sharpen your pencil! You will not need to write any code, unless it is to run a toy simulation to build your intuitions.
(Note that the following question may change wording. It is in beta at the moment. Current version Thursday 11:25 pm.)
Fantasy Congressional delegations. Consider a state composed of N districts with equal population, where Democratic candidates together receive a fraction of the total vote V0, where V0 is in [0,1]. The goal is to construct fantasy delegations of N seats with the same vote share, and see how the number of Democratic seats is distributed.
Construct these fantasy delegations from districts with vote share v_i, where v_i are the vote shares from all 435 districts nationwide. Collectively, this national distribution of v_i’s has mean vote share mu, median M, standard deviation sigma. (Note that the national distribution is not quite normal, and includes delta functions at 0 and 1. If you wish to describe the distribution using other parameters, that is OK.)
Now draw N districts at random from the national distribution, with replacement (i.e. you can use a district multiple times). A district election is won by whichever candidate gets a vote share greater than 0.5. The resulting delegation has a some total average vote share V, and a number of Democratic seats S. For a specific vote share V0 (i.e. you only accept delegations that ended up with this specific vote share), what is the mean of S? What is its standard deviation?
Extra credit: Estimate how quickly the distribution of S approaches the asymptotic distribution, whether that distribution is normal or otherwise. (Is it normal?)
Just to clarify, say v_i is the vote share for district i, and = mu, and – ^2 = sigma^2. We still need to assume some PDFs for the individual v_i. Unless they are all delta functions a square PDF should be ok (?)
Then we pick N from this sample (with replacement) and calculate for this subsample. And S = sum(v_sub_i > 0.5). I do not quite understand what you are asking in the question that begins “For a given vote share V0 …”. Plot of S vs ?
Amitabh, are you familiar with Sage math? It is built on python. I go through some of the tutorials every now and again to try and keep what little of my math skills that remain.
http://www.sagemath.org/
It’s not clear to me here what exact probability we’re looking for. I would assume that we’re looking for the probability that the party breakdown of a state’s delegation would have happened for a randomly chosen districting? Except that neglects incumbency, so perhaps what we’re looking for is the probability that the _expected_ party breakdown of the current districting would occur for a random districting?
For example, if there are ten districts in a state with a 50/50 party breakdown, you’d expect a fair districting to produce an expectation of five of each party, give or take.
That doesn’t preclude “safe” seats. You could accomplish this by concentrating all the voters of one party in five districts and all of the other in the remaining districts. It’s not clear that this is better for the body politic than having ten evenly-split districts, but just looking at the overall breakdown of the delegation seems like the simplest thing that could possibly work.
In any case, we at least want to preclude the classic gerrymander in which, say, district 1 is 80/20 for one party while districts 2-10 are (approx.) 52/48 for the other. This is bad because winning expectation is a highly non-linear function of voter percentage. Having a 52/48 advantage doesn’t mean a 52% chance of winning, and therefore an expected .52 of a representative. I don’t know the exact numbers (you’ll need to, though!), but say this translates to an 80% chance of winning, while an 80/20 split translates to a 99%+ chance of winning. Then this districting means an expected 7.2/2.8 split in the delegation that should be 5/5. It should be possible to show that a such an expected split is highly unlikely.
So, if I’ve got this right, you want to do two things:
1) Calculate the expected split from the current districting, neglecting incumbency, using a function to translate the voter split into a win probability. This function could be calculated up front (and thus argued over endlessly by interested parties), and might or might not take into account what portion are registered independent, what portion are registered overall and how much the proportion of independents is affected by such things as whether the state has open primaries. It might be tempting just to look at the results over the last N elections, but I doubt that would give you enough data points, and we need such a function anyway for step 2.
2) Calculate the distribution of expected party representation in the delegation across all possible districtings. That is, how likely is a 50/50 split, or a 72/28 split and so forth. Since we’re dealing in expected values it’s fine to talk about fractional representatives (e.g., 7.2 representatives in the example above), but expect ridicule from anyone opposing this socialist plot to take away our liberties. Also expect wrangling about whether we’re looking at all possible partitions of the precincts or just ones that produce contiguous districts. Intuitively, it ought not to matter much, and the former is much easier to calculate, but expect further ridicule. Much fun to be had.
Assuming this is all more-or-less correct, it should be fine to use Monte Carlo techniques for step 2, which I believe is what you’re suggesting:
* Find the average district size
* A large number of times:
** Pick precincts at random until you’re close enough to the average district size
** Apply the win percentage function to the voter split in the resulting district, getting a number between 0 and 1
This should give you a distribution, which you can the use to vet whether the current districting is fair. For example, my guess is that for a state split 50/50, the .72 above is going to be far out on the tail of the distribution, even if the individual precincts are heavily partisan.
We’d want some sort of justification that it’s OK to average over districts instead of over partitions of the state into districts, and to allow non-contiguous districts. I can’t think of obvious cases where those approximations would fail, but I’m not a statistician
Actually, I’m pretty sure I’m _not_ talking about quite the same approach you’re taking (having read a little more), but it seems to me that the key points are
1) The non-linearity — 52% of voters does not mean .52 expected representatives
2) The quantity to measure is how likely could the current expected delegation have come about by chance.
Number of districts matters. Wyoming’s delegation is going to be 100% Republican no matter what. Anomalies are only meaningful in the larger states.
Finally, thinking it through, the Monte Carlo probably needs to be over random contiguous partitions, not random subsets of the precincts. The challenge then is to generate fairly distributed partitions off the vertex-weighted adjacency graph of precincts. That has a number of implications, particularly if you require that the partitions meet further criteria, such as compactness.
Sorry – obviously my original post was too brief. I am not looking for all of this basic stuff – I will write about that in the future. In a sense I am going off half-cocked with the advanced question, and saving the basic explaining for later. For this reason I am moderating obvious comments pretty heavily. I’m letting through ones that move the conversation forward.
Basically agree with most of your points, though toward the end you veer off a bit into Monte Carlo territory, a wrong direction from my point of view.
tl;dr for those who have not read my paper:
Seats-votes curves follow a nonlinear function. Knowing that function is necessary to understand when a deviation has occurred. That is the subject of my Prong #1, in which I calculate what a nongerrymandered seats-votes curve should look like. It looks a lot like a cumulative normal distribution. Numerical simulations the curve using resampling methods are not that hard (though I would be interested in anyone who could find a weakness in the assumptions). Beyond numerical simulations, I am looking for a way to derive those distributions in “closed form,” i.e. with integrals, distributions, and the Central Limit Theorem.
Anyway, no matter how one calculates that nonlinear function, once one has it, further statistics can be used to identify when a substantial deviation has occurred. Such deviations are less likely in small states – but small states are also harder to gerrymander, for more or less the same reason. Practically speaking, there is no need to analyze states with fewer than about 7 districts. Therefore the case of Wyoming is not of direct relevance.
(I think my original comment is still “awaiting moderation”, which may make this thread a bit less comprehensible to most readers)
So yeah, just finished reading the paper. The approach I described, with contiguous districts, looks very much like some of the other approaches that the courts didn’t like. All of them would fall under your first prong: One way or another calculate a distribution of expected outcomes (pretending that incumbents have no advantage in House races) and then see if the actual result is “too unlikely”, e.g., more than a sigma away from the mean.
One way or another you’ll end up using Monte Carlo methods, whether the Monte Carlo method you chose (picking sets of districts from around the country), or using random contiguous partitions, or random sampling of the precincts in the states without regard to contiguity or full partitioning.
My guess would be that it doesn’t much matter which one you choose, and you could probably put together a generic table once and for all with number of seats on one axis, vote margin on the other and in each cell an expected range of “not-suspicious” outcomes. For seats = 1, that’s always “exactly one”, which is just to say that a single-seat delegation is a degenerate case — my original point and apparently yours too.
FWIW, I think your way of populating that table (more or less) is a nice one. As a practical matter, go with whatever method the judges like, no? Unless someone can prove it makes a difference, anything along those lines, together with the other two prongs, ought to be enough to establish a prima facie case.
Combining that with two other prongs as a sanity check is a good idea as well, though as you point out the three are liable to be fairly strongly correlated. It would be interesting to detail under what circumstances they would disagree.
You have to admit that your other comment was ginormous in length. 😉
I developed analysis #1 to address all SCOTUS concerns expressed in the Vieth and other opinions. It could be that I am reading their opinions too optimistically. Did you read my summary of previous objections?
This discussion and others have made me realize that my analysis #2 and #3 probably escape the previous problems better, simply by being more transparent. I cannot shake the feeling that the methods to date have been either too complicated for enough Justices to trust, or have not taken into account the “zone of ambiguity” I expressed in my paper on SSRN. Justice Scalia has specifically dinged the “>50% of votes electing <50% of seats" criterion as being subject to noise, though he put it slightly differently.
More broadly, I have this crazy idea that judges might like tests they can do on the back of an envelope whilst the expert testimony is presented.
I guess it boils down to whether we need the “too unlikely” prong. My personal feeling as a geek is that that’s what we’re _really_ measuring, so it should stay in.
I suppose the two ways to do it are 1) just to say that explicitly and leave it up to the courts to decide what “too unlikely” means, and 2) give a formula, however it’s derived.
Option 1 seems more like how the courts think, but the feeling seems to be that there’s no good recipe for any of this, and for gauging unlikeliness in particular. Which is what you’re trying to address.
With that in mind, maybe a table like what I described, with some academic muscle behind it, would do: “Here’s this formula our geeks gave us to tell if a result is unlikely. We can use it in conjunction with the other two tests.”
I think your fantasy football method is elegant and more accurate, especially since it uses real districts across the US as a baseline. Unfortunately, it probably skirts the hairy edge of what the “intelligent layman” can be expected to grok.
I have no idea if any of this will fly. I definitely believe that it’s a worthwhile effort to try to bridge the gap between statistical understanding and the law as you’re doing. If not this particular formulation, maybe it’ll at least throw some weight on the “this could be feasible after all” side, especially since you explicitly give a well-defined standard (N sigmas) to address Scalia ,who has a good point in this case at least.
If any of this happens, lawyers will be arguing over it. That much I’m sure of.
JR Mole’s points are well taken. Re the likelihood of a chance result for a current delegation, that’s more than a math question. Or maybe it’s a question of understanding which Bayesian prior to apply.
In the world of local politics, nothing is left to chance, and every possible electoral advantage is pursued.
A better question is which rules should be applied to constrain – not to eliminate but to keep within some reasonable limit consistent with principles of democratic representation – the inevitable and normal tendency of competing political parties to gain maximum electoral benefit.
The Supreme Court has stated that there is some limit, and has more or less made a request for a definition of that limit. If implemented, the limit would debug this aspect of single-member districting, which is how the House is elected. It would be useful in all single-member-district systems.
All I want to say is, thank you SO much for doing this work. It is so important!
I know nothing of the underlying law, but a very simple approach would be to put a hard limit on the tortuosity of district borders. For example, require that the ratio of perimeter (squared) to the area of districts be less than some constant.