November 19, 2006

Mortality and epidemiological methodology -- Most of us have heard of the 2004 Lancet study which had estimated mortality in Iraq following the start of the war; this has recently been updated with a new study (Full article in PDF). Following on the criticism and defence of the original study, This American Life dedicated most of a program, What's in a number? 2006 edition to talking about the study and the methodology (including an interview with one of the authors). (via homunculus, Wolof and the Monkeyfilter comment archives)

The question is, how many of us really understand cluster sampling and confidence intervals to wade our way through the debate?* This post does have a simple explanation on the difference between the cluster sampling and "passive surveillance" (as used by Iraq Body Count), as well as calling for critics of the study to respond with more research, less rhetoric. * If you do understand cluster sampling and confidence intervals, please feel free to come into the thread and lord it over the rest of us, so long as you then share your knowledge in simple words for the simple minded (aka me). All of these links have appeared in comments from homunculus and Wolof, but I felt that these very valuable posts were being lost among the old threads, and have brought them forward. ----------------------------- What I found most interesting in material presented by This American Life was the fact that this methodology has been used to estimate mortality elsewhere in the world without such criticism, and has even been used by some of the policy makers who now denounce the current study.

posted by jb 18 years ago
  • The question is, how many of us really understand cluster sampling and confidence intervals to wade our way through the debate? Not me, for damn sure. What I found most interesting in material presented by This American Life was the fact that this methodology has been used to estimate mortality elsewhere in the world without such criticism, and has even been used by some of the policy makers who now denounce the current study. We like to hear statistics that reinforce our already held beliefs. If the numbers had been more in-line with people's expectations, there wouldn't have been so much criticism. Exacerbating this already extant tendency is the fact that these results were released right before an election, when people are far more critical and questioning of information that seems to support or undermine an administration. Add to this the fact that the study included causes of death such as car accidents and disease in the umbrella of "war related", it's not surprising that most people had a hard time accepting its results. Reviewing their rationale for including these numbers, they make perfect sense, but the fact is that *I* certainly hadn't considered the full, far reaching scope of the effects of war in regards to mortality rates. Like many, I just figured it would come down to a list of how many were shot by this side, and how many were shot by the other side.
  • if it goes on long enough war eats everyone except the politicians whom we know will lie about the causes and the consequences and pretend that peace is somehow an enemy that death is somehow glorious and so what if those folk die the child of two's our enemy the invalid's our enemy that young man was an enemy and so it's right they die
  • During the This American Life segment, there is a discussion of why the original study (and presumably the followup) were released just before the American election. It was meant to have a political effect, and my first reaction was "that's so partisan". But then the segment went on, and it was clear that they hoped not to end a political party - they didn't seem to care about Rep/Dem - but to change or end the war, because they hate war.
  • Of course, their numbers include all who have died, of whom a large number were from violence. I assumed that any other count only includes those confirmed as killed by violence. And frankly, who is this Fred Kaplan person? a columnist? Why should I trust a columnist over a peer-reviewed article in a medical journal? Lancet is no North Rustico Journal of General Practice either, it's a highly respected journal. But Kaplan's rebuttal (to their explanation of methodology - death rates higher in Europe because of age structure, neat) seems to be citing his own column. And complaining that the sample wasn't random, but the neighbourhoods were randomly sampled, and even more this time. This is an established methodology. Is he trying to save face, by creating rhetorical holes in the argument? I think many people are swayed by such criticism, even when baseless, just because they don't really know understand what is happening.
  • I really think it's time mandyman explained this. Reason being is she knows the field, although there are doubtless a couple of others here. *summons the lovely Miss Amanda*