A New Critique of the Lancet Studies
David Kane of Harvard University has put forward a new critique of the Lancet studies on the excess deaths attributable to the invasion of Iraq and the subsequent conflict. The gist of the critique is pretty simple overall: when one includes all the relevant information the results of the Lancet studies, at least the first one, are statistically insignificant.
The only reason why the authors found a statistically significant result is that they threw out the data on Fallujah. One could argue that Fallujah is such an extreme outlier that throwing the result out makes sense. However, it is also reasonable to at least include results for all the data in the paper, which for the most part was not done. Further, when one has to throw out data to get a statistically significant result it is usually viewed with some skepticism since one is indeed not using all relevant information. That is, it is usually better to use all the information one has than to limit oneself to just a subset of the information.
Now one thing should be made clear, unlike some commenters, David Kane does not appear, at least in the article linked above, to be saying that things have gotten better. The message appears to be that we can’t tell if things have gotten better or worse using all of the date from the Lancet studies. Indeed, as Iraq Body Count has argued,
Do the American people need to believe that 600,000 Iraqis have been killed before they can turn to their leaders and say “enough is enough”? The number of certain civilian deaths that has been documented to a basic standard of corroboration by “passive surveillance methods” surely already provides all the necessary evidence to deem this invasion and occupation an utter failure at all levels.
Why so many defenders of the Lancet studies seem married to the results is beyond me. Pointing to a fraction of the numbers in the Lancet studies like 60,000 excess deaths speaks very, very poorly of the U.S.’s presence in Iraq and of the invasion overall.
Over at Deltoid, there is more discussion as well. The discussions there are very heavy with statistical concepts so consider yourself warned if you plunge in over there. But one thing did catch my eye,
3) Goodness knows that this discussion would be a lot more productive if the authors were to release their code. Elizabeth Johnson, the graduate student who actually did the calculations, does not respond to my e-mails or phone calls.–David Kane
This seems to be a disturbing trend with some of the scientific researchers who support causes espoused by the Left side of the political spectrum. Mann, et. al. have been very reluctant to pass on both data and code for their global warming research, and now we have the authors of the Lancet studies not sharing code? I think any research that is geared towards having some impact on public policy should also be put into the public domain. That means all code, data, and so forth.