Iraq Body Count Looks at the Latest Lancet Study
A few days ago James posted about the latest Lancet study on the number of excess deaths due to the Iraq war. Now the Iraq Body Count has released some interesting comments about the study. Overall, the main conclusion seems to be that one should be cautious about the results of this new study.
However, our view is that there is considerable cause for scepticism regarding the estimates in the latest study, not least because of a very different conclusion reached by another random household survey, the ILCS, using a comparable method but a considerably better-distributed and much larger sample. This latter study gave a much lower estimate for violent deaths up until April 2004, despite that period being associated with the smallest number of observed deaths in the latest Lancet study.
Also, it is pointed out that the agreement between the latest Lancet study and the most recent is somewhat exaggerated.
Additionally, claims that the two Lancet studies confirm each other’s estimates are overstated. Both the violent and non-violent post-invasion death estimates are actually quite different in the two studies.
The press release also has five implications regarding the latest study that Iraq Body Count considers somewhat anomalous.
The first is as I pointed out the sheer magnitude of the numbers for the latter period in the study.
On average, a thousand Iraqis have been violently killed every single day in the first half of 2006, with less than a tenth of them being noticed by any public surveillance mechanisms.
[snip]
If the Lancet extrapolation is sound, this would imply a further 920 violent deaths every day (1000 minus 80) which have been recorded by neither officials nor the media. As these are averages, some days would see many more deaths, and others substantially fewer, but in either case, all of them would remain unnoticed.
If we consider the Lancet’s June 2005 — June 2006 period, whose violent toll it estimates at 330,000, then daily estimates become lower but would still require 768 unrecorded violent deaths for every 67 that are recorded. The IBC database shows that the average number of people killed in any one violent attack is five. Therefore it would require about 150 unreported, average-size, violent assaults per day to account for 768 deaths.
[snip]
One possible way of explaining such a very large number of small-scale unreported assaults is to suppose that many of these are the result of “secret” killings which have resulted from abduction, execution by gunfire, or beheading. But 42% of the 330,000 Lancet-estimated violent deaths in this final 13-month period are ascribed to “explosives/ordnance”, car bombs, or air strikes, all of which carry a fairly heavy and hardly ‘secret’ toll (and will generally create at least 3 times as many wounded).
[snip]
Lancet estimates 150 people to have died from car bombs alone, on average, every day during June 2005-June 2006. IBC’s database of deadly car bomb incidents shows they kill 7-8 people on average. Lancet’s estimate corresponds to about 20 car bombs per day, all but one or two of which fail to be reported by the media. Yet car bombs fall well within the earlier-mentioned category of incidents which average 6 unique reports on them.
This implication was the first thing that caught my attention. That large numbers of people would have to be killed and at the same time it would have to be virtually unreported.
The other implications are,
2. Some 800,000 or more Iraqis suffered blast wounds and other serious conflict-related injuries in the past two years, but less than a tenth of them received any kind of hospital treatment.
3. Over 7% of the entire adult male population of Iraq has already been killed in violence, with no less than 10% in the worst affected areas covering most of central Iraq.
4. Half a million death certificates were received by families which were never officially recorded as having been issued.
5. The Coalition has killed far more Iraqis in the last year than in earlier years containing the initial massive “shock and awe” invasion and the major assaults on Falluja.
To me all of these do not help the case of the Lancet study. When the official number of death certificates turns out to about 50,000 and the Lancet study is claiming 11 times that number some alarms should be going off somewhere about a possible problem with the data.
Now it is possible that all of these factors are true and that the Lancet study is quite correct in terms of its estimates. However, this suggests that there is extremely widespread corruption in Iraq when it comes to death certificates. Which leads us to wonder what value producing a death certificate is when it comes to studies like the Lancet study. That the media is ignoring or ignorant of the vast majority of attacks in Iraq.
And lest anyone think that Iraq Body Count is shilling for the hawks/stay the course crowd, consider this quote,
Do the American people need to believe that 600,000 Iraqis have been killed before they can turn to their leaders and say “enough is enough”? The number of certain civilian deaths that has been documented to a basic standard of corroboration by “passive surveillance methods” surely already provides all the necessary evidence to deem this invasion and occupation an utter failure at all levels.
On 9/11 3,000 people were violently killed in attacks on the USA. Those events etched themselves into the soul of every American, and reverberated around the world. In December 2005 President George Bush acknowledged 30,000 known Iraqi violent deaths in a country one tenth the size of the USA. That is already a death toll 100 times greater in its impact on the Iraqi nation than 9/11 was on the USA. That there are more deaths that have not yet come to light is certain, but if a change in policy is needed, the catastrophic roll-call of the already known dead is more than ample justification for that change.
The bottomline is that one should be cautious about accepting the numbers in the Lancet study. Now, that doesn’t mean the everything in Iraq is fine or that the Bush Administration is handling the situation correctly.
The REAL bottom line is that we should be asking why the Defense Department is not keeping numbers? How in the hell can they gauge effectiveness of the war if they don’t know how many people are being killed by the fighting?
It is no wonder that the mission hasnt been accomplished yet!
Well, I cannot comment on this study directly, but I can say that statistics has been one the many things in life that continues to confuse me. In some of the work I do I have the need for statistical analysis and it seems that whenever I make an inquiry on how best to analyze data, I always seem to be offered several statistical tests to use. I feel this usually occurs despite my being rather specific about the hypothesis I wish to test. I am not sure that scientific papers are reviewed as as rigorously for their statistical analysis as they should be but that is just a guess. In the end, as one of my mentors told me, when all of the analysis is completed you should be able to look at it with an unbiased eye (ie. even if it does not support your hypothesis) and feel it just makes sense. Perhaps their statistical analysis in this paper is valid but their sampling is biased. It would be cool if a statistician who reads this blog could comment.
T-
How do you propose the DOD keep tabs on things that U.S. soldiers are not a part of?
Generally, the more someone knows about statistics, the more plausible they seem to find the Lancet study.
It’s obviously a rough figure, but it seems entirely plausible that, counting insurgents, half a million Iraqis have been killed.
See this interesting bit on the number of air strikes in Iraq.
Actually, the more people claim to know about statistics, the more plausible they seem to find the study. What it really comes down to is that people who have taken Intro to Stats 101 and know a few buzzwords find it easier to find better excuses as to why the Lancet study could be right.
On the other hand, people who know a bit more about applied statistics keep finding really major ways the Lancet study could have been manipulated by one or two people, and important reasons to not believe its results at all.
I already noted the “where are the wounded” problem in a blog post elsewhere… there should be a couple of million wounded from those attacks, with a couple of hundred thousand undergoing treatment in hospitals or clinics at any given time. Every hospital and emergency room in Iraq should be absolutely overwhelmed, 24/7, with the hordes of people undergoing treatment from that day’s gunshot and shrapnel wounds.
Every street scene you see in Baghdad should show dozens of people limping along with crutches, bandages, or the like, with a significant amount of people showing up with lost limbs or serious disfigurements.
The reality? Not so much.
The “could have been” is an honest quote that shows more about you than them. Of course, the stats could have been manipulated, same as the CPI or the unemployment rate or polling data or any other common statistical finding used by people. The question is then do you distrust all surveys, unless they reinforce your own hidden biases.
Just because the numbers don’t fit your concrete perception that the Iraq War has been a blessing for the Iraqi people, doesn’t mean that they aren’t true. And its obvious that despite a seemingly even-handed analysis, not done by some partisan hack, that the people who are spending their times trying to poke holes are the ones with the bias problem.
I already noted the “where are the wounded†problem in a blog post elsewhere… there should be a couple of million wounded from those attacks
Typical pulled-out-of-my-butt “analysis.”
Beheadings don’t typically leave a lot of wounded. And on your “theory,” the 800,000 dead in Rwanda should’ve been accompanied by millions of wounded … who aren’t there! so the Rwandan genocide never happened!
LJD- It should be part of the DOD’s intellegence gathering capacity. Sure, the vast majority of the killing is done by Iraqis with the vast majority of the victims being Iraqis. But that doesn’t obviate the fact that such information would be useful for military planners–especially when the US civilian “leadership” keeps stressing the importance of winning the “hearts and minds” of Iraqi civilians.
As these studies have shown, you don’t have to “be part of” something in order to “keep tabs” on it–in fact that is the essence of intellegence gathering.
It is obviously something that the Defence leadership–beginning with Rumsfled–is not interested in, since most independent analysts of Iraq understood quite well that there would be significant problems with occupying the country.
I think this is right. Example: Kevin Drum. To date I’ve been thoroughly underwhelmed by his statistical abilities, but he sounds convincing.
Not quite. While it is true that all could have been manipulated, far more people are involved with say the unemployment rate, the consumer price index and further, the methods are well known and researchers all over the world look at them. So the CPI/Unemployment manipulation isn’t quite as straight forward as Lancet study manipulation. This of course doesn’t mean that the study was manipulated. Bias can creep into a study for completely “honest” reasons in the sense that the methodology might be flawed. My understanding is that the methodology is for studying things like diseases, if that is the case it might be inappropriate for studying things like intentional violence.
Seriously, your comment demonstrates a serious lack of understanding of statistics. The word bias is in many regards a technical one when it comes to statistics. An estimator in one context can be just fine, but in another it can be biased. Spotting the context can sometimes be problematic for the researcher. Moreover, extending an estimator to a new area can have the same effect. There is no malice, simply an error that leads to bias.
So for you to say that bias must mean something malicious demonstrates to me that you are not really all that knowledgeable about statistics.
Not necessarily. Suppose that Iraq Body Count is right and the average number killed in a roadside bombing is say, 8. What is the number of woundedd; 2x, 3x? If it is 2.5x then we are talking about 20 wounded people. If we have say 160,000 dead due to roadside bombings then we’d expect 400,000 wounded. This is one of the things Iraq Body Count points too when raising a warning flag about this study,
What they are saying is that the number of wounded should be much, much higher based on other statistical analysis and data. Then using the Lancet numbers we should get a very large number of wounded…wounded that don’t seem to be going to the hospital. That is itself unusual to the people at Iraq Body Count because their experience is that people wounded by IEDs tend to seek medical attention and if they are too wounded to do so, people responding to the attack do it for them.
Soooo…are you implying that the 655,000 were all beheaded? I know that can’t be right, but the study looks at excess deaths which includes things like getting hit by a car and not being able to get to a hospital in time since the local hospital was hit by an air strike or some such.
The two situations may or may not, be analogous. Was the Rwanda genocide done using IEDs, air strikes and gun fire? Were hospitals and infrastructure targeted? If the answer is generally no, then you might want to re-think this argument.
It doesn’t make a bit of difference whether 65,000 were killed or 650,000: those who believe each way will take their own finding as a given. And whatever the actual number it’s far too high.
To my mind the question is what means will reduce the number? One side says staying the course (which presumably means continuing to train Iraqi police and military) is the only alternative. The other says withdrawing our forces from Iraq will reduce the number of deaths there. Since most of the deaths in Iraq are Iraqis being killed by other Iraqis, I think that’s a stretch.