BlogMatrix
 

Lancet report author: ran for the Democrats; will "try to help [them] win in November"

edit David Janes 2006-10-20 20:33 UTC add comment  ·

Tim Blair is a god. Look what he dug up:

"Michael Arcuri is a strong candidate, and I came to the realization that my staying in the race would only make it more difficult for him to win in November," Roberts said Wednesday morning. "I think it’s critically important that we elect a Democrat and that Democrats take control of the House of Representatives."

Chenango County Democrat Les Roberts, 44, withdrew Wednesday from the 24th Congressional District race.

[...] Roberts said, "Republican control of the Congress and White House in recent years has given us the most destructive governance since the Vietnam War."

The pre-emptive war against Iraq and record deficits fueled by "tax gifts for the richest few" have left the United States greatly weakened, he said.

In recent years, "one-party rule has degraded the Constitution and American civil liberties dramatically," Roberts continued. "We need to do something about that this year."

Roberts said he would try to help Arcuri win in November, and later in the morning, Arcuri had kind words for his former opponent.

Weird that (as Blair notes) no newspaper reports of the Lancet study could find the time to mention that he ran for the Democrats this year.

Read all posts on the Lancet study.

StatsGuy: statistically unsound and unreliable

edit David Janes 2006-10-19 12:25 UTC add comment  ·

StatsGuy writes about the Lancet study (via Kathy). I'm just going to include a small snippet, this a long post with lots of stats and survey stuff that you should really read through:

The interview team went to 1849 households in urban areas of Iraq and encountered only 15 refusals and only 16 residences where neither the head of the household nor a spouse was in.  Don’t forget that they only went to each household once: there was no follow-up whatever.  If I ran a door-to-door survey with a response rate of 98.3% on the first go-round, I’d think I’d died and gone to statisticians’ heaven.  That is nothing short of miraculous.  That response rate implies that family heads in urban Iraq are virtually always at home.

Reading through the following paragraphs should really capture you interest -- the survey team was probably one of the most speediest and effective teams ever fielded by humanity. Or perhaps something else was going on.

Stats guy also writes (in a separate post) that the Lancet report authors ignored other studies on Iraq:

This I found odd.  Articles in academic and professional journals that address topics of controversy generally include references to previously published studies and discuss the perspective the current article takes vis-à-vis the views and findings of those earlier studies.  That is how scientific knowledge advances—by critically engaging published findings of other scholars and specialists.

The authors of the 2006 Lancet article, however, appear uninterested in critical engagement with the ILCS estimate of Iraqi deaths.  Yet we know that the Lancet researchers are aware of the ILCS, for they refer to it twice in their footnotes.  The first page mentions "surveys that assessed the burden of conflict on the population" and the fact that "insufficient water supplies, non-functional sewerage, and restricted electricity supply . . . create health hazards", and for these the ILCS is footnoted.

But as for critical discussion of the enormous difference between the ILCS estimate of deaths and the estimates generated from both Lancet surveys, the authors don't want to touch that.  They don't even acknowledge its existence.

Read all posts on the Lancet study.

"A bogus study on Iraq casualties"

edit David Janes 2006-10-19 12:13 UTC add comment  ·

Steven E Moore on the Lancet study in OpinionJournal. First, about the survey method:

Survey results frequently have a margin of error of plus or minus 3% or 5%--not 1200%.

The group--associated with the Johns Hopkins Bloomberg School of Public Health--employed cluster sampling for in-person interviews, which is the methodology that I and most researchers use in developing countries. Here, in the U.S., opinion surveys often use telephone polls, selecting individuals at random. But for a country lacking in telephone penetration, door-to-door interviews are required: Neighborhoods are selected at random, and then individuals are selected at random in "clusters" within each neighborhood for door-to-door interviews. Without cluster sampling, the expense and time associated with travel would make in-person interviewing virtually impossible.

However, the key to the validity of cluster sampling is to use enough cluster points. In their 2006 report, "Mortality after the 2003 invasion of Iraq: a cross-sectional sample survey," the Johns Hopkins team says it used 47 cluster points for their sample of 1,849 interviews. This is astonishing: I wouldn't survey a junior high school, no less an entire country, using only 47 cluster points.

Neither would anyone else. For its 2004 survey of Iraq, the United Nations Development Program (UNDP) used 2,200 cluster points of 10 interviews each for a total sample of 21,688. True, interviews are expensive and not everyone has the U.N.'s bank account. However, even for a similarly sized sample, that is an extraordinarily small number of cluster points. A 2005 survey conducted by ABC News, Time magazine, the BBC, NHK and Der Spiegel used 135 cluster points with a sample size of 1,711--almost three times that of the Johns Hopkins team for 93% of the sample size.

Do the Lancet study's author stand behind their words?

Curious about the kind of people who would have the chutzpah to claim to a national audience that this kind of research was methodologically sound, I contacted Johns Hopkins University and was referred to Les Roberts, one of the primary authors of the study. Dr. Roberts defended his 47 cluster points, saying that this was standard. I'm not sure whose standards these are.

Appendix A of the Johns Hopkins survey, for example, cites several other studies of mortality in war zones, and uses the citations to validate the group's use of cluster sampling. One study is by the International Rescue Committee in the Democratic Republic of Congo, which used 750 cluster points. Harvard's School of Public Health, in a 1992 survey of Iraq, used 271 cluster points. Another study in Kosovo cites the use of 50 cluster points, but this was for a population of just 1.6 million, compared to Iraq's 27 million.

When I pointed out these numbers to Dr. Roberts, he said that the appendices were written by a student and should be ignored. Which led me to wonder what other sections of the survey should be ignored.

Strangely, the Lancet study didn't bother to ask questions that would let the results be verified against other known facts:

With so few cluster points, it is highly unlikely the Johns Hopkins survey is representative of the population in Iraq. However, there is a definitive method of establishing if it is. Recording the gender, age, education and other demographic characteristics of the respondents allows a researcher to compare his survey results to a known demographic instrument, such as a census.

Dr. Roberts said that his team's surveyors did not ask demographic questions. I was so surprised to hear this that I emailed him later in the day to ask a second time if his team asked demographic questions and compared the results to the 1997 Iraqi census. Dr. Roberts replied that he had not even looked at the Iraqi census.

And so, while the gender and the age of the deceased were recorded in the 2006 Johns Hopkins study, nobody, according to Dr. Roberts, recorded demographic information for the living survey respondents. This would be the first survey I have looked at in my 15 years of looking that did not ask demographic questions of its respondents. But don't take my word for it--try using Google to find a survey that does not ask demographic questions.

Read all posts on the Lancet study.

"Reality Check" on the Lancet study by the Iraq Body Count

edit David Janes 2006-10-16 19:58 UTC add comment  ·

The Iraq Body Count (of which I'm not a fan, for other reasons) writes a "reality check" for the Lancet Study which claims 650,000+ Iraqis have been killed in the last three years:

A new study has been released by the Lancet medical journal estimating over 650,000 excess deaths in Iraq. The Iraqi mortality estimates published in the Lancet in October 2006 imply, among other things, that:

  1. On average, a thousand Iraqis have been violently killed every single day in the first half of 2006, with less than a tenth of them being noticed by any public surveillance mechanisms;
  2. Some 800,000 or more Iraqis suffered blast wounds and other serious conflict-related injuries in the past two years, but less than a tenth of them received any kind of hospital treatment;
  3. Over 7% of the entire adult male population of Iraq has already been killed in violence, with no less than 10% in the worst affected areas covering most of central Iraq;
  4. Half a million death certificates were received by families which were never officially recorded as having been issued;
  5. The Coalition has killed far more Iraqis in the last year than in earlier years containing the initial massive "Shock and Awe" invasion and the major assaults on Falluja. 

Point #4 had occured to me also.

Read all posts on the Lancet study.

Only 63?

edit David Janes 2006-10-15 21:32 UTC 1  comment  ·

The Toronto Star reports (via Michael K):

Suspected Shiite militiamen killed at least 46 Sunni Arabs in a weekend rampage of revenge killing in a city north of Baghdad, an Interior Ministry official said Sunday, raising the toll in the latest sectarian bloodletting there to 63.

And don't forget that if you the Lancet's report is correct, several hundred to maybe a thousand other people were killed on Sunday also, but somehow no one managed to notice to report it.

Read all posts on the Lancet study.

Uh oh

edit David Janes 2006-10-13 19:21 UTC add comment  ·  ·

I fear Bruce (and perhaps Damian too) may have slipped and bumped his head; how else to explain this posting on the Lancet article:

The actual statement of claim is pretty simple when you reduce it to the nub. The Iraqi survey team claims to have randomly interviewed households totalling 12,801 individuals, and found that exactly 300 of them had died violently between the U.S invasion and July, 2006. Survivors produced approximately 240 death certificates that confirmed this.

That math (even if you use only the certified deaths) works out to 5.5 violent deaths per thousand people per annum in Iraq. Extrapolate that to the entire population of Iraq and you get a number in the 450,000 range. Assume the other 60 undocumented violent deaths were truthful reports, as well, and you're up to 600K.

The methodology as defined in the study is as sound as any other scientific study (more on this later). The simplest statistical sample size calculations tell us that if the real number of violent deaths so far in Iraq had been, say, 60,000, then there should have been around 30 certifiable violent deaths in a sample of this size, not 240 (or 300).

Now, there's nothing I particular disagree with in what Bruce is saying here. However, there's one other fact you can deduce from this data: to switch the projected number of deaths from 60,000 to 600,000, all you need to do is deliberately mis-sample between 210 and 270 households in the survey of 12,801 household; to spell it out, to go to households where you know someone died instead of a "random" sample about 2% of the time. I could come with similar numbers for Toronto if you gave me a budget and a handful of undergrads with a sense of humour.

Who would do such a dastardly thing? Lying lays near the top of the toolbox of the political extremist and it's hard to argue, given the timing etc. that the purpose of the Lancet articles are anything but political in nature.

You of course can judge for yourself whether anti-war activist Horton (who's not afraid, or ashamed, to appear on the same stage as the odious certifiable nut George Galloway) is such an extremist.

Read all posts on the lancet study.

655,000 Iraqi deaths -- even CNN is dubious

edit David Janes 2006-10-11 10:19 UTC 4 comments  ·

CNN reports:

A controversial new study contends nearly 655,000 Iraqis have died because of the war, suggesting a far higher death toll than other estimates.

The timing of the survey's release, just a few weeks before the U.S. congressional elections, led one expert to call it "politics."

In the new study, researchers attempt to calculate how many more Iraqis have died since March 2003 than one would expect without the war. Their conclusion, based on interviews of households and not a body count, is that about 600,000 died from violence, mostly gunfire. They also found a small increase in deaths from other causes like heart disease and cancer.

"Deaths are occurring in Iraq now at a rate more than three times that from before the invasion of March 2003," Dr. Gilbert Burnham, lead author of the study, said in a statement.

The study by Burnham, of the Johns Hopkins Bloomberg School of Public Health, and others is to be published on the Web site of The Lancet, a medical journal.

An accurate count of Iraqi deaths has been difficult to obtain, but one respected group puts its rough estimate at closer to 50,000. And at least one expert was skeptical of the new findings.

The Lancet, you'll remember, was responsible for another phony-baloney "study" of Iraq war written by a crowd of anti-war crazies that used very dubious statistical techniques (i.e. go to the neighbourhood where the worst fighting is and extrapolate to the entire population) to come up with similar numbers in the past.

The last Lancet "study" came out just before the last US election. Remember when doctors used to be interested in medicine?

More interesting is their claim that "Deaths are occurring in Iraq now at a rate more than three times that from before the invasion of March 2003". I.e. if the invasion had not have happened, we would have expected to see 220,000 deaths over the last 3 years under Saddam's regime, as compared to the Iraq Body Count's 50,000. (Note that there's probably issues with the 200,000+ number too, since that rate be based on another phoney story).

That's not to say things haven't gone to ratshit recently; there's a lot more to be said there too.

Read all posts on the Lancet study.