Wednesday, May 30, 2012

WASHINGTON, DC: Circumcision studies to be re-checked

Mother Nature Network
May 30, 2012

New project aims to double-check international aid program results

How much do adult male circumcision programs in Africa reduce HIV rates [...if at all]? Does group lending, which encourages repayment by putting people in support groups, make it easier for peasants in Thailand to pay back loans? There are high-profile studies that answer those questions, said Benjamin Wood, a researcher at the International Initiative for Impact Evaluation's Washington, D.C. office. But his organization thinks there's still a piece missing – studies that double-check the results from aid program research. "It's a bit of a hole in the field," he said. So the initiative is looking to fund repeat studies, which have been perpetually unpopular in almost any field of science.

Getting the same results twice or several times over is an important part of showing a study is true. Repeating experiments can also resolve debates about studies some experts think were poorly done. They help the U.S. government, which spent $15 billion on international aid in 2011, and private donors, who spend billions more, decide how to spend their budgets. Yet repeats are rarely published because researchers don't have any incentive to perform them.

One problem is that scientists build their reputations by publishing new, exciting papers. "Academics have to get published. That's their job," Wood said. Academic journals are less likely to publish a repeat, however, especially if the repeat that finds the original researchers did everything right, Wood told InnovationNewsDaily. Papers that point out something is wrong are more popular, he said.

That leads to another problem. Because repeat studies tend to be negative, researchers are reluctant to release their original data for replication.

So a new project Wood is leading will give researchers money to do repeat studies. His organization will publish preliminary results on their website, so good work doesn't get lost just because it's a repeat. The initiative won't favor negative results, so original researchers shouldn't worry a repeat request will necessarily hurt them.

In fact, original researchers could benefit from cooperating on a replication study, he said. A positive repeat will bring positive publicity to the original study and make it seem more reliable, so more people will read it. "These are all things that are helpful for the original researchers," Wood said.

The initiative will also post on their site if the researchers won't give up their data within three months of getting a request. The listing isn't exactly meant to embarrass researchers, according to Wood. "I think it's just kind of an identification," he said, "and maybe if you're unable to produce your original dataset, it starts to raise questions."

Wood is now looking for experts to nominate studies to get repeated. He wants influential or controversial studies, especially.... The International Initiative for Impact Evaluation may perform in-house a repeat on a study about male circumcision for HIV transmission prevention.



  1. The ethical norm in my neck of the intellectual woods is that investigators must give a copy of their data to anyone with a university or govt. research institute letterhead, provided that the investigators have written all the articles using that data that they wish to write. Further, authors of empirical studies must humour attempted critical replication of their work. The original investigators may charge money for a copy of their data, especially if the data were produced for the study and required nontrivial time and trouble. How much to charge has not been resolved.

    I would welcome a reanalysis of the raw data from the 3 notorious African randomised clinical trials. But I do not predict that such a reanalysis will find much. My reservations about those clinical trials pertain to the protocols, in particular the following 2 details:

    (1) The trials were not allowed to run 5-10 years. Hence the trials shed no light on the possibility of risk compensation, and of circumcision only delaying the inevitable;

    (2) There is an allegation that the circumcised treatments were given anti-AIDS instruction and free condoms, while the intact controls were not. If this allegation is true, the trials are totally compromised.

  2. The list of factors that compromise these "studies," if it they can even be called that, is rather long. It looks like there is a lot of data that the "researchers" have failed to release. Read about it here: