NOTTINGHAM: African HIV trials fudged registration dates

So what else did they play fast and loose with?

Ripe Tomatoes, September 13, 2013


Circumcision and HIV

Two out of three key trials registered late

Does circumcision of adult men reduce HIV transmission? Initial scepticism, surely the foreskin reduces friction between penis and vagina, and prevents abrasions and blood transfer, and conflicting observational data, were eventually overcome by three randomised trials, one each from South Africa (click here), Kenya (click here) and Uganda (click here). In each trial men were randomly allocated to immediate circumcision or to wait for two years. All three trials showed about half the number of new HIV infections in the immediate circumcision group. The relevant Cochrane review (click here) summarises them as follows:

“The resultant incidence risk ratio (IRR) was 0.50 at 12 months with a 95% confidence interval (CI) of 0.34 to 0.72; and 0.46 at 21 or 24 months (95% CI: 0.34 to 0.62). These IRRs can be interpreted as a relative risk reduction of acquiring HIV of 50% at 12 months and 54% at 21 or 24 months following circumcision.”

The Cochrane reviewers judged “the potential for significant biases affecting the trial results [as] low to moderate”. Referring to selective reporting they wrote “All three trials clearly stated in their protocols that the primary outcome was HIV incidence. The risk of bias due to incomplete outcome reporting is therefore low in all three trials.”

The trial protocols have not been published so we have to trust that the Cochrane reviewers interpreted them correctly. But if they relied on trial registration documents (click here, for South Africa, here for Kenya and here for Uganda) they would have been misled. They they all indicate more of less the same primary outcomes and planned sample sizes as in the respective papers. But according to the date of first entry, only the Kenyan trial was prospectively registered. The South African trial was registered a year and a half after recruitment ended and seven days before the results were published! The Ugandan one a month after recruitment ended,and a month before publication.

	Recruitment period	Date of 1st publication of results	1st trial registration date	Registry planned sample size	Primary outcome in registry	Primary outcome in paper
South Africa	July 2002 – Feb 2004	26 July 2005	July 19 2005	3274 planned. 3274 in paper	HIV infection at 3, 12 and 21 months	All HIV infections at 3, 12 and 21 months
Kenya	Feb 2002 – Dec 12 2006	24 Feb 2007	April 23 2003	Initially 3,000. Altered to 2887 in registry. 2784 in paper	HIV incidence at 2 years Complications of circumcision	HIV incidence after three interim analyses
Uganda	August 2002 – Dec 2006	24 Feb 2007	Jan 23 2007	5,000 planned 4996 in paper.	HIV acquisition no time point specified	HIV incidence

It looks like PLOS One and The Lancet wanted to publish sexy high impact trials, found they weren’t registered, so got the authors to do it retrospectively, and hoped no-one would notice! All three trials were funded by the public sector.

Jim Thornton (Professor of Obstetrics and Gynaecology at Nottingham University)

WASHINGTON, DC: Circumcision studies to be re-checked

Mother Nature Network
May 30, 2012

New project aims to double-check international aid program results

How much do adult male circumcision programs in Africa reduce HIV rates [...if at all]? Does group lending, which encourages repayment by putting people in support groups, make it easier for peasants in Thailand to pay back loans? There are high-profile studies that answer those questions, said Benjamin Wood, a researcher at the International Initiative for Impact Evaluation's Washington, D.C. office. But his organization thinks there's still a piece missing – studies that double-check the results from aid program research. "It's a bit of a hole in the field," he said. So the initiative is looking to fund repeat studies, which have been perpetually unpopular in almost any field of science.

Getting the same results twice or several times over is an important part of showing a study is true. Repeating experiments can also resolve debates about studies some experts think were poorly done. They help the U.S. government, which spent $15 billion on international aid in 2011, and private donors, who spend billions more, decide how to spend their budgets. Yet repeats are rarely published because researchers don't have any incentive to perform them.

One problem is that scientists build their reputations by publishing new, exciting papers. "Academics have to get published. That's their job," Wood said. Academic journals are less likely to publish a repeat, however, especially if the repeat that finds the original researchers did everything right, Wood told InnovationNewsDaily. Papers that point out something is wrong are more popular, he said.

That leads to another problem. Because repeat studies tend to be negative, researchers are reluctant to release their original data for replication.

So a new project Wood is leading will give researchers money to do repeat studies. His organization will publish preliminary results on their website, so good work doesn't get lost just because it's a repeat. The initiative won't favor negative results, so original researchers shouldn't worry a repeat request will necessarily hurt them.

In fact, original researchers could benefit from cooperating on a replication study, he said. A positive repeat will bring positive publicity to the original study and make it seem more reliable, so more people will read it. "These are all things that are helpful for the original researchers," Wood said.

The initiative will also post on their site if the researchers won't give up their data within three months of getting a request. The listing isn't exactly meant to embarrass researchers, according to Wood. "I think it's just kind of an identification," he said, "and maybe if you're unable to produce your original dataset, it starts to raise questions."

Wood is now looking for experts to nominate studies to get repeated. He wants influential or controversial studies, especially.... The International Initiative for Impact Evaluation may perform in-house a repeat on a study about male circumcision for HIV transmission prevention.

...