Heroes Not Zombies

There have been several studies which have shown that there is evidence of selection bias in the publication of drug trials. In this post here I gave a couple of examples of concerns about both the sheer number of trials conducted vs the number of publication outlets, and the huge percentage of registered trials which never make it as far as even being offered for publication.

Here’s another piece of that jigsaw. Reviewers from the Cochrane Library found that trials which demonstrated a positive effect of the drug studied were much more likely to be published than those which didn’t.

“This publication bias has important implications for healthcare. Unless both positive and negative findings from clinical trials are made available, it is impossible to make a fair assessment of a drug’s safety and efficacy,”

They found that not only were negative trials less likely to be published at all, but those which were tended to be published between and one and four years after the positive trials.

Results from one of the five studies in the review indicated that investigators and not editors might be to blame. The reasons most commonly given for not publishing were that investigators thought their findings were not interesting enough or did not have time. “The registration of all clinical trial protocols before they start should make it easier to identify where we are missing results,” says Kay Dickersin from Johns Hopkins University in Baltimore, USA, another of the researchers on this project.

In other words, those carrying out the research were the ones responsible for deciding not to bother publishing unless the results supported the use of the drug being studied. Sadly, the New England Journal of Medicine study I quoted in the linked post at the beginning of this one, found that 30% of registered trials didn’t make it to publication! So it will take more than a simple registration process to sort out this distortion of the evidence base.