Heroes Not Zombies

There have been several studies which have shown that there is evidence of selection bias in the publication of drug trials. In this post here I gave a couple of examples of concerns about both the sheer number of trials conducted vs the number of publication outlets, and the huge percentage of registered trials which never make it as far as even being offered for publication.

Here’s another piece of that jigsaw. Reviewers from the Cochrane Library found that trials which demonstrated a positive effect of the drug studied were much more likely to be published than those which didn’t.

“This publication bias has important implications for healthcare. Unless both positive and negative findings from clinical trials are made available, it is impossible to make a fair assessment of a drug’s safety and efficacy,”

They found that not only were negative trials less likely to be published at all, but those which were tended to be published between and one and four years after the positive trials.

Results from one of the five studies in the review indicated that investigators and not editors might be to blame. The reasons most commonly given for not publishing were that investigators thought their findings were not interesting enough or did not have time. “The registration of all clinical trial protocols before they start should make it easier to identify where we are missing results,” says Kay Dickersin from Johns Hopkins University in Baltimore, USA, another of the researchers on this project.

In other words, those carrying out the research were the ones responsible for deciding not to bother publishing unless the results supported the use of the drug being studied. Sadly, the New England Journal of Medicine study I quoted in the linked post at the beginning of this one, found that 30% of registered trials didn’t make it to publication! So it will take more than a simple registration process to sort out this distortion of the evidence base.

Read Full Post »

Clinical epidemiology has been cleverly spun into something referred to as “Evidence Based Medicine” (“EBM”). It’s such a powerful spin that we are seeing the term “evidence based” being used widely now to justify any decision made by any authority – whether that be politicians, civil servants, educators or scientists. In fact, the phrase is thrown around so freely and unthinkingly that it’s quickly losing its original meaning, instead becoming a code term for “fact” or undeniable Truth. “Evidence based” is a label now which is supposed to convey that the statement to which it is attached has a high value. The idea of “evidence based” as promoted by those who use the term is founded on the belief that what constitutes “truth” is, however, almost exclusively, the physical and the measurable. Let me quote from  Maya Goldenberg, who wrote about this in Social Science and Medicine. Volume 62, Issue 11, June 2006 –

Reflecting on how the popular idea of “patient-centred care” remains largely unrealized in clinical practice, Van Weel and Knottneurus (1999) note that while physicians are encouraged to make diagnoses in physical, psychological, and social terms, “the EBM that is currently promoted either restricts itself to physical evidence alone, or casts such evidence at the top of a hierarchy that tends to devalue any evidence ‘lower down’”. The hierarchy of evidence promotes a certain scientistic accounting of the goals of medicine, which, the worry goes, is incommensurable with the proposed reorientation of medical practice toward the patient’s search for meaning in the illness experience. The bridging of scientistic “measure” and existential “meaning” has received some attention in the critical EBM literature with the general consensus that we need an “integrated” model of evidence that properly reflects modern health care’s constitution by diverse academic traditions—including the humanities, social sciences, and the pure and applied sciences—that rely on equally diverse notions of evidence. While EBM values evidence that is statistical in nature and general in its application, and therefore places quantitative data derived through the application of recognised study designs at the top of its pre-graded hierarchies of evidence, the phenomenological approaches rooted in hermeneutics, ethnography, sociology, and anthropology, regard evidence as primarily narrative, subjective, and historical in nature. Unlike the impersonal and generalisable measures undertaken in EBM, this conception of evidence is illustrated in case histories, clinical encounters, and qualitative studies such as in-depth interviews and focus groups. The features of the medical encounter and the illness experience emphasised by medical phenomenologists and proponents of a more “humane” medicine suggest the need to reconsider what constitutes the goals of medicine and flip EBM’s hierarchy of evidence on its head. The quantitative measures and generalisations that come out of controlled trials and biostatistical analysis are not conducive to the questions of meaning that medical phenomenology wants to address and make central to medicine.

Goldenberg helpfully nails down the key issues – “ While EBM values evidence that is statistical in nature and general in its application, and therefore places quantitative data derived through the application of recognised study designs at the top of its pre-graded hierarchies of evidence, the phenomenological approaches rooted in hermeneutics, ethnography, sociology, and anthropology, regard evidence as primarily narrative, subjective, and historical in nature” – The EBM approach is a statistical approach. It tells us something about probabilities, derived from studies of large, supposedly homogenous groups. It doesn’t give us certainty about either effectiveness of a treatment, or about prognosis, in any individual patient. Nor does it give us any insights into either the experience of illness, or the experience of therapeutic recovery for patients.
The use of clinical epidemiology alone in the application of health policy or therapeutic practice is neither rational nor sensible. Health and illness are experiential. Human experiences can only be conveyed by human beings. If we want a more humane form of medical practice which is a closer fit with individual reality then we need to develop our phenomenological understanding and give such research considerably greater consideration than we currently do instead of dismissing the unquantifiable as irrelevant.

Read Full Post »