Heroes Not Zombies

Research studies routinely use different ways to present their conclusions. In particular they present “relative risk”, “absolute risk”, and/or “numbers needed to treat” information. Gigerenzer is brilliantly clear about this. He says

Gigerenzer shows how drugs companies and authorities routinely use Relative Risk to emphasise the potential benefits of their treatment while at the same time presenting the potential harms as Absolute Risks to minimize the impression of adverse potential.

This week in the BMJ, there is a study where researchers have looked at the use of relative and absolute risk reporting in studies which examine inequalities in health. Their argument is quite modest – that to fully understand any study BOTH relative and absolute risks should be reported. They say this is best practice. So, how often does it occur?

Almost never.

75% (258/344) of all articles reported only relative measures in the full text; among these, 46% (119/258) contained no information on absolute baseline risks that would facilitate calculation of absolute effect measures. 18% (61/344) of all articles reported only absolute measures in the full text, and 7% (25/344) reported both absolute and relative measure

But in fact, the literature is even more skewed than those conclusions suggest because

We found that nearly 90% of studies with quantitative estimates in the abstract presented only relative measures, and 75% of all articles reported only relative measures in the full text.

Why is the bias towards relative risk even greater in the abstract than in the full text? Because its the abstract which most people read, and it is probably the abstract which is used to write the PR headlines.

Does this matter? Yes it does. The authors argue that the choices of relative or absolute risk reporting influence policy making.

Makes you wonder how often the “evidence” is produced to fit the policy, rather than the policy being produced from the evidence…….and that applies across the board from the writing of guidelines and protocols, to decisions about health care spending, to individual doctors deciding what treatment to offer an individual patient.