I just came across something called the MacNamara Fallacy – as best I can see it was first described by Charles Handy in his book, The Empty Raincoat (haven’t read it yet but just ordered a second hand copy through Amazon marketplace for a penny!) – Robert MacNamara was US Secretary of Defence during the Vietnam War.
One of the worst characteristics of the current approach to health (an approach shared in other spheres like education, management and so on) is the tendency to measure what can be easily measured and then base every decision on that, disregarding as unimportant whatever cannot be easily measured.
Here’s the quote (this time referred to by Dr David Haslam in the RCGP journal –
Haslam D (2007). British Journal of General Practice 57:545, 987-993.)
-
The first step is to measure whatever can be easily measured. This is OK as far as it goes.
-
The second step is to disregard that which can’t be easily measured, or to give it an arbitrary quantitative value. This is artificial and misleading.
-
The third step is to presume that what can’t be measured easily isn’t important. This is blindness.
-
The fourth step is to say that what can’t be easily measured doesn’t exist. This is suicide.
Heroes Not Zombies 
Spot on. MacNamara was the man behind the body count, which caused some pretty poor behaviour – see “The Trap” part I, by Adam Curtis – it is on Google Video here. I don’t have any solid empirical evidence, but my years in marketing taught me this: the easier something is to measure, the less useful that measure is. Looking at it the other way, the more useful a measure is, the more difficult it is to capture.
Thanks Jamin – I think your observation is really fascinating there. You’re in marketing? You know I think there’s so much we’ve failed to progress in terms of our human knowledge and understanding because we’ve been failing to study phenomena in inter-disciplinary ways. What you’ve observed about measurement has important implications for understanding health and the provision of health services. I love the way you summed up there – taking the two viewpoints of easier measurements being less useful and useful measures being difficult to capture. Great! Thank you
I market hightech products to businesses, so I’m not a top expert on consumer marketing :).
Applying a bit of information theory, you could state it more this way:
Easy to obtain measures are, by definition, simplistic. They are usually based on a single variable with a narrow scope and are highly objective.
The most meaningful measures, by definition, cover a wide range of attributes, and subjective and so are hard to capture.
When you deal with people, you are dealing with subjective measures (inside their head). Managers have to deal with objective measures (external perspective). That is a non-trivial translation.
One way around this is to look for correlations – single variables that show what is going off. For example, from today’s news, patients want a good ambulance service. A simple measure of a good ambulance service is response time (response time and rating of service correlate).
So, you measure response time and manage to that. The challenge with using a simple variable that correlates is that as soon as you manage using it, you break the correlation: You dispatch paramedics in cars – now people get the response time, but don’t get back to hospital (you improve the variable, but not the services). Or you choose to measure the variable in a favourable way (eg from the point at which the emergency operator gets the details, rather than from point at which the phone answers – this is part of what the government is trying to fix I think).
I’d be interested to hear a summary and opinions on the book from you!
Hey thanks for this. I think your comments are really helpful and thought-provoking.
Yes, that’s exactly what we see and in health care there’s a special version of that. It’s best understood in considering the difference between ‘disease’ and ‘illness’. (see this post of mine for more about that – https://heroesnotzombies.wordpress.com/2007/04/09/the-human-spectrometer/ )
Disease can usually be measured because disease is the physical, objective lesion. For example you can measure tumour size, or you can measure blood sugar levels or whatever. Illness however is subjective and as such cannot be measured. Pain, itch, nausea, lethargy, dizziness, you name it….they can’t be measured. But for a patient it’s exactly these subjective symptoms which are making the greatest impact on their lives. In fact, we know that diseases are only losely related to symptoms – its a non-linear relationship.
So if we create a health care system based on what’s measurable we never quite satisfy its purpose – to relieve suffering
[…] 1, 2008 by bobleckridge Here’s the trail…….a colleague emailed me about MacNamara’s Fallacy, which was mentioned in an article, by David Haslam, in the Journal of the Royal College of […]