Even though medical research is expected to generate scientific truth, unfortunately we sometimes see exaggerated and misleading conclusions. This could lead some readers into taking wrong decisions.
Most people tend to blindly believe the authors’ conclusions given in the abstract of each study, and look no further. This is unfortunately not an advisable strategy. Often, research studies use terminology and jargon that decorates their findings, leading lay people to believe that the effect was much larger than it actually was.
There are numerous biases associated with research, and these are not limited to healthcare. Awareness of basic statistics helps identify a few pitfalls.This article is written to highlight one common flaw - something that can easily be spotted.
Let’s start with a typical real-life medical question.
A new blood pressure pill has hit the market, and many people are switching to the newer pill because it is apparently “50% safer” than the existing medication. We want to find out the facts, and decide to look up the research study.
The authors compared two medications A and B for hypertension in 1000 patients. Both were equally good at controlling blood pressure. B is new, and more expensive. According to the study, B is claimed to be “50% safer” than the other. How do we decide if that’s important?
To answer this question, let us look at another example.
Which taxi to choose?
Imagine the year is 1975, and we are at a taxi stand in Delhi. There are two choices. Ambassador and Fiat Premier Padmini. Which one do we choose for safety? Most people who know about cars would pick either one.
Now, let’s imagine a hypothetical research study that had studied the safety of taxi cars.
What if the author of that study concluded that “Ambassador taxi is 50% less likely to have an accident than Fiat”?
Will that make us change the decision?
Many people who hear that study conclusion would say “Oh my God! I didn’t know that. Thanks for letting us know. We will pick the Ambassador. Fiat is too dangerous”
The truth is, there is no need to panic. Research studies are known to exaggerate their findings to grab attention. That is not the same as fraudulent research or falsification. Statistics is a tool that can be used to present even the most trivial findings in the most dramatic format, without being dishonest.
As someone said, if we torture data long enough, it will confess to anything we want.
Let’s look at the hypothetical taxi study in detail.
Imagine that the authors studied 100,000 taxi cars each of Ambassador and Fiat, tracked their accident history over 1 year. They found that 3 accidents occurred in the Ambassador group, while 6 occurred in the Fiat group over the same time period.
They conclude that travelling in an Ambassador taxi is 50% less likely to result in an accident.
Such conclusions often appear verbatim in news media headlines, without much further details.
A typical headline will be: “Ambassador taxi 50% safer than Fiat”
Now let’s look at their calculations.
The research question is to compare the accident rate of the two cars, when used as taxis.
Findings:
Ambassador: 3 per 100,000 (0.003%)
Fiat: 6 per 100,000 (0.006%)
The difference is 0.006%-0.003%=0.003%
Which means that if we pick the Ambassador car, our accident rate is 0.003% less than that of Fiat.
Wait a minute. How is that even possible, when the study conclusions stated “50% safer”? How could they say that? Could there be a mistake?
There is no mistake here. The authors simply used the most impressive parameter to express their findings.
You must be logged in to post a comment.