Reviewed by: Yeong Sek Yee & Khadijah Shaari
This article is a summary of Chapter 1 “MISSING DATA” of the bestselling book, BAD PHARMA by Dr Ben Goldacre which was released in Oct 2012. The author Dr Ben Michael Goldacre, graduated in 1995 with a first-class honors degree in medicine from Magdalen College, Oxford, obtained an MA in philosophy from Kings College London, and undertook clinical training at UCL Medical School, qualifying as a medical doctor in 2000 and as a psychiatrist in 2005. As of 2012 he is Wellcome research fellow in epidemiology at the London School of Hygiene and Tropical Medicine.
“Missing Data” in the context of clinical trials/research means data that are not reported by the drug companies when the clinical trials/research are completed or not completed. Obviously these are data that are “negative” or “unflattering” for the drug being tested. And obviously, these “unfavourable” results will not be included in their submissions to the respective regulators. In other words, only favourable results are submitted or published.
Dr Goldacre strongly argued that the clinical trials undertaken by drug companies routinely reach conclusions favourable to the company. These positive results are achieved in a number of ways:
Sometimes the industry-sponsored trials are flawed by design (for example by comparing the new drug with something that is known to be rubbish or an existing drug at an inadequate dose or perhaps a placebo sugar pill that does almost nothing).
Trial patients can be chosen very carefully, to make a positive result more likely.
In addition, the data is studied as the trial progresses, and if the trial seems to be producing negative data it is stopped prematurely and the results are not published.
On the other hand, if the trial is producing positive data, it may be stopped early so that longer-term effects are not examined.
Drug companies are free to conduct as many trials as they wish…they can then choose which ones to publish. This is known as “publication bias.”
According to Dr Goldacre, “the problem of negative studies going missing has been known for almost as long as people have been doing serious science” This was formally documented by a psychologist Theodore Sterling in 1959. He went through every paper published in the 4 big psychology journals of the time, and found that 286 out of 294 reported a statistically significant result. This, he explained, was plainly fishy. Sterling was the first to put these ideas into a formal academic context, but the basic truth had been recognized for many centuries. Francis Bacon explained in 1620 that we often mislead ourselves by only remembering the times something worked, and forgetting those when it didn’t.
Watch Dr Goldacre on YouTube…What Doctors Don’t Know About the drugs they prescribe. https://www.youtube.com/watch?v=RKmxL8VYy0M
Why Does “Missing Data” matter?
1) According to Dr Goldacre:
Evidence is the only way we can possibly know if something works or doesn’t work in medicine. In the real world of medicine, published evidence is used to make treatment decisions.
When doctors are misled about the effects of the medicines they use, they can end up making decisions that can cause avoidable suffering, or even death. We (doctors) might choose unnecessarily expensive treatments; having been misled into thinking they are more effective than cheaper older drugs. This wastes money, ultimately depriving patients of other treatments….
While doctors are kept in the dark, patients are exposed to inferior treatments, ineffective treatments, unnecessary treatments, and unnecessary expensive treatments that are no better than cheap ones…
2) According to the British Medical Journal:
In Jan 2012, Britain’s top medical journal, the British Medical Journal disclosed that “missing data is a serious problem in clinical research given that it distorts the scientific record and prevents clinical decisions from being based on the best evidence available.” As part of an in-depth BMJ review on the subject, experts on bmj.com warn that patients can be harmed through missing clinical trial data, leading to unnecessary costs to health systems. Read the 2 articles below for further information.
a) Article: Missing Data – A Serious Problem in Clinical Research http://www.medicalnewstoday.com/articles/240032.php
b) Article: Patients at risk after scientists withhold test results from clinical trials of new medicines Read more: http://www.dailymail.co.uk/health/article-2082116/Patients-risk-scientists-withhold-test-results-clinical-trials-new-medicines.html#ixzz2cJb8VxLZ
has published several papers on the subject, assessing the causes, the extent, and consequences of unpublished evidence. The papers confirm that “large proportions of evidence from human trials remain unreported, and that a lot of what is reported is inadequate.”
In an editorial, Dr. Richard Lehman from the University of Oxford and BMJ Clinical Epidemiology Editor, Dr. Elizabeth Loder, revealed a “culture of haphazard publication and incomplete data disclosure,” calling for more vigorous regulation and full access to raw trial data to enable scientists to gain a better understanding of the benefits and harms of many kinds of treatment.
Besides the above, Dr Marcia Angell, MD very bluntly describes what “bias” is in clinical research. Chapter 6 of her best-seller (which all doctors and patients should read) THE TRUTH ABOUT THE DRUG COMPANIES asks “How Good Are New Drugs?” The main points in the chapter are:
Bias is now rampant in drug trials…this can be built into the study design, as is the case with placebo-controlled trials.
Another form of bias is to compare the new drug not just with a placebo but with an old drug given at too low a dose or the old drug can be administered incorrectly,
The trials can be designed to be too brief to be meaningful. That is true of many trials of drugs that need to be taken long term…sometimes treatments look pretty good for a short time but are not effective and may be harmful for long term use. (Is that why a lot of drugs are “recalled” after some time in the market?)….See the list below:
LINK: http://en.wikipedia.org/wiki/List_of_withdrawn_drugs
One of the most common ways to bias trials is to present only part of the data—the part that makes the product look good—and ignores the rest. This is the most dramatic form of bias…out-and-out suppression of negative results.
Is this how new drugs are “scientifically” tested? Dr Angell also gives us an idea how new drugs are “scientifically” approved:
The FDA may approve the drug on the basis of minimal evidence. For example, the agency usually requires simply that the drug work better than a placebo in 2 clinical trials, even if it doesn’t in other trials.
But companies publish only the positive results, not the negative ones. Often in fact, they publish positive results more than once, in slightly different forms in different journals, and
The FDA has no control over this selective publishing. This practice leads doctors to believe that drugs are much better than they are, and the public comes to share this belief, on the basis of media reports.
(Additionally, Dr Goldacre clearly pointed out in BAD PHARMA that…”Regulators frequently approve drugs that are vaguely effective, with serious side effects, on the off-chance that they might be useful to someone, somewhere, when other interventions aren’t an option.”)
To conclude, let us quote Dr Goldacre again…..
”For many of the most important diseases that patients present with, we (the doctors) have no idea which of the widely used treatments is best, and, as a consequence, people suffer and die unnecessarily. Patients, the public, and even many doctors live blissful ignorance of this frightening reality, but in the medical literature, it has been pointed out again and again.”
“Missing Data ….poisons the well for everybody. If proper trials are never done, if trials with negative results are withheld, we simply cannot know the true effects of the treatments that we use”
After you have read both BAD PHARMA and THE TRUTH ABOUT THE DRUGS COMPANIES (there are many other such books), you will understand why Dr Russell Blaylock, a former professor of neurosurgery blatantly proclaimed that…..evidence-based medicine is the biggest con job in the history of the world (see page 161 in KNOCKOUT..Interviews with Doctors Who are Curing Cancer by Suzanne Somers).
SOME FURTHER REFERENCES ON “MISSING DATA”
1) Missing Data in Clinical Studies (Statistics in Practice) by Dr Geert Molenberghs and, Michael Kenward … provides a comprehensive account of the problems arising when data from clinical and related studies are incomplete,
2) Article: Addressing missing data in clinical trials. http://www.ncbi.nlm.nih.gov/pubmed/21242367
3) Article: Missing Data May Skew Clinical Trials. http://health.usnews.com/health-news/news/articles/2012/10/03/missing-data-may-skew-clinical-trials
3) Talk: Handling of Missing Data in Clinical Trials (26 min) http://hstalks.com/main/view_talk.php?t=550&r=15&j=754&c=252
4) Spot On London 2012: Ben Goldacre’s Keynote Speech http://www.youtube.com/watch?v=cUvd4zZckrs
Filed under: Book & Research Report 