The Interpretation of Evidence
Evidence should be the deciding factor in an argument. If the evidence is on your side, then surely your opponent will concede. Often this is not the case, often the two sides of an argument will interpret the same evidence in ways which appear to support their own beliefs. How can this happen? Let us examine an imaginary trial for a new medical treatment…
The inventors of “The Treatment” believe it provides relief from the symptoms of a condition we’ll call “The Condition”. So they commission a trial to prove it works. They gather 1000 patients who have The Condition, and give them The Treatment. 80% of the patients in the study report that they feel much better after one week. The Treatment is declared a success!
But wait, the skeptics hear about this and they ask “How do you know it was really a success, maybe they would have gotten better without The Treatment?” and so the trial is re-run, this time with a control group. 1000 patients are given Treatment and 1000 are given nothing. After a week 80% of the patients who got The Treatment reported that they felt much better, whereas only 50% of the patients who received no treatment reported feeling better. The Treatment is declared a success!
But the skeptics are not convinced, they say “Perhaps The Treatment is just a placebo, and people would say they felt better if you gave them anything that they thought was medicine”. So the trial is re-run with a control group and a placebo group. The placebo group are given a blank white pill made of sugar with no medicine in it but they are not told they’re in the placebo group (this is known as a blind trial). At the end of the trial, the results for the Treatment group and the control group are the same as before, 80% feel better with The Treatment and only 50% feel better with nothing. But 70% of the placebo group reported feeling better. The treatment is declared a success!
But those pesky skeptics won’t give up. They say “The placebo you used in that trial wasn’t very convincing! The Treatment is actually very elaborate — involving long consultations and complex rituals to apply, the placebo needs to match as closely as possible to The Treatment for the trial to be fair. Also, the doctors administering the placebo can’t know they’re in the placebo group or they might behave differently and give the game away (this would make it a double-blind trial, neither the patient nor the doctors know who is in the placebo group). And while we’re at it, the people gathering the data at the end of the trial mustn’t know which patients were in which group either (which would make this a triple-blind trial).”
The inventors of The Treatment are now running out of money and while they agree to run the triple-blind trial, they insist that this will be the last trial before they release their product. The final trial is run, with an authentic placebo and careful concealment of all the patient groupings. The results after one week are as follows:
Control group (no treatment) — 50% feel better.
Placebo group — 68% feel better.
Treatment group — 72% feel better.
The Treatment is declared a success! The inventors of The Treatment release it to the public and announce that it has no side-effects, is entirely natural, and it may relieve the symptoms of many conditions — further study is required to discover its true potential!
This trial is cited by fans of The Treatment to prove that it works.
The same trial is cited by skeptics as evidence that it doesn’t work.
How can these two groups interpret the same evidence in two different ways?
The believers like to focus on the positive. They note that every time the trial was run, The Treatment was better than everything else. Even in the rigorous triple-blind trial, The Treatment beat the placebo by 4%. They like that it’s all natural, and they feel like when they’re getting The Treatment (which involves a very long consultation with the doctor who will administer The Treatment) they are being looked after and respected — which they don’t feel at all when they visit their GP or a hospital. They don’t mind the high cost of The Treatment, because it is tailored to their specific needs and they feel it works for them. They think that people criticizing The Treatment are being closed-minded. Just because science can’t explain how The Treatment actually operates doesn’t mean it’s fake, there’s more to the world than science can explain — and besides, science hasn’t come up with a cure for The Condition yet anyway, so you may as well take The Treatment because there’s nothing else available!
The skeptics take a different view. They notice that every time the trial was improved, the difference between The Treatment and the placebo group diminished. The conclusion they draw from this is that The Treatment is nothing more than an effective placebo. They feel the believers in The Treatment are being closed-minded for refusing to update their beliefs when evidence shows them to be incorrect.
A year after the final trial, some of the patients who received The Treatment are interviewed. It turns out their symptoms returned later on, and blood tests showed that they still had The Condition. So The Treatment hadn’t actually cured them, it just made them feel better for a while. Believers in The Treatment reject this report, pointing out that it was funded by a big pharmaceutical company that’s trying to develop its own cure for The Condition, and you can’t trust big pharmaceutical companies (except the one that sells The Treatment, you can trust them).
