The Dogma of Evidence-based Medicine

In po­lite so­ciety it’s cur­rently fash­ion­able to be in fa­vor of Ev­i­dence-based Medicine and pro­claim that we don’t have enough of it. In this ar­ti­cle I want to ar­gue that this prefer­ence isn’t backed up by good rea­sons. The paradigm of Ev­i­dence-based Medicine isn’t backed up by ev­i­dence that proves the virtues of the paradigm but by faith.

What’s Ev­i­dence-based Medicine in the first place? The term was defined in a sci­en­tific pa­per by Guy­att et al in their pa­per “Ev­i­dence-Based Medicine—A New Ap­proach to Teach­ing the Prac­tice of Medicine” in 1992. Ac­cord­ing to the pa­per Ev­i­dence-based medicine re­quires new skills of the physi­cian, in­clud­ing effi­cient liter­a­ture search­ing and the ap­pli­ca­tion of for­mal rules of ev­i­dence eval­u­at­ing the clini­cal liter­a­ture. Ev­i­dence-based Medicine was sup­posed to be about re­place the­ory-based medicine with em­piri­cally-backed medicine.

They make the as­sump­tion that physi­ci­ans who learn the skill of liter­a­ture search­ing and ap­pli­ca­tions of for­mal rules of ev­i­dence will pro­duce bet­ter clini­cal re­sults for their pa­tients. The­o­ret­i­cally there are valid rea­sons why some­one might be­lieve in this as­sump­tion. For a com­mu­nity who sincerely be­liefs in ev­i­dence-based think­ing in­stead of prac­tic­ing be­lief-in-be­lief I would how­ever ex­pect that they test their as­sump­tions.

It would be pos­si­ble to run a con­trol­led study whereby some doc­tors get more classes on learn­ing those effi­cient liter­a­ture search­ing skills and the skills of ap­pli­ca­tion of for­mal rules of ev­i­dence. If the cost of that ex­per­i­ment would be too big, it would even be pos­si­ble to seek cor­re­la­tion ev­i­dence. To my knowl­edge, no­body tried to run ei­ther study.

I opened a ques­tion on Skep­tics.Stack­Ex­change to find out whether any­body could find stud­ies who prove core as­sump­tions of Ev­i­dence-Based Medicine and no­body replied with stud­ies that val­i­dated the idea that teach­ing doc­tors more of those ev­i­dence based skills im­proves pa­tient out­comes.

Brienne Yud­kowsky wrote on Face­book that she thinks that the Ham­ming ques­tion for epistemic ra­tio­nal­ity might be “To which top­ics, or un­der what cir­cum­stances, do you ap­ply differ­ent epistemic laws?“. For many peo­ple medicine is such a field. The ma­jor­ity of sup­posed defen­ders of Ev­i­dence-based Medicine ac­cept with­out ev­i­dence from con­trol­led stud­ies that those Ev­i­dence-based meth­ods of prac­tic­ing medicine are bet­ter. At the same time, they fight al­ter­na­tive medicine paradigms for not pro­vid­ing enough stud­ies that back up their claims.

Ac­cord­ing to the core as­sump­tion in Ev­i­dence-based Medicine re­sults that are found in one pa­tient pop­u­la­tion gen­er­ally gen­er­al­ize to other pa­tients pop­u­la­tions. If that would be true it should be easy to repli­cate stud­ies. In re­al­ity repli­ca­tion of­ten fails even when there’s a lot of at­ten­tion in­vested to get com­pa­rable pa­tient pop­u­la­tions.

In real world clini­cal set­tings the pa­tient pop­u­la­tion is more di­verse than the care­fully cho­sen pa­tient pop­u­la­tion of a trial. In the clini­cal trial pa­tients of­ten only take one drug and while in nor­mal clini­cal prac­tice pa­tients of­ten take mul­ti­ple drugs to fight mul­ti­ple dis­eases.

Another part of the core Ev­i­dence-based Medicine dogma is the du­al­is­tic no­tion that doc­tors should fo­cus on cre­at­ing clini­cal effects for their pa­tients through proper in­ter­ven­tion and not through placebo effects. This means that while pa­tients don’t care if they get bet­ter be­cause of mind or mat­ter, doc­tors are pri­mar­ily fo­cused on the mat­ter. An al­ter­na­tive ther­a­pist who might get clini­cal effects for their pa­tients by spend­ing an hour talk­ing to them get re­jected in fa­vor of a doc­tor who in­ter­views a pa­tient for 5 min­utes and then gives them a pill. Th­ese dog­matic be­liefs about how to think about the placebo effect are also largely formed with­out sci­en­tific in­ves­ti­ga­tion of the placebo effect. There’s a strong dou­ble stan­dard about what kinds of be­liefs need stud­ies to back them up and what can be ac­cepted with­out em­piri­cal ev­i­dence be­cause they make the­o­retic sense.

There’s a be­lief that placebo blind­ing pro­ce­dures gen­er­ally re­sult in pa­tients not know­ing whether or not the the pa­tient got the placebo or verum. Rabkin et al in­ves­ti­gated in their pa­per “How blind is blind?” how well pa­tients can tell what they got. 78% of the pa­tients and 87% of the doc­tors could cor­rectly dis­t­in­guish be­tween placebo and verum when they were asked. A re­search com­mu­nity that would sincerely be­lief in the tenets of ev­i­dence-based medicine would start ask­ing pa­tients in ev­ery trial for their sub­jec­tive be­lief of whether they got placebo. They be­hav­ior of the com­mu­nity we do have that keeps fol­low­ing their es­tab­lished rit­u­als with­out ques­tion­ing those rit­u­als looks more like be­lief-in-be­lief.

One hy­poth­e­sis is that pa­tients know whether they take a placebo or verum be­cause verum has side effects. In an en­vi­ron­ment where the placebo con­trol­led effects of an­tide­pres­sant as Kirsch et al de­scribed in their pa­per only makes on av­er­age 1.8 out of a 50 point scale, there’s the ques­tion whether an­tide­pres­sants with high side-effect un­blind them­selves and are thus bet­ter in di­rect com­par­i­son to an­tide­pres­sants with less side effects. Un­for­tu­nately, the eth­i­cal re­view boards don’t care about those is­sues and rather fo­cus on pre­vent­ing con­sent forms get­ting signed with pen­cils.

There’s one Ev­i­dence-based Medicine be­lief that will look very strange to fu­ture stu­dents who want to make sense of our be­liefs. It’s the be­lief that the blind man sees bet­ter. The be­lief that it’s bad to clearly see the ob­ject un­der in­ves­ti­ga­tion in all it’s de­tails. It’s true that the prac­tice of blind­ing can helps us from fal­ling vic­tim to var­i­ous bi­ases but hav­ing ac­cess to less data also pre­vents us from see­ing real pat­terns. Iron­i­cally, this blindly leads to re­searchers not be­ing in­ter­ested in the sub­jec­tive ex­pe­rience of their pa­tients to the point that they don’t gather data about whether the pa­tients think that they got verum.

Why do we think we need Ev­i­dence-based Medicine in the first place? We don’t want to trust in hu­man au­thor­i­ties. We want sci­ence to free us of the need to trust au­thor­i­ties. In­stead of ask­ing us how we can de­velop jus­tified trust in hu­man au­thor­i­ties, we dream for ob­jec­tive knowl­edge that tran­scends hu­man au­thor­i­ties.

I pro­posed in my post about Pre­dic­tion-based Medicine a sys­tem in which we let doc­tors make pre­dic­tions about the out­comes of their treat­ment and use the qual­ity of those pre­dic­tions to es­tab­lish au­thor­ity. Once we solve the prob­lem of trust the knowl­edge pro­duc­tion it­self can get more di­verse. One sci­en­tists might un­der­stand a dis­ease bet­ter by do­ing phe­nomenolog­i­cal in­ves­ti­ga­tion of the sub­jec­tive ex­pe­rience of pa­tients. Another sci­en­tists might use a lot of sen­sors and run ma­chin­ing learn­ing al­gorithms to bet­ter un­der­stand dis­ease. Both profit if they don’t have to fit in­side the bu­reau­cracy of Ev­i­dence-based Medicine and can fo­cus on pro­duc­ing knowl­edge that helps doc­tors make bet­ter pre­dic­tions about how to treat their pa­tients.

It won’t be as Hah­ne­mann said “Wer heilt, hat Recht” (“He who cures is right”) but “He who can pre­dict in ad­vance that he will cure the pa­tient and then ac­tu­ally cures the pa­tient is right”.