I got the impression from the article that at the beginning, EBM was successful (and made many people in the pharma industry angry) because by using meta-analyses and similar tools it exposed the flaws in existing treatments.
But later the industry adapted, and started producing the kind of fake evidence that can better fool EBM. For example, instead of sponsoring studies, they started sponsoring the meta-analyses; instead of twisting numbers in one experiment, which can be revealed by comparison with other experiments, they started twisting the very comparisons of the experiments. Or instead of doing dubious studies with low numbers of participants, now they are doing dubious studies with huge numbers of participants (to get more weight in the meta-analysis) but still without sharing data and protocols.
In other words, EBM may suffer from the effects of Goodhart’s law. Which doesn’t necessarily mean that it’s completely doomed; but it means that after the initial easy success it will become more difficult because more people will be trying to “hack” it.
Short version of the important parts, please?
I got the impression from the article that at the beginning, EBM was successful (and made many people in the pharma industry angry) because by using meta-analyses and similar tools it exposed the flaws in existing treatments.
But later the industry adapted, and started producing the kind of fake evidence that can better fool EBM. For example, instead of sponsoring studies, they started sponsoring the meta-analyses; instead of twisting numbers in one experiment, which can be revealed by comparison with other experiments, they started twisting the very comparisons of the experiments. Or instead of doing dubious studies with low numbers of participants, now they are doing dubious studies with huge numbers of participants (to get more weight in the meta-analysis) but still without sharing data and protocols.
In other words, EBM may suffer from the effects of Goodhart’s law. Which doesn’t necessarily mean that it’s completely doomed; but it means that after the initial easy success it will become more difficult because more people will be trying to “hack” it.
Basically, that. Also, “medicine is a very large high-inertia thing and trying to move it kinda failed”.