I’ve similarly griped here in the past about the mistaken ways medical tests are analyzed here and elsewhere, but I think you over complicated things.
The fundamental error is misinterpreting a failure to reject a null hypothesis for a particular statistical test, a particular population, and a particular treatment regime as a generalized demonstration of the null hypothesis that the medication “doesn’t work”. And yes, you see it very often, and almost universally in press accounts.
You make a good point about how modeling response = effect + error leads to confusion. I think the mistake is clearer written as “response = effect + noise”, where noise is taken as a random process injecting ontologically inscrutable perturbations of the response. If you start with the assumption that all differences from the mean effect are due to ontologically inscrutable magic, you’ve ruled out any analysis of that variation by construction.
I’ve similarly griped here in the past about the mistaken ways medical tests are analyzed here and elsewhere, but I think you over complicated things.
The fundamental error is misinterpreting a failure to reject a null hypothesis for a particular statistical test, a particular population, and a particular treatment regime as a generalized demonstration of the null hypothesis that the medication “doesn’t work”. And yes, you see it very often, and almost universally in press accounts.
You make a good point about how modeling response = effect + error leads to confusion. I think the mistake is clearer written as “response = effect + noise”, where noise is taken as a random process injecting ontologically inscrutable perturbations of the response. If you start with the assumption that all differences from the mean effect are due to ontologically inscrutable magic, you’ve ruled out any analysis of that variation by construction.