Yeah, I didn’t mean to imply that causal modeling wasn’t the obvious solution—you’re right about the existence of the leukemia threshold. But I guess in my experience of these mistakes, I often see people taking the action “try to do superior statistical techniques” and that not working for them (including rationalists and not just terrible science reporting sites), whereas I think “identify the places where your model is terrible and call that out” is a better first step for knowing how to build the superior models.
In the ventilator case, for example, I’m not trying to advocate blindly following common sense, but I do think it’s important to incorporate common sense heavily. If people said, “There’s no evidence for respirators working, maybe hemoglobin is being denatured”, I certainly wouldn’t advocate for more common sense. But instead I tend to see “the statistics show respirators aren’t working, maybe we shouldn’t use them”, which seems to imply that common sense isn’t being given a say at all. It seems to me like always having common sense as one of your causal models is both an easy sell and a vital piece of the machine making sure your statistical techniques don’t go off the rails at any of their many opportunities.
Yeah, I didn’t mean to imply that causal modeling wasn’t the obvious solution—you’re right about the existence of the leukemia threshold. But I guess in my experience of these mistakes, I often see people taking the action “try to do superior statistical techniques” and that not working for them (including rationalists and not just terrible science reporting sites), whereas I think “identify the places where your model is terrible and call that out” is a better first step for knowing how to build the superior models.
In the ventilator case, for example, I’m not trying to advocate blindly following common sense, but I do think it’s important to incorporate common sense heavily. If people said, “There’s no evidence for respirators working, maybe hemoglobin is being denatured”, I certainly wouldn’t advocate for more common sense. But instead I tend to see “the statistics show respirators aren’t working, maybe we shouldn’t use them”, which seems to imply that common sense isn’t being given a say at all. It seems to me like always having common sense as one of your causal models is both an easy sell and a vital piece of the machine making sure your statistical techniques don’t go off the rails at any of their many opportunities.