The message is really that the impact of a mistake has to be assessed in light of the entire body of evidence supporting the experts’ view. Sometimes a single mistake really does undermine a case, but mostly not.
I don’t think the heuristic I’m advocating is terribly prone to abuse:
If a statistician and an expert on [other field] disagree on [other field], I advocate tending to side with the expert on [other field], especially if there’s a strong consensus among experts on [other field] on that point.
If couse when a statistician and an expert on [other field] disagree on statistics, I advocate trusting the statistician.
Maybe I should add tack this on to the end of the post?
Your two points easily combine into one: when two experts in different fields disagree, you should trust the expert in whose field of expertise the point of contention is. That’s not a particularly new piece of advice (and I personally am suspicious of its generality).
No, it’s not particularly novel. The novel thing is warning against thinking that becasue stats is used in so many other disciplines, stats is an exception.
Well, that does depend on “in whose field of expertise the point of contention is”. If a professional statistician says that a particular study screwed up its statistical analysis, I have little reason (ceteris paribus) to disbelieve him. That, of course, doesn’t mean the paper should go into the wastebasket, maybe it just needs a minor correction—but it’s very hard to generalize about such things.
Certainly, when statisticians start to talk about, say, biology, you should be suspicious—but no more than you should be suspicious of climate scientists talking about economics.
Thank you for taking our ability to dismiss experts we don’t like up one meta-level.
The message is really that the impact of a mistake has to be assessed in light of the entire body of evidence supporting the experts’ view. Sometimes a single mistake really does undermine a case, but mostly not.
I don’t think the heuristic I’m advocating is terribly prone to abuse:
If a statistician and an expert on [other field] disagree on [other field], I advocate tending to side with the expert on [other field], especially if there’s a strong consensus among experts on [other field] on that point.
If couse when a statistician and an expert on [other field] disagree on statistics, I advocate trusting the statistician.
Maybe I should add tack this on to the end of the post?
Your two points easily combine into one: when two experts in different fields disagree, you should trust the expert in whose field of expertise the point of contention is. That’s not a particularly new piece of advice (and I personally am suspicious of its generality).
No, it’s not particularly novel. The novel thing is warning against thinking that becasue stats is used in so many other disciplines, stats is an exception.
Well, that does depend on “in whose field of expertise the point of contention is”. If a professional statistician says that a particular study screwed up its statistical analysis, I have little reason (ceteris paribus) to disbelieve him. That, of course, doesn’t mean the paper should go into the wastebasket, maybe it just needs a minor correction—but it’s very hard to generalize about such things.
Certainly, when statisticians start to talk about, say, biology, you should be suspicious—but no more than you should be suspicious of climate scientists talking about economics.