One method of solving the problem is looking at empirical performance on objective metrics. For example, we can test different healing methods using RCTs and see which actually work. Or, if “beauty” is defined as “something most people consider beautiful”, we can compare designers by measuring how their designs are rated by large groups of people. Of course, if such evidence is not already available, then producing it is usually expensive. But, this is something money can buy, at least in principle. Then again, it requires the arbiter to at least understand how empirical evidence works. Louis XV, to eir misfortune, did not know about RCTs.
Agree that empirical performance is a very important way to assess experts.
Unfortunately it can be tricky. In the RCT example, you need expertise to be able to evaluate the RCT. It’s not just about knowing about their existence, but also you’d need to be able to eg avoid p-hacking, file-drawer effects and other methodological issues. Especially in a high stakes adversarial landscape like national politics. Joe Biden himself doesn’t have enough expertise to assess empirical performance using RCTs. And it’s unclear if even any of his advisors can.
Louis XV, to eir misfortune, did not know about RCTs.
But if snake oil was poisonous, and it was known that anyone who ingests it dies immediately afterward, then this information would be of value to Lous XV, in ignoring/banning at least one sort of pseudo-physician.
One method of solving the problem is looking at empirical performance on objective metrics. For example, we can test different healing methods using RCTs and see which actually work. Or, if “beauty” is defined as “something most people consider beautiful”, we can compare designers by measuring how their designs are rated by large groups of people. Of course, if such evidence is not already available, then producing it is usually expensive. But, this is something money can buy, at least in principle. Then again, it requires the arbiter to at least understand how empirical evidence works. Louis XV, to eir misfortune, did not know about RCTs.
Agree that empirical performance is a very important way to assess experts.
Unfortunately it can be tricky. In the RCT example, you need expertise to be able to evaluate the RCT. It’s not just about knowing about their existence, but also you’d need to be able to eg avoid p-hacking, file-drawer effects and other methodological issues. Especially in a high stakes adversarial landscape like national politics. Joe Biden himself doesn’t have enough expertise to assess empirical performance using RCTs. And it’s unclear if even any of his advisors can.
But if snake oil was poisonous, and it was known that anyone who ingests it dies immediately afterward, then this information would be of value to Lous XV, in ignoring/banning at least one sort of pseudo-physician.