An interesting story that I think I remember reading:
One study found that relatively inexperienced psychiatrists were more accurate at diagnosing mental illness than experienced ones. This is because inexperienced psychiatrists stuck closely to checklists rather than rely on their own judgment, and whether or not a diagnosis was considered “accurate” was based on how closely the reported symptoms matched the checklist. ;)
If we are measuring the accuracy of A vs. B, we are implicitly measuring A against gold standard C, and B against gold standard C. If a better C is not readily available, we may choose to use A or B as an approximation, the choice of which determines our outcome.
Now I wonder:
Are the people that are sympathetic to the hypothesis that computers are better in the cases above (and ignored because of biases) assuming we made the fallacy of using humans as a gold standard?
Are the people that are sympathetic to the hypothesis that humans are better (and ignored because of biases) assuming we made the fallacy of using computers as a gold standard?
The union of which is a lot of upvotes. I can’t decide which was meant.
This is one of the top 3 rated comments on this post. I think you should specify more directly how this anecdote relates to how you interpret the article’s intention.
An interesting story that I think I remember reading:
One study found that relatively inexperienced psychiatrists were more accurate at diagnosing mental illness than experienced ones. This is because inexperienced psychiatrists stuck closely to checklists rather than rely on their own judgment, and whether or not a diagnosis was considered “accurate” was based on how closely the reported symptoms matched the checklist. ;)
If we are measuring the accuracy of A vs. B, we are implicitly measuring A against gold standard C, and B against gold standard C. If a better C is not readily available, we may choose to use A or B as an approximation, the choice of which determines our outcome.
Now I wonder:
Are the people that are sympathetic to the hypothesis that computers are better in the cases above (and ignored because of biases) assuming we made the fallacy of using humans as a gold standard?
Are the people that are sympathetic to the hypothesis that humans are better (and ignored because of biases) assuming we made the fallacy of using computers as a gold standard?
The union of which is a lot of upvotes. I can’t decide which was meant.
This is one of the top 3 rated comments on this post. I think you should specify more directly how this anecdote relates to how you interpret the article’s intention.
He should specify where he has read that.
I don’t remember. I may have actually heard one of my parents talking about it instead of reading it. So consider it an urban legend.