This seems like a Procrustean-bed attempt at forcing disparate areas of human endeavor into a single reference class of “fields of expertise” in order to use an outside view to evaluate the correlation between expert consensus and trustworthiness. It is understandably tempting, since, if successful, it would give an outsider a tool to evaluate the reliability of the experts’ statements. Unfortunately, these “fields of expertise” are vastly different. Some use the scientific method, like biology and environmental sciences, and maybe economics (?), others are sometimes not even classified as science (like philosophy or history), yet others are best categorized as an exercise in biases and meme propagation, like this biblical scholarship thing.
Not surprisingly, while one can confidently predict that expert opinions are unreliable when there is little consensus, there is no easy way to evaluate reliability from the outside view only when consensus is high, since it can stem from unchecked shared biases as easily as from the earnest application of scientific research.
My impression is that bias sometimes produces majority opinions, sometimes even fairly large majorities in the 70-80% range, but rarely produces anything like >95% majorities, and when it does the causes are going to be obvious like religion and nationalism. Do you have a counter-example?
I don’t have the stats to make one, sorry. The idea of continental drift was shunned by the experts for a long time. It took decades for astronomers to start taking black holes seriously. The germ theory of disease was not an instant hit, either. The heliocentric view. Troy being a real place. I am not sure about percentages in any of these cases.
Was the reluctance of scientists to accept these ideas the result of bias? Do you think non-experts of the time could have figured out these ideas were correct?
In the case of the heliocentric view, it would have been hard for anybody to find evidence for the correct view prior to the observations of Tycho Brahe. The earlier observations were too imprecise and error-filled to be much help in distinguishing between the leading hypotheses; Copernicus wasn’t notably better or worse at explaining the older data than Ptolemy. But Brahe’s far superior data showed that they both had to be wrong, and Kepler’s heliocentric view was the only theory that managed to fit Brahe’s much more precise observations (and Copernicus got perhaps undeserved retroactive credit for also being heliocentric, even though he got the details quite wrong). I’m not as familiar with the others, but at least in the case of Troy and black holes I can see plenty of legitimate justification for the experts being skeptical before really strong new evidence emerged.
I’d go farther and note that Kepler was the real iconoclast, in that he didn’t simply throw more epicycles into either the Copernican or the Ptolemaic model.
The earlier observations were too imprecise and error-filled
Do you have a quantitative statement of the error bars on normal (ie, not Brahe) astronomical data? I have heard it claimed that Ptolemy’s data is systematically biased in favor of his model, which is only meaningful if the error bars are small enough that such data could distinguish models. (though there’s also the issue of whether anyone understood error bars)
IAWYC, but note that Copernicus pointed out a real problem with the existing Aristotelian theory. The physics (or natural philosophy) underlying Ptolemy said that the minds associated with the heavens, by contemplating perfect activity, moved in regular circles. If you try to take these motions literally, it doesn’t work—you need a non-circular motion to turn the “equant” into a mechanical explanation. While we could easily hand-wave this, it still made sense to look for other explanations. So Copernicus had something besides pure luck going for him.
This seems like a Procrustean-bed attempt at forcing disparate areas of human endeavor into a single reference class of “fields of expertise” in order to use an outside view to evaluate the correlation between expert consensus and trustworthiness. It is understandably tempting, since, if successful, it would give an outsider a tool to evaluate the reliability of the experts’ statements. Unfortunately, these “fields of expertise” are vastly different. Some use the scientific method, like biology and environmental sciences, and maybe economics (?), others are sometimes not even classified as science (like philosophy or history), yet others are best categorized as an exercise in biases and meme propagation, like this biblical scholarship thing.
Not surprisingly, while one can confidently predict that expert opinions are unreliable when there is little consensus, there is no easy way to evaluate reliability from the outside view only when consensus is high, since it can stem from unchecked shared biases as easily as from the earnest application of scientific research.
My impression is that bias sometimes produces majority opinions, sometimes even fairly large majorities in the 70-80% range, but rarely produces anything like >95% majorities, and when it does the causes are going to be obvious like religion and nationalism. Do you have a counter-example?
I don’t have the stats to make one, sorry. The idea of continental drift was shunned by the experts for a long time. It took decades for astronomers to start taking black holes seriously. The germ theory of disease was not an instant hit, either. The heliocentric view. Troy being a real place. I am not sure about percentages in any of these cases.
Was the reluctance of scientists to accept these ideas the result of bias? Do you think non-experts of the time could have figured out these ideas were correct?
In the case of the heliocentric view, it would have been hard for anybody to find evidence for the correct view prior to the observations of Tycho Brahe. The earlier observations were too imprecise and error-filled to be much help in distinguishing between the leading hypotheses; Copernicus wasn’t notably better or worse at explaining the older data than Ptolemy. But Brahe’s far superior data showed that they both had to be wrong, and Kepler’s heliocentric view was the only theory that managed to fit Brahe’s much more precise observations (and Copernicus got perhaps undeserved retroactive credit for also being heliocentric, even though he got the details quite wrong). I’m not as familiar with the others, but at least in the case of Troy and black holes I can see plenty of legitimate justification for the experts being skeptical before really strong new evidence emerged.
I’d go farther and note that Kepler was the real iconoclast, in that he didn’t simply throw more epicycles into either the Copernican or the Ptolemaic model.
Do you have a quantitative statement of the error bars on normal (ie, not Brahe) astronomical data? I have heard it claimed that Ptolemy’s data is systematically biased in favor of his model, which is only meaningful if the error bars are small enough that such data could distinguish models. (though there’s also the issue of whether anyone understood error bars)
IAWYC, but note that Copernicus pointed out a real problem with the existing Aristotelian theory. The physics (or natural philosophy) underlying Ptolemy said that the minds associated with the heavens, by contemplating perfect activity, moved in regular circles. If you try to take these motions literally, it doesn’t work—you need a non-circular motion to turn the “equant” into a mechanical explanation. While we could easily hand-wave this, it still made sense to look for other explanations. So Copernicus had something besides pure luck going for him.
But Copernicus also tried to make the heavenly bodies move in circles. Kepler was the one who finally threw that mistake out.