″ that doesn’t make their strongest beliefs the most trustworthy—indeed, they’re the ones we ought to downgrade the most”
Not likely. Their strongest beliefs will be the most trustworthy, even though they are downgraded the most, because they start start out higher. It would be a very unlikely calibration graph indeed which assigned a lower probability to 99.9% assigned probabilities than to 95% assigned probabilities.
Maybe. My initial response was to consider editing my wording to redact the implication you’re objecting to.
But, it seems pretty plausible to me that even someone who works hard to be calibrated has many coexisting heuristics with different calibration graphs of their own, so that it’s quite likely that beliefs at the 99.9% level are different in kind from beliefs at the 95% level, because the mental process which would spit out such a high confidence is different. Then, why expect calibration around 99.5%, based on calibration around 95%?
And if someone cites million-to-one confidence, then yeah, something is probably up with that! Maybe they’ve actually got millions of samples to generalize from, but even so, are they generalizing correctly? It seems like there is some reason to expect the graph to dip a bit at the end.
″ that doesn’t make their strongest beliefs the most trustworthy—indeed, they’re the ones we ought to downgrade the most”
Not likely. Their strongest beliefs will be the most trustworthy, even though they are downgraded the most, because they start start out higher. It would be a very unlikely calibration graph indeed which assigned a lower probability to 99.9% assigned probabilities than to 95% assigned probabilities.
Maybe. My initial response was to consider editing my wording to redact the implication you’re objecting to.
But, it seems pretty plausible to me that even someone who works hard to be calibrated has many coexisting heuristics with different calibration graphs of their own, so that it’s quite likely that beliefs at the 99.9% level are different in kind from beliefs at the 95% level, because the mental process which would spit out such a high confidence is different. Then, why expect calibration around 99.5%, based on calibration around 95%?
And if someone cites million-to-one confidence, then yeah, something is probably up with that! Maybe they’ve actually got millions of samples to generalize from, but even so, are they generalizing correctly? It seems like there is some reason to expect the graph to dip a bit at the end.