I agree that ‘credence’ and ‘frequency’ are different things. But round here the word ‘probability’ does refer to credence rather than frequency. This isn’t a mistake; it’s just the way we’re using words.
Okay, but I’ve also seen rationalists use point estimates for probability in a way that led them to mess up Bayes, and such that it would be clear if they recognized the probability was uncertain (e.g., I saw this a few times related to covid predictions). I feel like it’s weird to use “frequency” for something that will only happen (or not happen) once, like whether the first AGI will lead to human extinction, though ultimately I don’t really care what word people are using for which concept.
I think it’s less a mistake of using point estimates, but rather not realizing certain probabilities are correlated, so you can’t just multiply them out.
When a coin is spinning, it’s either 100% going to land heads, or 100% going to land tails, I just don’t know which. If I knew every detail of every atom in the room, and had sufficient compute, I could tell up front. So should I use credence instead of probability there?
Probability is a statement about your state of mind and your knowledge. In so far as it’s about frequencies it’s about frequencies from an anthropic perspective—what percentage of minds in the same state of mind as myself are in a world where the coin lands head, and what percentage are in a world where it lands tails.
I agree that ‘credence’ and ‘frequency’ are different things. But round here the word ‘probability’ does refer to credence rather than frequency. This isn’t a mistake; it’s just the way we’re using words.
Okay, but I’ve also seen rationalists use point estimates for probability in a way that led them to mess up Bayes, and such that it would be clear if they recognized the probability was uncertain (e.g., I saw this a few times related to covid predictions). I feel like it’s weird to use “frequency” for something that will only happen (or not happen) once, like whether the first AGI will lead to human extinction, though ultimately I don’t really care what word people are using for which concept.
I think it’s less a mistake of using point estimates, but rather not realizing certain probabilities are correlated, so you can’t just multiply them out.
I’m not sure this distinction is real.
When a coin is spinning, it’s either 100% going to land heads, or 100% going to land tails, I just don’t know which. If I knew every detail of every atom in the room, and had sufficient compute, I could tell up front. So should I use credence instead of probability there?
Probability is a statement about your state of mind and your knowledge. In so far as it’s about frequencies it’s about frequencies from an anthropic perspective—what percentage of minds in the same state of mind as myself are in a world where the coin lands head, and what percentage are in a world where it lands tails.