I think there is a disconnect here related to different usages of the word “confidence”. You say in the OP:
he is more extreme in his confidence that things will be ok
Which I would interpret as being 1- P(not-okay), in other words 1 − 0.026 = 97.4% for BB, very confident.
On the other hand, I think many people probably believe that extinction from misaligned AI is very unlikely apriori, and so might use “confidence” in a sense that is relative to priors. To understand why people might do this, lets imagine that I said “there is a 55% chance that irrelevant AI blogger The Floating Droid will win the 2028 US presidential election”. Now imagine someone said “Wow! That’s insanely overconfident!”. I think people would be a bit suspicious if I responded that it was actually pretty unconfident because it quotes a probability near 50%.
I think this different usages of “confidence” is also relevant to the OP since it is reviewing BBs statements. For example, your statement:
Reads to me as a suggestion of hypocrisy or contadiction. BB accuse YS of being overconfident when really he is the one who is being extremely confident! But for this to be the case, we would need to evaluate the statement using the notion of “confidence” that was intended by BB. Its not clear to me that your post is actually using the same notion of “confidence”.
Glad the comment was helpful. I will register my prediction that BB most likely meant the “relative to priors” meaning rather than the one that you use in the OP. I also think among people who aren’t steeped in the background of AI risk, this would be the significantly more common interpretation upon reading what BB wrote.
I think there is a disconnect here related to different usages of the word “confidence”. You say in the OP:
Which I would interpret as being 1- P(not-okay), in other words 1 − 0.026 = 97.4% for BB, very confident.
On the other hand, I think many people probably believe that extinction from misaligned AI is very unlikely apriori, and so might use “confidence” in a sense that is relative to priors. To understand why people might do this, lets imagine that I said “there is a 55% chance that irrelevant AI blogger The Floating Droid will win the 2028 US presidential election”. Now imagine someone said “Wow! That’s insanely overconfident!”. I think people would be a bit suspicious if I responded that it was actually pretty unconfident because it quotes a probability near 50%.
I think this different usages of “confidence” is also relevant to the OP since it is reviewing BBs statements. For example, your statement:
Reads to me as a suggestion of hypocrisy or contadiction. BB accuse YS of being overconfident when really he is the one who is being extremely confident! But for this to be the case, we would need to evaluate the statement using the notion of “confidence” that was intended by BB. Its not clear to me that your post is actually using the same notion of “confidence”.
Hmmm… Good point. I’ll reach out to Bentham’s Bulldog and ask him what he even means by “confidence.” Thanks.
Glad the comment was helpful. I will register my prediction that BB most likely meant the “relative to priors” meaning rather than the one that you use in the OP. I also think among people who aren’t steeped in the background of AI risk, this would be the significantly more common interpretation upon reading what BB wrote.