‘Unexpected things happen all the time’ isn’t necessarily a reason to be less surprised by specific especially unexpected things. The reason Eliezer’s post isn’t crazy super surprising isn’t that surprising things are (surprisingly?) common; it’s that it’s relatively ordinary (in-character, etc.) for its reference class.
Except surprising things are surprisingly common. Most people overestimate the likelihood that their model is correct.
But this doesn’t seem like a great example of that, yeah. I was sort of pattern-matching this into the wider discussion (is it worth figuring out if the earth will crash into the sun?)
‘Unexpected things happen all the time’ isn’t necessarily a reason to be less surprised by specific especially unexpected things. The reason Eliezer’s post isn’t crazy super surprising isn’t that surprising things are (surprisingly?) common; it’s that it’s relatively ordinary (in-character, etc.) for its reference class.
Except surprising things are surprisingly common. Most people overestimate the likelihood that their model is correct.
But this doesn’t seem like a great example of that, yeah. I was sort of pattern-matching this into the wider discussion (is it worth figuring out if the earth will crash into the sun?)