“p(Doom) if we build superintelligence soon under present conditions using present techniques”
This has the issue that superintelligence won’t be developed using present techniques, and a lot of the question is about whether it’ll be okay to develop superintelligence using the different techniques that will be used at the time, and so this number will potentially drastically overestimate the risk.
I of course have no problem with asking people questions about situations where the risk is higher or lower than you believe actual risk to be. But I think that focusing on this question will lead to repeated frustrating interactions where one person quotes the answer to the stated question, and then someone else conflates that with their answer to what will happen if we continue on the current trajectory, which for some people is extremely different and for some people isn’t.
I guess same question: “do you think there is a better question here one should ask instead? or do you think people should really stop trying to ask questions like this in a systematic way?”.
This has the issue that superintelligence won’t be developed using present techniques, and a lot of the question is about whether it’ll be okay to develop superintelligence using the different techniques that will be used at the time, and so this number will potentially drastically overestimate the risk.
I of course have no problem with asking people questions about situations where the risk is higher or lower than you believe actual risk to be. But I think that focusing on this question will lead to repeated frustrating interactions where one person quotes the answer to the stated question, and then someone else conflates that with their answer to what will happen if we continue on the current trajectory, which for some people is extremely different and for some people isn’t.
I guess same question: “do you think there is a better question here one should ask instead? or do you think people should really stop trying to ask questions like this in a systematic way?”.