I generally don’t think it’s a good idea to put a probability on things where you have a significant ability to decide the outcome (i.e. probability of getting divorced), and instead encourage you to believe in pausing.
In this case, I can at least talk about the probability of a multi decade pause (with the motivation of delaying AI etc) if I were to be hit by a bus tomorrow. My number is unchanged, around 3%. (Maybe there are some good arguments for higher, I’m not sure.)
I agree that if everyone in my decision-theoretic reference class stopped trying to pause AI (perhaps because of being hit by buses), the chance of a pause is near 0.
In this case, I can at least talk about the probability of a multi decade pause (with the motivation of delaying AI etc) if I were to be hit by a bus tomorrow. My number is unchanged, around 3%. (Maybe there are some good arguments for higher, I’m not sure.)
I agree that if everyone in my decision-theoretic reference class stopped trying to pause AI (perhaps because of being hit by buses), the chance of a pause is near 0.