(1) have goals, (2) which will be long term, (
In many cases, you also need incorrigibility, and stability under improvement.
I’m not sure I understand; are you saying that given these, we have high P(Doom), or that these are necessary to be safe even if GIs have only short term goals? Or something else entirely?
You need them for high P(doom) because otherwise the AI is corrigible, or stops at the thousandth paperclip.
In many cases, you also need incorrigibility, and stability under improvement.
I’m not sure I understand; are you saying that given these, we have high P(Doom), or that these are necessary to be safe even if GIs have only short term goals? Or something else entirely?
You need them for high P(doom) because otherwise the AI is corrigible, or stops at the thousandth paperclip.