Yeah, at least “literally everyone dies” has a concrete ending even though it doesn’t have concrete intermediate concrete steps. Gradual disempowerment seem less concrete on both the ending and the intermediate steps, so it becomes even less action-relevant.
Permanent disempowerment I’m talking about is distinct from “gradual disempowerment”. The latter is a particular way in which a state of disempowerment might get established at some point, but reaching that state could also be followed by literal extinction, so it won’t be permanent or distinct from the extinction outcomes, and it could also be reached in other ways. Indeed the rapid RSI AGIs-to-ASIs story suggests an abrupt takeover rather than gradual systemic change caused by increasing dependence of humanity on AIs.
Yeah, at least “literally everyone dies” has a concrete ending even though it doesn’t have concrete intermediate concrete steps. Gradual disempowerment seem less concrete on both the ending and the intermediate steps, so it becomes even less action-relevant.
Permanent disempowerment I’m talking about is distinct from “gradual disempowerment”. The latter is a particular way in which a state of disempowerment might get established at some point, but reaching that state could also be followed by literal extinction, so it won’t be permanent or distinct from the extinction outcomes, and it could also be reached in other ways. Indeed the rapid RSI AGIs-to-ASIs story suggests an abrupt takeover rather than gradual systemic change caused by increasing dependence of humanity on AIs.