I think there are a few assumptions here that don’t quite match what’s being proposed. As I understood it, the manifesto implies that we should spend more resources to solve alignment/control. It also doesn’t imply that the coexistence would necessarily be equal; Yamakawa-sensei acknowledges that there is limited value humanity could provide to an all-mighty superintelligence. However, the manifesto encourages exploring avenues in which humanity could maintain some aspects of symbiosis or flourishment, even in the case that control fails.
One way to think about it could be as finding alternatives to S-risks in worlds where ASI happens and is not entirely aligned nor misaligned, but somewhere in between. Depending on one’s priors, the probability of that happening is not negligible.
I think there are a few assumptions here that don’t quite match what’s being proposed. As I understood it, the manifesto implies that we should spend more resources to solve alignment/control. It also doesn’t imply that the coexistence would necessarily be equal; Yamakawa-sensei acknowledges that there is limited value humanity could provide to an all-mighty superintelligence. However, the manifesto encourages exploring avenues in which humanity could maintain some aspects of symbiosis or flourishment, even in the case that control fails.
One way to think about it could be as finding alternatives to S-risks in worlds where ASI happens and is not entirely aligned nor misaligned, but somewhere in between. Depending on one’s priors, the probability of that happening is not negligible.