Fair enough. I guess I didn’t think carefully about it before. I assumed that s-risks were much less likely than x-risks (true) and so they could be discounted (false). It seems like the right way to imagine the landscape of superintelligences is a vast flat plain (paperclippers and other things that kill everyone without fuss) with a tall thin peak (FAIs) surrounded by a pit that’s astronomically deeper (FAI-adjacent and other designs). The right comparison is between the peak and the pit, because if the pit is more likely, I’d rather have the plain.
Fair enough. I guess I didn’t think carefully about it before. I assumed that s-risks were much less likely than x-risks (true) and so they could be discounted (false). It seems like the right way to imagine the landscape of superintelligences is a vast flat plain (paperclippers and other things that kill everyone without fuss) with a tall thin peak (FAIs) surrounded by a pit that’s astronomically deeper (FAI-adjacent and other designs). The right comparison is between the peak and the pit, because if the pit is more likely, I’d rather have the plain.