There are really good possible worlds (e.g. a happy billion-year future), and really bad possible worlds (e.g. a billion-year “torture world”). Compared to these, a world in which humans go extinct and are replaced by paperclip maximizers is just mildly bad (or maybe even mildly good, e.g. if we see some value in the scientific and technological progress of the paperclip maximizers).
If the really good worlds are just too hard to bring about (since they require that the problem of human alignment is solved), perhaps people should focus on deliberately bringing about the mildly good/bad worlds (what you call “neutral worlds”), in order to avoid the really bad outcomes.
In other words, Clippy’s Modest Proposal boils down to embracing x-risk in order to avoid s-risk?
Summarizing your idea:
There are really good possible worlds (e.g. a happy billion-year future), and really bad possible worlds (e.g. a billion-year “torture world”). Compared to these, a world in which humans go extinct and are replaced by paperclip maximizers is just mildly bad (or maybe even mildly good, e.g. if we see some value in the scientific and technological progress of the paperclip maximizers).
If the really good worlds are just too hard to bring about (since they require that the problem of human alignment is solved), perhaps people should focus on deliberately bringing about the mildly good/bad worlds (what you call “neutral worlds”), in order to avoid the really bad outcomes.
In other words, Clippy’s Modest Proposal boils down to embracing x-risk in order to avoid s-risk?
That’s the gist of it.