I interpret weirdtopia to mean ‘actually good, overall, but in a way that feels horrifying or strange’
This could be a reason for me not to call it a “w-risk”. But this also highlights the slippery nature of some of the boundaries here.
My central idea of a w-risk and a weirdtopia, is that it’s a world where the beings in it are happy, because it’s being optimized/governed according to their values—but those values are not ours, and yet those beings are us, and/or our descendants, after being changed by some process to which we would not have consented beforehand, if we understood its nature.
On the other hand, your definition of weirdtopia could also include futures in which our present values are being satisfied, “but in a way that feels horrifying or strange” if it’s described to us in the present. So it might belong to my fourth category—all risks successfully avoided—and yet we-in-the-present would reject it, at least at first.
This could be a reason for me not to call it a “w-risk”. But this also highlights the slippery nature of some of the boundaries here.
My central idea of a w-risk and a weirdtopia, is that it’s a world where the beings in it are happy, because it’s being optimized/governed according to their values—but those values are not ours, and yet those beings are us, and/or our descendants, after being changed by some process to which we would not have consented beforehand, if we understood its nature.
On the other hand, your definition of weirdtopia could also include futures in which our present values are being satisfied, “but in a way that feels horrifying or strange” if it’s described to us in the present. So it might belong to my fourth category—all risks successfully avoided—and yet we-in-the-present would reject it, at least at first.