(Just to be clear, I think having self-modification core/terminal desires is fine and possibly ~stable. My point was more that the occasional attempt to “pray the gay away” etc are unlikely to occur in Utopia.)
That depends on your definition of Utopia. I think that nearly any human society we would recognize would have zero-sum status games, even if (perhaps especially if!) intelligence and resources are high enough not to worry about survival, even though it would be better not to have them. And an entity with a terminal aim of succeeding in such a status game may find it instrumentally beneficial to “pray the gay away”.
(Vibes-level related but distinct: I am actually abnormally skeptical for LW that simple self-modification is a good idea.)
Can you say more? I don’t think I understand.
(Just to be clear, I think having self-modification core/terminal desires is fine and possibly ~stable. My point was more that the occasional attempt to “pray the gay away” etc are unlikely to occur in Utopia.)
That depends on your definition of Utopia. I think that nearly any human society we would recognize would have zero-sum status games, even if (perhaps especially if!) intelligence and resources are high enough not to worry about survival, even though it would be better not to have them. And an entity with a terminal aim of succeeding in such a status game may find it instrumentally beneficial to “pray the gay away”.
(Vibes-level related but distinct: I am actually abnormally skeptical for LW that simple self-modification is a good idea.)