(it’d also be pretty surprising to me if dogmatic settlers stayed the same particular flavor of dogmatic for more than a couple thousand years (really more than a few hundred)
What’s preventing AI-powered value lock-in in this scenario? E.g., people telling intent-aligned AIs “keep me faithful to X” as part of competitive virtue/loyalty signaling?
I guess I agree that nonzero of this will happen but I think this requires a kind of weird combination of “strategic about the singularity” and “believing in religious orthodoxy” and that specifically caching out into not evolving at all over time. I expect this to be a pretty small share of the Lightcone given the constraints on who would opt in.
And I think over a billion years people who enforcedly believe in Islam or most equivalent things would probably still find ways to create good beautiful/alien-in-the-expected-good-ways tapestry of posthuman experiences.
Between those I don’t find myself too worked up about it
There maybe can be global rules like “no permacommitments without some degree of awareness of opportunity cost
Can you explain why it requires “strategic about the singularity”? I think as soon as it’s possible to tell one’s AI assistant “keep me faithful to X” (and have the AI do a competent job of this), someone among the billions of religious believers is bound to do it, and then the practice will spread via imitation and competitive signaling (similar to other “tech” for enforcing faith, like “hell for non-believers”). This process does not seem to require anyone being strategic. What am I missing?
What’s preventing AI-powered value lock-in in this scenario? E.g., people telling intent-aligned AIs “keep me faithful to X” as part of competitive virtue/loyalty signaling?
I guess I agree that nonzero of this will happen but I think this requires a kind of weird combination of “strategic about the singularity” and “believing in religious orthodoxy” and that specifically caching out into not evolving at all over time. I expect this to be a pretty small share of the Lightcone given the constraints on who would opt in.
And I think over a billion years people who enforcedly believe in Islam or most equivalent things would probably still find ways to create good beautiful/alien-in-the-expected-good-ways tapestry of posthuman experiences.
Between those I don’t find myself too worked up about it
There maybe can be global rules like “no permacommitments without some degree of awareness of opportunity cost
Can you explain why it requires “strategic about the singularity”? I think as soon as it’s possible to tell one’s AI assistant “keep me faithful to X” (and have the AI do a competent job of this), someone among the billions of religious believers is bound to do it, and then the practice will spread via imitation and competitive signaling (similar to other “tech” for enforcing faith, like “hell for non-believers”). This process does not seem to require anyone being strategic. What am I missing?