We could broaden our moral circle to recognize that AIs—particularly agentic and sophisticated ones—should be seen as people too. … From this perspective, gradually sharing control over the future with AIs might not be as undesirable as it initially seems.
Is what you’re proposing just complete, advance capitulation to whoever takes over? If so, can I have all your stuff? If you change your values to prioritize me in your moral circle, it might not be as undesirable as it initially seems.
I agree that if we change ourselves to value the welfare of whoever controls the future, then their takeover will be desirable by definition. It’s certainly a recipe for happiness—but then why not just modify your values to be happy with anything at all?
“I expect my view to get more popular over time.”
I agree, except I think it mostly won’t be humans holding this view when it’s popular. Usually whoever takes over is glad they did, and includes themselves in their own moral circle. The question from my point of view is: will they include us in their moral circle? It’s not obvious to me that they will, especially if we ourselves don’t seem to care.
This reminds me of Stewie from Succession: “I’m spiritually and emotionally and ethically and morally behind whoever wins.”
Is what you’re proposing just complete, advance capitulation to whoever takes over? If so, can I have all your stuff? If you change your values to prioritize me in your moral circle, it might not be as undesirable as it initially seems.
I agree that if we change ourselves to value the welfare of whoever controls the future, then their takeover will be desirable by definition. It’s certainly a recipe for happiness—but then why not just modify your values to be happy with anything at all?
I agree, except I think it mostly won’t be humans holding this view when it’s popular. Usually whoever takes over is glad they did, and includes themselves in their own moral circle. The question from my point of view is: will they include us in their moral circle? It’s not obvious to me that they will, especially if we ourselves don’t seem to care.
This reminds me of Stewie from Succession: “I’m spiritually and emotionally and ethically and morally behind whoever wins.”