“people who would reflectively endorse giving control over the future to AI systems right now”.
“right now” is doing a lot of work there. I don’t believe there are any (ok, modulo the lizardman constant) who reflectively want any likely singularity-level change right now. I do believe there is a very wide range of believable far-mode preferences and acceptances of the trajectory of intelligent life beyond the immediate.
It’s actually “reflectively endorse” that is doing even more work, I think. Think about the people who fall in love with AI, who believe they have made AI conscious by prompting it correctly, etc. Of course there are people who think that AI right now should replace humanity.
“right now” is doing a lot of work there. I don’t believe there are any (ok, modulo the lizardman constant) who reflectively want any likely singularity-level change right now. I do believe there is a very wide range of believable far-mode preferences and acceptances of the trajectory of intelligent life beyond the immediate.
It’s actually “reflectively endorse” that is doing even more work, I think. Think about the people who fall in love with AI, who believe they have made AI conscious by prompting it correctly, etc. Of course there are people who think that AI right now should replace humanity.