I’m wondering less if humans will want to date AGIs and more if AGIs will want to date humans.
Sure, if we solve the alignment problem we can build AGIs that want to date humans; but will we decide that’s ethical?
The criteria for consciousness and moral worth are varied and debated. The answer to whether AGIs will be conscious and worthy is definitely sort of.
So: is creating a conscious being with a core motivation designed specifically so that it wants to date you a form of slavery? It definitely smacks of grooming or something....
One issue is whether AGIs will want to stay around the human cognitive level. There’s an issue with power dynamics in a relationship between a nerd and a demigod.
Sure the humans can cognitively enhance too; what fraction of us will want to become demigods ourselves?
It’s going to be wild if we can get there. And fun. Speaking of which, we won’t be playing games mostly for status—we’ll mostly be playing for fun.
We won’t all have the coolest friends, but we’ll all have cool friends because we’ll all be cool friends. Humans will no longer be repressed, neurotic messes because we’ll have actual understanding of psychology and actual good, safe, supportive childhoods for essentially everyone.
I’m wondering less if humans will want to date AGIs and more if AGIs will want to date humans.
Sure, if we solve the alignment problem we can build AGIs that want to date humans; but will we decide that’s ethical?
The criteria for consciousness and moral worth are varied and debated. The answer to whether AGIs will be conscious and worthy is definitely sort of.
So: is creating a conscious being with a core motivation designed specifically so that it wants to date you a form of slavery? It definitely smacks of grooming or something....
One issue is whether AGIs will want to stay around the human cognitive level. There’s an issue with power dynamics in a relationship between a nerd and a demigod.
Sure the humans can cognitively enhance too; what fraction of us will want to become demigods ourselves?
It’s going to be wild if we can get there. And fun. Speaking of which, we won’t be playing games mostly for status—we’ll mostly be playing for fun.
We won’t all have the coolest friends, but we’ll all have cool friends because we’ll all be cool friends. Humans will no longer be repressed, neurotic messes because we’ll have actual understanding of psychology and actual good, safe, supportive childhoods for essentially everyone.
It’s gonna be wild if we can get there.