What’s sane? That’s socially constructed. What’s good at reproducing, at getting more resources? That is not. Absent some mechanism to keep values from drifting ems will experience massive, swift value drift. This will be the case even if they stay within the space of current human neurological diversity, which they won’t.
The only way the future isn’t a hardscrabble hell is with a friendly Singleton.
I’d say absent some mechanism to keep them sane, an old enough EM couldn’t even be considered intelligent, and would only be a danger to the rest of us in that it’s a resource drain if we can’t stomach euthanasia.
We may or may not naturally have such a mechanism.
By “current human neurological diversity” do you mean staying within what we now consider a sane human (in which case values drift will be limited to what a sane human would value), or do you mean that they’re not modding their mind?
My opinion is the same for if you mean that EMs are modding themselves, except for the part about that there might be a built-in mechanism for keeping them sane.
What’s sane? That’s socially constructed. What’s good at reproducing, at getting more resources? That is not. Absent some mechanism to keep values from drifting ems will experience massive, swift value drift. This will be the case even if they stay within the space of current human neurological diversity, which they won’t.
The only way the future isn’t a hardscrabble hell is with a friendly Singleton.
I’d say absent some mechanism to keep them sane, an old enough EM couldn’t even be considered intelligent, and would only be a danger to the rest of us in that it’s a resource drain if we can’t stomach euthanasia.
We may or may not naturally have such a mechanism.
By “current human neurological diversity” do you mean staying within what we now consider a sane human (in which case values drift will be limited to what a sane human would value), or do you mean that they’re not modding their mind?
My opinion is the same for if you mean that EMs are modding themselves, except for the part about that there might be a built-in mechanism for keeping them sane.