There are so, so many versions of an AI future where we wind up on the wrong end of Darwin.
If the agents can mutate/learn, and if more successful agents replicate, then there will be natural selection for the most effective replicators. This would undermine any kind of alignment in much the same way that cancer eventually escapes the body’s defenses.
You don’t want to be on the wrong end of Darwin, especially not in the long run.
There are so, so many versions of an AI future where we wind up on the wrong end of Darwin.
If the agents can mutate/learn, and if more successful agents replicate, then there will be natural selection for the most effective replicators. This would undermine any kind of alignment in much the same way that cancer eventually escapes the body’s defenses.
You don’t want to be on the wrong end of Darwin, especially not in the long run.