That’s true, but evolution carefully gave us our goals/values to align us with each other.
With the caveat that some humans do have goals and values orthogonal to those of other humans. The result is generally some combination of shunning, exile, imprisonment, killing, fining, or other sanctions, as determined by whichever side has more power to impose its will.
Right—so humans having different goals seems like evidence that AGIs would have even more different goals by default without the evolutionary pressures.
With the caveat that some humans do have goals and values orthogonal to those of other humans. The result is generally some combination of shunning, exile, imprisonment, killing, fining, or other sanctions, as determined by whichever side has more power to impose its will.
Right—so humans having different goals seems like evidence that AGIs would have even more different goals by default without the evolutionary pressures.
Agreed, yes.