The moral design question then is, supposing we were to suddenly encounter an extraterrestrial sapient species, do we want our AGIs to be on the human side, or on the all evolved intelligences count equally side?
I’d say something in between. Do I want the AGI to just genocide any aliens it meets on the simple basis that they are not human, so they do not matter? No. Do I want the AGI to stay neutral and refrain from helping us or taking sides were we to meet the Thr’ax Hivemind, Eaters of Life and Bane of the Galaxy, because they too are sapient? Also no. I don’t think there’s an easy question to where we draw the line between “we can find a mutual understanding, so we should try” and “it’s clearly us or them, so let’s make sure it’s us”.
I’d say something in between. Do I want the AGI to just genocide any aliens it meets on the simple basis that they are not human, so they do not matter? No. Do I want the AGI to stay neutral and refrain from helping us or taking sides were we to meet the Thr’ax Hivemind, Eaters of Life and Bane of the Galaxy, because they too are sapient? Also no. I don’t think there’s an easy question to where we draw the line between “we can find a mutual understanding, so we should try” and “it’s clearly us or them, so let’s make sure it’s us”.