This assumes there is a fact of the matter about whether moral realism is true
I am a well known moral realism / moral antirealism antirealist.
But the sort of moral realism that’s compatible with antirealism is, as far as I see it, a sort of definitional reification of whatever we think of as morality. You can get information from outside yourself about morality, but it’s “boring” stuff like good ethical arguments or transformative life experiences, the same sort of stuff a moral antirealist might be moved by. For the distinction to majorly matter to an AI’s choices—for it to go “Oh, now I have comprehended the inhuman True Morality that tells me to do stuff you think is terrible,” I think we’ve got to have messed up the AI’s metaethics, and we should build a different AI that doesn’t do that.
I am a well known moral realism / moral antirealism antirealist.
But the sort of moral realism that’s compatible with antirealism is, as far as I see it, a sort of definitional reification of whatever we think of as morality. You can get information from outside yourself about morality, but it’s “boring” stuff like good ethical arguments or transformative life experiences, the same sort of stuff a moral antirealist might be moved by. For the distinction to majorly matter to an AI’s choices—for it to go “Oh, now I have comprehended the inhuman True Morality that tells me to do stuff you think is terrible,” I think we’ve got to have messed up the AI’s metaethics, and we should build a different AI that doesn’t do that.