but I think I was just redefining Friendliness in an incorrect way that Eliezer wouldn’t endorse.
Beware of the urge to censor thoughts that disagree with authority. I personally agree that there is a serious issue here—the issue of moral antirealism, which implies that there is no “canonical human notion of goodness”, so the terminology “Friendly AI” is actually somewhat misleading, and it might be better to say “average human extrapolated morality AGI” when that’s what we want to talk about, e.g.
“an average human extrapolated morality AGI would oppose a paperclip maximizer”.
Then it sounds less onerous to say that you disagree with what an average human extrapolated morality AGI would do than that you disagree with what a “Friendly AI” would do, because most people on this forum disagree with averaged-out human morality (for example, the average human is a theist). Contrast:
“What, you disagree with the FAI? Are you a bad guy then?”
Beware of the urge to censor thoughts that disagree with authority. I personally agree that there is a serious issue here—the issue of moral antirealism, which implies that there is no “canonical human notion of goodness”, so the terminology “Friendly AI” is actually somewhat misleading, and it might be better to say “average human extrapolated morality AGI” when that’s what we want to talk about, e.g.
Then it sounds less onerous to say that you disagree with what an average human extrapolated morality AGI would do than that you disagree with what a “Friendly AI” would do, because most people on this forum disagree with averaged-out human morality (for example, the average human is a theist). Contrast: