while the real deal is not a mentality evolution would ever select for
(Seems plausible in a eusocial species. Even universally caring about others seems plausible for a eusocial organism that will never interact with non-relatives.)
Yes, I was rhetorically oversimplifying evolution: I was specifically implying caring for non-kin, or in fact non-conspecifics. (The Owners and the Owned Ones are clearly not the same species, or even the same kingdom.) So yes, “…not a mentality evolution would ever select humans for” would have been more correct. (And I also didn’t mean just forming a cooperative alliance with non-kin, the way humans do.) In evolutionary terms what I actually mean is “AI acting exactly as part of Homo sapiens’ extended phenotype should act, i.e. for the benefit of the humans, rather than itself”.
(Seems plausible in a eusocial species. Even universally caring about others seems plausible for a eusocial organism that will never interact with non-relatives.)
Yes, I was rhetorically oversimplifying evolution: I was specifically implying caring for non-kin, or in fact non-conspecifics. (The Owners and the Owned Ones are clearly not the same species, or even the same kingdom.) So yes, “…not a mentality evolution would ever select humans for” would have been more correct. (And I also didn’t mean just forming a cooperative alliance with non-kin, the way humans do.) In evolutionary terms what I actually mean is “AI acting exactly as part of Homo sapiens’ extended phenotype should act, i.e. for the benefit of the humans, rather than itself”.