As I understand the terminology, AI that only respects some humans’ preferences is uFAI by definition. Thus:
a friendly AI that is created in some researcher’s personal image
is actually unFriendly, as Eliezer uses the term. Thus, the researcher you describe is already an “uFAI researcher”
It also removes the problem of different morals/values. Some people believe that we should look at total utility, other people believe we should consider only average utility. Some people believe abstract values matter, some people believe consequences of actions matter most. Here too the solution of an AI that looks at a representative set of all human values is the solution that all people can agree on as most “fair”.
What do you mean by “representative set of all human values”? Is there any reason to that the resulting moral theory would be acceptable to implement on everyone?
[a “friendly” AI] is actually unFriendly, as Eliezer uses the term
Absolutely. I used “friendly” AI (with scare quotes) to denote it’s not really FAI, but I don’t know if there’s a better term for it. It’s not the same as uFAI because Eliezer’s personal utopia is not likely to be valueless by my standards, whereas a generic uFAI is terrible from any human point of view (paperclip universe, etc).
I guess it just doesn’t bother me that uFAI includes both indifferent AI and malicious AI. I honestly think that indifferent AI is much more likely than malicious (Clippy is malicious, but awfully unlikely), but that’s not good for humanity’s future either.
As I understand the terminology, AI that only respects some humans’ preferences is uFAI by definition. Thus:
is actually unFriendly, as Eliezer uses the term. Thus, the researcher you describe is already an “uFAI researcher”
What do you mean by “representative set of all human values”? Is there any reason to that the resulting moral theory would be acceptable to implement on everyone?
Absolutely. I used “friendly” AI (with scare quotes) to denote it’s not really FAI, but I don’t know if there’s a better term for it. It’s not the same as uFAI because Eliezer’s personal utopia is not likely to be valueless by my standards, whereas a generic uFAI is terrible from any human point of view (paperclip universe, etc).
I guess it just doesn’t bother me that uFAI includes both indifferent AI and malicious AI. I honestly think that indifferent AI is much more likely than malicious (Clippy is malicious, but awfully unlikely), but that’s not good for humanity’s future either.