Sure, but why would anyone likely build such an AI?
Because computer programs do what they’re programmed to do, without taking into account the actual intention of the user.
Creating an AGI that does take into account what people really want (bearing in mind that the AGI is massively more intelligent than the people wanting the things) is, it seems to me, what the whole Friendly thing is about. If you know how to do that, you’ve solved Friendliness.
Edit: With added complications such as people not knowing what they want, people having conflicting goals, people wanting different things once there’s a powerful AI doing stuff, etc etc
Because computer programs do what they’re programmed to do, without taking into account the actual intention of the user.
Creating an AGI that does take into account what people really want (bearing in mind that the AGI is massively more intelligent than the people wanting the things) is, it seems to me, what the whole Friendly thing is about. If you know how to do that, you’ve solved Friendliness.
Edit: With added complications such as people not knowing what they want, people having conflicting goals, people wanting different things once there’s a powerful AI doing stuff, etc etc