Yeah. Though actually it’s more of a simplified version of a more serious problem.
One day you may give AI precise set of instructions, which you think would make good. Like find a way of curing diseases, but without harming patients, and without harming people for the sake of research and so on.
And you may find that your AI is perfectly friendly, but it wouldn’t yet mean it actually is. It may simply have learned human values as a mean of securing its existence and gaining power.
EDIT: And after gaining enough power it may as well help improve human health even more or reprogram human race to think unconditionaly that diseases were eradicated.
Yeah. Though actually it’s more of a simplified version of a more serious problem.
One day you may give AI precise set of instructions, which you think would make good. Like find a way of curing diseases, but without harming patients, and without harming people for the sake of research and so on. And you may find that your AI is perfectly friendly, but it wouldn’t yet mean it actually is. It may simply have learned human values as a mean of securing its existence and gaining power.
EDIT: And after gaining enough power it may as well help improve human health even more or reprogram human race to think unconditionaly that diseases were eradicated.