The thing in question isn’t collecting barbie dolls, it’s understanding correctly. An .AI that sits at the end of a series of self improvements has got to be pretty good at that.
You can say it will have only instrumental rationality, and will start getting things wrong in order to pursue its ultimate goal of word domination, and I can say that if instrumental rationality is dangerous, don’t bulld it that way.
No, it’s preferences the problem, not understanding. Why would an AI sitting at the end of a series of self improvements choose to interpret ambiguous coding in the way we prefer?
I can say that if instrumental rationality is dangerous, don’t bulld it that way.
How do you propose to build an AI without instrumental rationality or preventing that from developing? And how do you propose to convince AI designers to go down that route?
The thing in question isn’t collecting barbie dolls, it’s understanding correctly. An .AI that sits at the end of a series of self improvements has got to be pretty good at that.
You can say it will have only instrumental rationality, and will start getting things wrong in order to pursue its ultimate goal of word domination, and I can say that if instrumental rationality is dangerous, don’t bulld it that way.
No, it’s preferences the problem, not understanding. Why would an AI sitting at the end of a series of self improvements choose to interpret ambiguous coding in the way we prefer?
How do you propose to build an AI without instrumental rationality or preventing that from developing? And how do you propose to convince AI designers to go down that route?
If it has epistemic rationality as a goal, it will default to getting things right rather than wrong.
Not only nstrumental rationality = epistemic rationality.
If it has epistemic rationality as a goal, it will default to acquiring true beliefs about the world. Explain how this will make it “nice”.
See above. The question was originally about interpreting directives. You have switched to inferring morality apriori.