If you meet a maximizer, open fire in lieu of “hello”.
Which is why a “Friendly” AI needs to be a meta-maximizer, rather than a mere first-order maximizer. In order for an AI to be “friendly”, it needs to recognize a set of beings whose utility functions it wishes to maximize, as the inputs to its own utility function.
Which is why a “Friendly” AI needs to be a meta-maximizer, rather than a mere first-order maximizer. In order for an AI to be “friendly”, it needs to recognize a set of beings whose utility functions it wishes to maximize, as the inputs to its own utility function.