And yet if they all switched to being cliquebots they would not only drive out the defectbots, but would have far more children in the long run.
And if they all switched to being paperclip maximisers they would make more paperclips. Neither of these are going to help them maximise their actual utility function.
And if they all switched to being paperclip maximisers they would make more paperclips. Neither of these are going to help them maximise their actual utility function.
Fair point. I failed to spot initially that the trick lay in equivocating between “maximise direct descendants” and “maximise all descendants”.