Yeah, in one sense this question seems impossible to answer—“help me comprehend something incomprehensible to me by definition.”
But there’s another type of answer; of the utility functions that are alien in OP’s sense, it is possible that most will share patterns discernible by humans. OP could be asking what those patterns are.
I’m not sure how worthwhile it is to try to predict the high-level behavior of a generic superintelligence whose goals we don’t understand.
Yeah, in one sense this question seems impossible to answer—“help me comprehend something incomprehensible to me by definition.”
But there’s another type of answer; of the utility functions that are alien in OP’s sense, it is possible that most will share patterns discernible by humans. OP could be asking what those patterns are.
I’m not sure how worthwhile it is to try to predict the high-level behavior of a generic superintelligence whose goals we don’t understand.