“no sudden moves; think very carefully before doing anything”—doesn’t that basically amount to an admission that human minds aren’t up to this, that you ought to hurriedly self-improve just to avoid tripping over your own omnipotent feet?
This presents an answer to Eliezer’s “how much self improvement?”: there has to be some point at which the question “what to do” becomes fully determined and further improvement is just re-proving the already proven. So you improve towards that point and stop.
“no sudden moves; think very carefully before doing anything”—doesn’t that basically amount to an admission that human minds aren’t up to this, that you ought to hurriedly self-improve just to avoid tripping over your own omnipotent feet?
This presents an answer to Eliezer’s “how much self improvement?”: there has to be some point at which the question “what to do” becomes fully determined and further improvement is just re-proving the already proven. So you improve towards that point and stop.