It is well known that conservatism and caution can result in greater risks under some circumstances. That was the point of my “The risks of caution” and Max’s The Perils of Precaution. However, the message does not seem to be sinking in very well—and instead we have statements like: “Not only do we have to prove that the planned AGI will be friendly, the proof itself has to be short enough to be verifiable by humans”. I don’t think that is a sensible conclusion. It looks more like a naive and unrealistic approach to the issue.
It is well known that conservatism and caution can result in greater risks under some circumstances. That was the point of my “The risks of caution” and Max’s The Perils of Precaution. However, the message does not seem to be sinking in very well—and instead we have statements like: “Not only do we have to prove that the planned AGI will be friendly, the proof itself has to be short enough to be verifiable by humans”. I don’t think that is a sensible conclusion. It looks more like a naive and unrealistic approach to the issue.