The past is a foreign country. Look upon its works and despair.
From the perspective of human civilization of, say, three centuries ago, present-day humanity is clearly a superintelligence.
In any domain they would have considered important, we retain none of the values of that time. They tried to align our values to theirs, and failed abysmally.
With so few reasonable candidates for past superintelligences, reference-class forecasting the success of AI alignment looks bleak.
The past is a foreign country. Look upon its works and despair.
From the perspective of human civilization of, say, three centuries ago, present-day humanity is clearly a superintelligence.
In any domain they would have considered important, we retain none of the values of that time. They tried to align our values to theirs, and failed abysmally.
With so few reasonable candidates for past superintelligences, reference-class forecasting the success of AI alignment looks bleak.