It’s also entirely possible to avoid an AI takeover in the 20-30% of worlds where existing methods can’t scale to superhuman AI (or in the worlds where they do scale but our implementation falls short). In particular we might develop new methods that do solve the problem or coordinate not to build uncontrollable AI.
I was clarifying that I don’t mean “this is just a cakewalk in 70-80% of worlds.” There’s enough failure probability in those worlds that I think the default thing for a technical person to do is try to reduce it.
I generally think we’re going to need to rise to the occasion one way or the other.
It’s also entirely possible to avoid an AI takeover in the 20-30% of worlds where existing methods can’t scale to superhuman AI (or in the worlds where they do scale but our implementation falls short). In particular we might develop new methods that do solve the problem or coordinate not to build uncontrollable AI.
I was clarifying that I don’t mean “this is just a cakewalk in 70-80% of worlds.” There’s enough failure probability in those worlds that I think the default thing for a technical person to do is try to reduce it.
I generally think we’re going to need to rise to the occasion one way or the other.
Thanks, I appreciate your reply!