Perhaps I’m misunderstanding you, but “fairly likely...that we still get an AI takeover” and “very good chance… our implementation still falls short” seems at odds with a roughly “20-30% chance...for AI takeover”, no?
It’s also entirely possible to avoid an AI takeover in the 20-30% of worlds where existing methods can’t scale to superhuman AI (or in the worlds where they do scale but our implementation falls short). In particular we might develop new methods that do solve the problem or coordinate not to build uncontrollable AI.
I was clarifying that I don’t mean “this is just a cakewalk in 70-80% of worlds.” There’s enough failure probability in those worlds that I think the default thing for a technical person to do is try to reduce it.
I generally think we’re going to need to rise to the occasion one way or the other.
Perhaps I’m misunderstanding you, but “fairly likely...that we still get an AI takeover” and “very good chance… our implementation still falls short” seems at odds with a roughly “20-30% chance...for AI takeover”, no?
It’s also entirely possible to avoid an AI takeover in the 20-30% of worlds where existing methods can’t scale to superhuman AI (or in the worlds where they do scale but our implementation falls short). In particular we might develop new methods that do solve the problem or coordinate not to build uncontrollable AI.
I was clarifying that I don’t mean “this is just a cakewalk in 70-80% of worlds.” There’s enough failure probability in those worlds that I think the default thing for a technical person to do is try to reduce it.
I generally think we’re going to need to rise to the occasion one way or the other.
Thanks, I appreciate your reply!