Indeed. The problem of driving has been set up by car designers so that it should be easy for us to solve it. By contrast, the problem of creating a safe AGI has not been set up so that it should be easy for us to solve (if you don’t believe that there is a God who has made this the best of all possible worlds that is...). So I don’t think the analogy works. If you want to make an argument making use of human past performance, it would be better to use examples of problems (that we’ve solved) that weren’t set up by us (e.g. moon-landing, relativity theory, computers, science in general, etc).
(e.g. moon-landing, relativity theory, computers, science in general, etc).
Or nuclear weapon design. Chicago Pile-1 did work. Trinity did work. Little Boy did work. Burster-Able failed—but not catastrophically. Who knows if whatever the North Koreans cobbled together worked as intended—but it doesn’t seem to have destroyed anything it wasn’t supposed to. No-one has yet accidentally blown up a city. That’s something. Anyway, I’ll edit the post.
I don’t think it is pure chance, since it was designed in iterations around human capabilites.
Indeed. The problem of driving has been set up by car designers so that it should be easy for us to solve it. By contrast, the problem of creating a safe AGI has not been set up so that it should be easy for us to solve (if you don’t believe that there is a God who has made this the best of all possible worlds that is...). So I don’t think the analogy works. If you want to make an argument making use of human past performance, it would be better to use examples of problems (that we’ve solved) that weren’t set up by us (e.g. moon-landing, relativity theory, computers, science in general, etc).
Or nuclear weapon design. Chicago Pile-1 did work. Trinity did work. Little Boy did work. Burster-Able failed—but not catastrophically. Who knows if whatever the North Koreans cobbled together worked as intended—but it doesn’t seem to have destroyed anything it wasn’t supposed to. No-one has yet accidentally blown up a city. That’s something. Anyway, I’ll edit the post.