tangential comment: Regarding “I will define success as producing fission weapons before the end of war in Europe”. I’m not sure if this is the right criterion for success for the purpose of analogizing to AGI. It seems to me that “producing fission weapons before an Axis power does” is more appropriate.
And this seems overwhelmingly the case, yes: “theory of atomic bomb was considerably more advanced at the beginning of Manhattan project compared to our understanding of theory of aligned AGI”
tangential comment: Regarding “I will define success as producing fission weapons before the end of war in Europe”. I’m not sure if this is the right criterion for success for the purpose of analogizing to AGI. It seems to me that “producing fission weapons before an Axis power does” is more appropriate.
And this seems overwhelmingly the case, yes: “theory of atomic bomb was considerably more advanced at the beginning of Manhattan project compared to our understanding of theory of aligned AGI”