I doubt it’s all that qualitatively different than the sorts of summits humanity has surmounted before
This seems to imply that we have surmounted the fields of physics, that all available knowledge in all subfields has been acquired whereas the most that can be claimed is that we have reduced the degree of our ignorance in some of those subfields. We have not—by any stretch of the imagination—mastered the field. Indeed, if we think we cannot push our understanding of AI, and related alignment problems, further than our current degree of understanding of physics, I think that is a strong point for the “stop everything, while we still can” case.
This seems to imply that we have surmounted the fields of physics, that all available knowledge in all subfields has been acquired whereas the most that can be claimed is that we have reduced the degree of our ignorance in some of those subfields. We have not—by any stretch of the imagination—mastered the field. Indeed, if we think we cannot push our understanding of AI, and related alignment problems, further than our current degree of understanding of physics, I think that is a strong point for the “stop everything, while we still can” case.