All of this seems solid, but it seems to me there are two additional considerations that push in the opposite direction:
COVID and its aftermath seem to suggest that pandemics make society dumber and more reactive. I wonder if a surge in bioterror would reduce humanity’s decision-making capability at a critical time?
Releasing highly capable open weights models would seem to increase existential risk by bringing near-SOTA capabilities to more actors. (For example, North Korea is probably incapable of building a frontier model from scratch, but might be able to perform large-scale fine-tuning to obtain a variety of problematic capabilities including but not limited to CBRN development).
All of this seems solid, but it seems to me there are two additional considerations that push in the opposite direction:
COVID and its aftermath seem to suggest that pandemics make society dumber and more reactive. I wonder if a surge in bioterror would reduce humanity’s decision-making capability at a critical time?
Releasing highly capable open weights models would seem to increase existential risk by bringing near-SOTA capabilities to more actors. (For example, North Korea is probably incapable of building a frontier model from scratch, but might be able to perform large-scale fine-tuning to obtain a variety of problematic capabilities including but not limited to CBRN development).