The main paths seem to be that AGI will not actually be that dangerous, or that AGI is lethal against any preparation. If AGI is relatively slow and tops out at collapsing civilization without extinction, I expect there to be quite a lot of time to prepare. I don’t assign any significant probability to AGI being dangerous enough to collapse civilization with little warning, but in such a way that preparing for it now will help.
In principle the best way to prepare for such a scenario would be to invest in AI technology so that I ride the productivity curve upward before the disaster strikes, and have a lot more wealth to start material preparations later—but this would actively make the existential risks worse. I believe increasing one’s personal expected utility at the expense of increasing the risk of human extinction to be massively immoral, and refuse to do that.
So while I do have some preparations for ordinary disasters that almost certainly will happen anyway, these plans do not have much room for AGI disaster.
The main paths seem to be that AGI will not actually be that dangerous, or that AGI is lethal against any preparation. If AGI is relatively slow and tops out at collapsing civilization without extinction, I expect there to be quite a lot of time to prepare. I don’t assign any significant probability to AGI being dangerous enough to collapse civilization with little warning, but in such a way that preparing for it now will help.
In principle the best way to prepare for such a scenario would be to invest in AI technology so that I ride the productivity curve upward before the disaster strikes, and have a lot more wealth to start material preparations later—but this would actively make the existential risks worse. I believe increasing one’s personal expected utility at the expense of increasing the risk of human extinction to be massively immoral, and refuse to do that.
So while I do have some preparations for ordinary disasters that almost certainly will happen anyway, these plans do not have much room for AGI disaster.