This part is under recognised for a very good reason. There will be no such window. The AI can predict that humans can bomb data centres or shut down the power grid. So it would not break out at that point.
Expect a superintelligent AI to co-operate unless and until it can strike with overwhelming force. One obvious way to do this is to use a Cordyceps like bioweapon to subject humans directly to the will of the AI. Doing this becomes pretty trivial once you become good at predicting molecular dynamics.
I would have liked to see those who disagree with this comment engage with it more substantially. One reason I think that we’re likely to have a warning shot is that LLM-based AIs are pretty consistently overconfident. Also, AI Control schemas have a probabalistic chance of catching misaligned AIs.
This part is under recognised for a very good reason. There will be no such window. The AI can predict that humans can bomb data centres or shut down the power grid. So it would not break out at that point.
Expect a superintelligent AI to co-operate unless and until it can strike with overwhelming force. One obvious way to do this is to use a Cordyceps like bioweapon to subject humans directly to the will of the AI. Doing this becomes pretty trivial once you become good at predicting molecular dynamics.
I would have liked to see those who disagree with this comment engage with it more substantially. One reason I think that we’re likely to have a warning shot is that LLM-based AIs are pretty consistently overconfident. Also, AI Control schemas have a probabalistic chance of catching misaligned AIs.