Posting a brief exchange between myself and Sam Winter-Levy on this topic, relating to this piece of his.
Sam
Hi Tom,
Sam Winter-Levy here—I’m a fellow at the Carnegie Endowment for International Peace in DC. Really interesting piece on economic growth explosions. It overlaps with some work we’re doing at Carnegie on “decisive strategic advantages,” just wanted to share some quick reactions in case helpful!
I know you purposefully set aside military considerations, but I’m not sure I agree with the premise that outgrowing the world is a path to a “decisive strategic advantage without acting aggressively towards other countries.” In particular, I think that so long as nuclear deterrence and MAD hold, they are a binding obstacle to any alleged DSA, no matter how much AI turbocharges economic growth. Today, the US economy is 15x Russia’s and 1000x North Korea’s, but the US has no ability to impose “complete domination” over either of them, to put it mildly, nor would it even if AI dramatically boosted the US economy still further—in large part because of Russia and North Korea’s ability to credibly threaten to impose massive costs on the US through nuclear escalation on any issue of sufficient importance to either regime.
More broadly, I don’t think most claims about the existence of a DSA take nuclear deterrence seriously enough. Before nuclear weapons existed, a state’s ability to defend itself from coercion depended crucially on its economic size, since relative forces mattered and force size/quality depended ultimately on having a large population and an advanced economy. But that’s not true once you have a second-strike nuclear capability; nuclear weapons break the connection you emphasize between economic power and the ability to impose your political preferences on other nuclear-armed states. So I think even granting the argument that AGI could lead to some sort of economic explosion for one state, the amount of coercive leverage that would grant that state is likely to be highly limited, nowhere close to the Bostrom DSA definition you use, so long as nuclear deterrence remains intact.
In case of interest, here’s a piece we published recently on these issues, coauthored with Nikita Lalwani (former director for tech/nat sec on the Biden NSC). We make a version of the above argument about the importance of MAD for the existence of any alleged DSA, and then lay out how exactly AGI might threaten nuclear deterrence. Would welcome any feedback! We’re still tentatively thinking through these DSA issues ourselves.
Best,
Sam
Tom
Hi Sam,
Thanks for this great pushback!
I think you’re right that this is an oversight of the piece. But here are some responses:
In terms of controlling civilisation’s longrun future, outgrowing the world in the way we describe would be enough. We’ll eventually settle space and North Korea/Russia will be the tiniest fraction of total resources/‘land throughout space’. So if US controls everything but some countries on earth, they completely dominate the longrun future. Agree though the DSA definition i used is a bit different to that
If a country gains this kind of massive economic advantage, it will gain a massive tech advantage as well that, at some point, is likely to allow it to remove the second strike capability. This is more speculative, but we are here talking about the limits of technology and 1-month doubling nanobots etc. Still, this at least should be addressed by the post.
If a country gains this kind of massive economic advantage, it will gain a massive tech advantage as well that, at some point, is likely to allow it to remove the second strike capability. This is more speculative, but we are here talking about the limits of technology and 1-month doubling nanobots etc. Still, this at least should be addressed by the post.
I would go further, and say that if we condition on the US growing to become 70-99% of world GDP and growing super-exponentially for a time, then the fact that it can become a superpower in space pretty much neutralizes MAD, because it’s industry is by and large off-world, meaning that a nuclear attack on the US 50 years after the SIE ended is basically a non-factor, because lots of US people will be in space and able to annihilate North Korea (as an example).
And there’s no nuclear weapons equivalent outside of something like vacuum decay, which is very, very far off from our current state, such that we don’t need to worry about it.
Posting a brief exchange between myself and Sam Winter-Levy on this topic, relating to this piece of his.
Sam
Hi Tom,
Sam Winter-Levy here—I’m a fellow at the Carnegie Endowment for International Peace in DC. Really interesting piece on economic growth explosions. It overlaps with some work we’re doing at Carnegie on “decisive strategic advantages,” just wanted to share some quick reactions in case helpful!
I know you purposefully set aside military considerations, but I’m not sure I agree with the premise that outgrowing the world is a path to a “decisive strategic advantage without acting aggressively towards other countries.” In particular, I think that so long as nuclear deterrence and MAD hold, they are a binding obstacle to any alleged DSA, no matter how much AI turbocharges economic growth. Today, the US economy is 15x Russia’s and 1000x North Korea’s, but the US has no ability to impose “complete domination” over either of them, to put it mildly, nor would it even if AI dramatically boosted the US economy still further—in large part because of Russia and North Korea’s ability to credibly threaten to impose massive costs on the US through nuclear escalation on any issue of sufficient importance to either regime.
More broadly, I don’t think most claims about the existence of a DSA take nuclear deterrence seriously enough. Before nuclear weapons existed, a state’s ability to defend itself from coercion depended crucially on its economic size, since relative forces mattered and force size/quality depended ultimately on having a large population and an advanced economy. But that’s not true once you have a second-strike nuclear capability; nuclear weapons break the connection you emphasize between economic power and the ability to impose your political preferences on other nuclear-armed states. So I think even granting the argument that AGI could lead to some sort of economic explosion for one state, the amount of coercive leverage that would grant that state is likely to be highly limited, nowhere close to the Bostrom DSA definition you use, so long as nuclear deterrence remains intact.
In case of interest, here’s a piece we published recently on these issues, coauthored with Nikita Lalwani (former director for tech/nat sec on the Biden NSC). We make a version of the above argument about the importance of MAD for the existence of any alleged DSA, and then lay out how exactly AGI might threaten nuclear deterrence. Would welcome any feedback! We’re still tentatively thinking through these DSA issues ourselves.
Best,
Sam
Tom
Hi Sam,
Thanks for this great pushback!
I think you’re right that this is an oversight of the piece. But here are some responses:
In terms of controlling civilisation’s longrun future, outgrowing the world in the way we describe would be enough. We’ll eventually settle space and North Korea/Russia will be the tiniest fraction of total resources/‘land throughout space’. So if US controls everything but some countries on earth, they completely dominate the longrun future. Agree though the DSA definition i used is a bit different to that
If a country gains this kind of massive economic advantage, it will gain a massive tech advantage as well that, at some point, is likely to allow it to remove the second strike capability. This is more speculative, but we are here talking about the limits of technology and 1-month doubling nanobots etc. Still, this at least should be addressed by the post.
I would go further, and say that if we condition on the US growing to become 70-99% of world GDP and growing super-exponentially for a time, then the fact that it can become a superpower in space pretty much neutralizes MAD, because it’s industry is by and large off-world, meaning that a nuclear attack on the US 50 years after the SIE ended is basically a non-factor, because lots of US people will be in space and able to annihilate North Korea (as an example).
And there’s no nuclear weapons equivalent outside of something like vacuum decay, which is very, very far off from our current state, such that we don’t need to worry about it.