I think that space-based power grabs are unlikely as long as powers care about, and are equally-matched on, Earth.
This is the rough story that I think is unlikely to happen:
Two superpowers have roughly equal power on the Earth during the singularity, and remain roughly equal in power after both creating ASIs that are at least intent-aligned with them. They maintain mutually assured destruction on Earth. Superpower A is much more focused on building space infrastructure than Superpower B. Within a decade, Superpower A’s space infrastructure means that Superpower A has a decisive advantage superpower B.
This story to me seems unlikely because in this scenario, Superpower A probably still has most of its human population on Earth (relocating millions of people to space would probably be very slow). Therefore, as long as mutually assured destruction is maintained on Earth, Superpower B will retain a lot of its bargaining power despite having a disadvantage in space infrastructure.
Why would mutually assured destruction remain in place? Superpower A would be in a position to build things in space (e.g. a Sonnengewehr) that negate terrestrial deterrents and provide a decisive strategic advantage. I don’t see how Superpower B’s ASI could justify such a development.
More worryingly, the assumption that the ability to inflict damage on human populations remains a meaningful strategic lever post-ASI rests on the contemporary logic of people being a requirement for industry, for combat, for maintaining a minimum living standard for the population and for producing the next generation. Once warfare can be fully automated, the strategic calculus looks very different.
Presumably superpower B will precommit using their offense-dominant weapons before the retaliation-proof (or splendid first strike enabling) infrastructure is built. It’s technically possible today to saturate space with enough interceptors to blow up ICBMs during boost phase, but it would take so many years to establish full coverage that any opponent you’re hoping to disempower has time to threaten you preemptively. It also seems likely to me that AIs will be much better at making binding and verifiable commitments of this sort, which humans could never be trusted to make legitimately.
As far as whether the population remains relevant, that probably happens through some value lock-in for the early ASIs, such as minimal human rights. In that case, humans would stay useful countervalue targets, even if their instrumental value to war is gone.
This assumes a solution to the alignment problem that creates massive, legible strategic penalties for the ASI. If human rights were to become a terminal value for the ASI, then the contingencies at the heart of deterrence theory become unjustifiable since they establish conditions under which those rights can be revoked, thus contradicting the notion of human rights as a terminal value.
If human rights were to become a terminal value for the ASI, then the contingencies at the heart of deterrence theory become unjustifiable since they establish conditions under which those rights can be revoked, thus contradicting the notion of human rights as a terminal value.
I’m a bit unclear on what this is means. If you see preserving humans as a priority, why would threatening other humans to ensure strategic stability run against that? Countervalue targeting today works on the same principles, with nations that are ~aligned on human rights but willing to commit to violating them in retaliation to preserve strategic stability.
Assuming that issuing such threats isn’t itself a violation of human rights, if you genuinely see preserving humans as a terminal value, then you would not follow through on such a threat because a counterstrike would result in the opposite of what is demanded by your values. You could say that not following through on such a threat would weaken your strategic position due to a loss of credibility, but in this case you would be subordinating human preservation to credibility preservation, and thus to strategic calculus. If you sacrifice your terminal value for something else, then it wasn’t really your terminal value.
Two reasons for why deterrence works today are 1. countries do not treat human preservation as such as a terminal value, and 2. countries treat their adversary’s population as a negative value to the extent that it can be mobilized against them, thus making a counterstrike desirable.
Seems like this assumes that Superpower B is willing to do a preemptive nuclear strike on Superpower A even if Superpower A isn’t threatening Superpower B’s Earth population at all and the only threat is that they might steal space resources. I think in that situation, a preemptive nuclear strike would be considered a huge escalation, most possible Superpower B’s would likely be unwilling to do it, and most possible Superpower A’s would be unlikely to consider the threat of doing so to be credible (and would be likely to call the bluff for that reason).
In this case, the problem isn’t that superpower A is gaining an unfair fraction of resources, it’s that gaining enough of them would (presumably) allow them to assert a DSA over superpower B, threatening what B already owns. Analagously, it makes sense to precommit to nuking an offensive realist that’s attempting to build ICBM defenses, because it signals that they are aiming to disempower you in the future. You also wouldn’t necessarily need to escalate to the civilian population as a deterrent right away: instead, you could just focus on disabling the defensive infrastructure while it’s being built, only escalating further if A undermines your efforts (such as by building their defensive systems next to civilians).
Any plan of this sort would be very difficult to enforce with humans because of private information and commitment problems, but there are probably technical solutions for AIs to verifiably prove their motivations and commitments (ex: co-design).
The thing that worries me in this scenario is that Superpower A has the ability to expand to other star systems while Superpower B (and presumably everyone else on Earth) has very little say about it.
I don’t think Superpower B would be able to convincingly argue that they are willing to initiate their own destruction through MAD if they’re not given access to half of the galaxy or if their preferences for large scale space expansion are not taken into account. Also, assuming the leaders of superpower B don’t want to be deposed, they wouldn’t make such a threat anyway because it wouldn’t be in the interest of the people that the leaders of Superpower B represent.
I think interstellar travel will be a really pivotal point with a huge risk of locking in bad values for the long-term future. There is a selection bias: the actor most likely to gain dominance in space is not the actor most likely to have good values, so space power grabs risks locking in bad values for the long-term future.
I think that space-based power grabs are unlikely as long as powers care about, and are equally-matched on, Earth.
This is the rough story that I think is unlikely to happen:
This story to me seems unlikely because in this scenario, Superpower A probably still has most of its human population on Earth (relocating millions of people to space would probably be very slow). Therefore, as long as mutually assured destruction is maintained on Earth, Superpower B will retain a lot of its bargaining power despite having a disadvantage in space infrastructure.
Why would mutually assured destruction remain in place? Superpower A would be in a position to build things in space (e.g. a Sonnengewehr) that negate terrestrial deterrents and provide a decisive strategic advantage. I don’t see how Superpower B’s ASI could justify such a development.
More worryingly, the assumption that the ability to inflict damage on human populations remains a meaningful strategic lever post-ASI rests on the contemporary logic of people being a requirement for industry, for combat, for maintaining a minimum living standard for the population and for producing the next generation. Once warfare can be fully automated, the strategic calculus looks very different.
Presumably superpower B will precommit using their offense-dominant weapons before the retaliation-proof (or splendid first strike enabling) infrastructure is built. It’s technically possible today to saturate space with enough interceptors to blow up ICBMs during boost phase, but it would take so many years to establish full coverage that any opponent you’re hoping to disempower has time to threaten you preemptively. It also seems likely to me that AIs will be much better at making binding and verifiable commitments of this sort, which humans could never be trusted to make legitimately.
As far as whether the population remains relevant, that probably happens through some value lock-in for the early ASIs, such as minimal human rights. In that case, humans would stay useful countervalue targets, even if their instrumental value to war is gone.
This assumes a solution to the alignment problem that creates massive, legible strategic penalties for the ASI. If human rights were to become a terminal value for the ASI, then the contingencies at the heart of deterrence theory become unjustifiable since they establish conditions under which those rights can be revoked, thus contradicting the notion of human rights as a terminal value.
I’m a bit unclear on what this is means. If you see preserving humans as a priority, why would threatening other humans to ensure strategic stability run against that? Countervalue targeting today works on the same principles, with nations that are ~aligned on human rights but willing to commit to violating them in retaliation to preserve strategic stability.
Assuming that issuing such threats isn’t itself a violation of human rights, if you genuinely see preserving humans as a terminal value, then you would not follow through on such a threat because a counterstrike would result in the opposite of what is demanded by your values. You could say that not following through on such a threat would weaken your strategic position due to a loss of credibility, but in this case you would be subordinating human preservation to credibility preservation, and thus to strategic calculus. If you sacrifice your terminal value for something else, then it wasn’t really your terminal value.
Two reasons for why deterrence works today are 1. countries do not treat human preservation as such as a terminal value, and 2. countries treat their adversary’s population as a negative value to the extent that it can be mobilized against them, thus making a counterstrike desirable.
Seems like this assumes that Superpower B is willing to do a preemptive nuclear strike on Superpower A even if Superpower A isn’t threatening Superpower B’s Earth population at all and the only threat is that they might steal space resources. I think in that situation, a preemptive nuclear strike would be considered a huge escalation, most possible Superpower B’s would likely be unwilling to do it, and most possible Superpower A’s would be unlikely to consider the threat of doing so to be credible (and would be likely to call the bluff for that reason).
In this case, the problem isn’t that superpower A is gaining an unfair fraction of resources, it’s that gaining enough of them would (presumably) allow them to assert a DSA over superpower B, threatening what B already owns. Analagously, it makes sense to precommit to nuking an offensive realist that’s attempting to build ICBM defenses, because it signals that they are aiming to disempower you in the future. You also wouldn’t necessarily need to escalate to the civilian population as a deterrent right away: instead, you could just focus on disabling the defensive infrastructure while it’s being built, only escalating further if A undermines your efforts (such as by building their defensive systems next to civilians).
Any plan of this sort would be very difficult to enforce with humans because of private information and commitment problems, but there are probably technical solutions for AIs to verifiably prove their motivations and commitments (ex: co-design).
The thing that worries me in this scenario is that Superpower A has the ability to expand to other star systems while Superpower B (and presumably everyone else on Earth) has very little say about it.
I don’t think Superpower B would be able to convincingly argue that they are willing to initiate their own destruction through MAD if they’re not given access to half of the galaxy or if their preferences for large scale space expansion are not taken into account. Also, assuming the leaders of superpower B don’t want to be deposed, they wouldn’t make such a threat anyway because it wouldn’t be in the interest of the people that the leaders of Superpower B represent.
I think interstellar travel will be a really pivotal point with a huge risk of locking in bad values for the long-term future. There is a selection bias: the actor most likely to gain dominance in space is not the actor most likely to have good values, so space power grabs risks locking in bad values for the long-term future.