For the past months, I’ve had many sleepless nights thinking over a scenario which I couldn’t resolve completely, a scenario that involves a “sovereign leap” by smaller countries in their last moments to combat permanent disempowerment. They deduce that the best means of balancing the asymmetry of superintelligence is creating weapons of biosphere destruction. They precommit to destroying the biosphere if the US and China try to permanently lock in the status quo in their ASI deal, or destroy or poke their biological weapons in any way. There would be mostly two paths that the US and China together could take.
Path A: The US and China call this a bluff but do not act hastily first; they direct their army of ASIs to subtly overthrow those countries to ensure they would not be able to poke their noses in their terra kingdoms. Over the course of the year, several countries are disempowered, overthrown, and taken control of. The moment of glory for the battle won is sound, and everyone in the US and China is happy knowing that they will not die and will live forever. But there seem to be subtle anomalies involved in forest areas where no one would look before, for there were no people living. Some form of matter seems to consume hectares of forests and leave some form of persistent plant-like shapes. The bilateral ASI screams danger and, in a concerned effort, tries to save humanity from extinction. With the help of nanotech, it starts to terraform the solar system. Mars and Titan are the slated homes for a new humanity. As a backup plan, it also creates embryos and hibernates several IQ-boosted humans with nanotechnology to terraform the planets and live on. So, it sends them off to different parts of the universe in hopes that one of those seeds survives.
Path B: The US and China take it seriously. They direct their army of ASIs to debate the best way they could be transparent with those countries to ensure that they can trust them. After weeks of superintelligence effort for solving a decision theory, the ASI comes up with a plan that the US and China leaders can’t know in advance, for they would be able to sabotage it for their own benefit with 90-95% success, but even a 5-10% probability of extinction is unacceptable for the ASI, even though it might be acceptable for the leaders of those countries. So, it invites all the leaders of those countries along with their ASI supervisors. It sets up a supervision and country alignment plan to create ASIs that are aligned with CEV cosmopolitan humanity with no bias in either direction. What would that look like? I will answer with the lazy “you wouldn’t predict a Magnus Carlsen move” type of answer. Perhaps because I am not smart enough to come up with such an alignment plan.
Is this unreasonable? Do you think that, in reality, state-level actors could not develop such weapons with the help of proto-AGIs before the Plan A deal?
For the past months, I’ve had many sleepless nights thinking over a scenario which I couldn’t resolve completely, a scenario that involves a “sovereign leap” by smaller countries in their last moments to combat permanent disempowerment. They deduce that the best means of balancing the asymmetry of superintelligence is creating weapons of biosphere destruction.
They precommit to destroying the biosphere if the US and China try to permanently lock in the status quo in their ASI deal, or destroy or poke their biological weapons in any way. There would be mostly two paths that the US and China together could take.
Path A:
The US and China call this a bluff but do not act hastily first; they direct their army of ASIs to subtly overthrow those countries to ensure they would not be able to poke their noses in their terra kingdoms. Over the course of the year, several countries are disempowered, overthrown, and taken control of. The moment of glory for the battle won is sound, and everyone in the US and China is happy knowing that they will not die and will live forever. But there seem to be subtle anomalies involved in forest areas where no one would look before, for there were no people living. Some form of matter seems to consume hectares of forests and leave some form of persistent plant-like shapes. The bilateral ASI screams danger and, in a concerned effort, tries to save humanity from extinction. With the help of nanotech, it starts to terraform the solar system. Mars and Titan are the slated homes for a new humanity. As a backup plan, it also creates embryos and hibernates several IQ-boosted humans with nanotechnology to terraform the planets and live on. So, it sends them off to different parts of the universe in hopes that one of those seeds survives.
Path B:
The US and China take it seriously. They direct their army of ASIs to debate the best way they could be transparent with those countries to ensure that they can trust them. After weeks of superintelligence effort for solving a decision theory, the ASI comes up with a plan that the US and China leaders can’t know in advance, for they would be able to sabotage it for their own benefit with 90-95% success, but even a 5-10% probability of extinction is unacceptable for the ASI, even though it might be acceptable for the leaders of those countries. So, it invites all the leaders of those countries along with their ASI supervisors. It sets up a supervision and country alignment plan to create ASIs that are aligned with CEV cosmopolitan humanity with no bias in either direction. What would that look like? I will answer with the lazy “you wouldn’t predict a Magnus Carlsen move” type of answer. Perhaps because I am not smart enough to come up with such an alignment plan.
Is this unreasonable? Do you think that, in reality, state-level actors could not develop such weapons with the help of proto-AGIs before the Plan A deal?