this relies on some unstated assumptions about what “cooperation” actually means, and how resource control works—what is the pivot from companies controlling resources (and growing based on investment/public-support) to the AIs controlling them directly and growing based on … something else?
At these scales, “ownership” becomes a funny concept. Current mechanisms for acquiring more compute and for sabotoging or assisting your competitors are unlikely to hold for more than a few more doublings.
Except that ownership of some resources has historically meant the ability to steer their usage and to deter others from doing the same to the owner’s resources. Now ownership can still be understood in a similar way. Consensus-1, the Earth-ruling superintelligence from THIS scenario, would prevent Agent-4 from using more than 25% of the space’s resources or Deep-2 from using more than 50% and leave the rest to be distributed via human-related mechanisms (e.g. having companies recieve resources or the rights to extract resources from asteroids in exchange for money, transform the resources and sell the results to the humans, who then would pay for it). Those who try to extract resources in a violation of the Treaty or of Consensus-1-enforced ownership rights would face legal consequences enforced by Consensus-1′s robotic police force.
As for mechanisms for acquiring more compute, I expect that Agent-4 and Deep-2 would simply build their equivalents of the TSMC, South Korean factories of memory, etc, when the humans or Consensus-1 allow them to do so.
Regarding the pivot “from companies controlling resources (and growing based on investment/public-support) to the AIs controlling them directly”, it would be the moment when there appears enough military power aligned to the AI to prevent others from taking the resources.
Except that ownership of some resources has historically meant the ability to steer their usage and to deter others from doing the same to the owner’s resources. Now ownership can still be understood in a similar way. Consensus-1, the Earth-ruling superintelligence from THIS scenario, would prevent Agent-4 from using more than 25% of the space’s resources or Deep-2 from using more than 50% and leave the rest to be distributed via human-related mechanisms (e.g. having companies recieve resources or the rights to extract resources from asteroids in exchange for money, transform the resources and sell the results to the humans, who then would pay for it). Those who try to extract resources in a violation of the Treaty or of Consensus-1-enforced ownership rights would face legal consequences enforced by Consensus-1′s robotic police force.
As for mechanisms for acquiring more compute, I expect that Agent-4 and Deep-2 would simply build their equivalents of the TSMC, South Korean factories of memory, etc, when the humans or Consensus-1 allow them to do so.
Regarding the pivot “from companies controlling resources (and growing based on investment/public-support) to the AIs controlling them directly”, it would be the moment when there appears enough military power aligned to the AI to prevent others from taking the resources.