The post Why Not Subagents formalizes an argument for why subagents have an incentive to create contracts to effectively become a single agent. The post frames this as bad news. I think it points to a strategy for avoiding needing to build AI. I currently think of it as “becoming a singleton the hard way”, through contracts, mechanism design, financial engineering and regular engineering of the future. The second ingredient is human intelligence amplification via whatever intelligence enhancement techniques get the job done best at the current technology stack. If you are good at preserving logs and brains you can perhaps even delegate computing fair prices almost fully to the future with ample compute available! No conflict needed! This only works if the agents involved are not myopic enough to destroy the universe for a little fun building companies in the short run (which is where this plan sadly probably fails on Earth?). I am very happy for Earth to surprise me though :). I would argue you can view nationalization as a product of this force (see the first chapter of “Seeing Like a State” for details on how France needed to first unify its units to even tax things). I saw Richard Ngo complain somewhere about how the international cognitive elite is becoming disconnected from everyone else and how we need to go back to ethnonationalism. Speculative: I view the same process as the beginning of the next phase transition that could in principle be the beginning of the earth-wide singleton.
I saw Richard Ngo complain somewhere about how the international cognitive elite is becoming disconnected from everyone else and how we need to go back to ethnonationalism.
The post Why Not Subagents formalizes an argument for why subagents have an incentive to create contracts to effectively become a single agent. The post frames this as bad news. I think it points to a strategy for avoiding needing to build AI. I currently think of it as “becoming a singleton the hard way”, through contracts, mechanism design, financial engineering and regular engineering of the future. The second ingredient is human intelligence amplification via whatever intelligence enhancement techniques get the job done best at the current technology stack. If you are good at preserving logs and brains you can perhaps even delegate computing fair prices almost fully to the future with ample compute available! No conflict needed! This only works if the agents involved are not myopic enough to destroy the universe for a little fun building companies in the short run (which is where this plan sadly probably fails on Earth?). I am very happy for Earth to surprise me though :). I would argue you can view nationalization as a product of this force (see the first chapter of “Seeing Like a State” for details on how France needed to first unify its units to even tax things). I saw Richard Ngo complain somewhere about how the international cognitive elite is becoming disconnected from everyone else and how we need to go back to ethnonationalism. Speculative: I view the same process as the beginning of the next phase transition that could in principle be the beginning of the earth-wide singleton.
I hope there are more than just two options.