Strategic implications of AIs’ ability to coordinate at low cost, for example by merging

It seems likely to me that AIs will be able to co­or­di­nate with each other much more eas­ily (i.e., at lower cost and greater scale) than hu­mans cur­rently can, for ex­am­ple by merg­ing into co­her­ent unified agents by com­bin­ing their util­ity func­tions. This has been dis­cussed at least since 2009, but I’m not sure its im­pli­ca­tions have been widely rec­og­nized. In this post I talk about two such im­pli­ca­tions that oc­curred to me rel­a­tively re­cently.

I was re­cently re­minded of this quote from Robin Han­son’s Pre­fer Law To Values:

The later era when robots are vastly more ca­pa­ble than peo­ple should be much like the case of choos­ing a na­tion in which to re­tire. In this case we don’t ex­pect to have much in the way of skills to offer, so we mostly care that they are law-abid­ing enough to re­spect our prop­erty rights. If they use the same law to keep the peace among them­selves as they use to keep the peace with us, we could have a long and pros­per­ous fu­ture in what­ever weird world they con­jure. In such a vast rich uni­verse our “re­tire­ment in­come” should buy a com­fortable if not cen­tral place for hu­mans to watch it all in won­der.

Robin ar­gued that this im­plies we should work to make it more likely that our cur­rent in­sti­tu­tions like laws will sur­vive into the AI era. But (aside from the prob­lem that we’re most likely still in­cur­ring as­tro­nom­i­cal waste even if many hu­mans sur­vive “in re­tire­ment”), as­sum­ing that AIs will have the abil­ity to co­or­di­nate amongst them­selves by do­ing some­thing like merg­ing their util­ity func­tions, there will be no rea­son to use laws (much less “the same laws”) to keep peace among them­selves. So the first im­pli­ca­tion is that to the ex­tent that AIs are likely to have this abil­ity, work­ing in the di­rec­tion Robin sug­gested would likely be fu­tile.

The sec­ond im­pli­ca­tion is that AI safety/​al­ign­ment ap­proaches that aim to pre­serve an AI’s com­pet­i­tive­ness must also pre­serve its abil­ity to co­or­di­nate with other AIs, since that is likely an im­por­tant part of its com­pet­i­tive­ness. For ex­am­ple, mak­ing an AI cor­rigible in the sense of al­low­ing a hu­man to shut it (and its suc­ces­sors/​sub­agents) down or change how it func­tions would seem­ingly make it im­pos­si­ble for this AI to merge with an­other AI that is not cor­rigible, or not cor­rigible in the same way. (I’ve men­tioned this a num­ber of times in pre­vi­ous com­ments, as a rea­son why I’m pes­simistic about spe­cific ap­proaches, but I’m not sure if oth­ers have picked up on it, or agree with it, as a gen­eral con­cern, which partly mo­ti­vates this post.)

Ques­tions: Do you agree AIs are likely to have the abil­ity to co­or­di­nate with each other at low cost? What other im­pli­ca­tions does this have, es­pe­cially for our strate­gies for re­duc­ing x-risk?