[Question] Does it become easier, or harder, for the world to coordinate around not building AGI as time goes on?

(Or, is co­or­di­na­tion eas­ier in a long timeline?)

It seems like it would be good if the world could co­or­di­nate to not build AGI. That is, at some point in the fu­ture, when some num­ber of teams will have the tech­ni­cal abil­ity to build and de­ploy and AGI, but they all agree to vol­un­tar­ily de­lay (per­haps on penalty of sanc­tions) un­til they’re con­fi­dent that hu­man­ity knows how to al­ign such a sys­tem.

Cur­rently, this kind of co­or­di­na­tion seems like a pretty im­plau­si­ble state of af­fairs. But I want to know if it seems like it be­comes more or less plau­si­ble as time passes.

The fol­low­ing is my ini­tial think­ing in this area. I don’t know the rel­a­tive im­por­tance of the fac­tors that I listed, and there’s lots that I don’t un­der­stand about each of them. I would be glad for…

  • Ad­di­tional rele­vant fac­tors.

  • Ar­gu­ments that some fac­tor is much more im­por­tant than the oth­ers.

  • Cor­rec­tions, clar­ifi­ca­tions, or coun­ter­ar­gu­ments to any of this.

  • Other an­swers to the ques­tion, that ig­nore my thoughts en­tirely.

If co­or­di­na­tion gets harder over­time, that’s prob­a­bly be­cause...

  • Com­pute in­creases make de­vel­op­ing and/​or run­ning an AGI cheaper. The most ob­vi­ous con­sid­er­a­tion is that the cost of com­put­ing falls each year. If one of the bot­tle­necks for an AGI pro­ject is hav­ing large amounts of com­pute, then “hav­ing ac­cess to suffi­cient com­pute” is a gate­keeper crite­rion on who can build AGI. As the cost of com­put­ing con­tinues to fall, more groups will be able to run AGI pro­jects. The more peo­ple who can build an AGI, the harder it be­comes to co­or­di­nate all of them into not de­ploy­ing it.

    • Note that It is un­clear to what de­gree there is cur­rently, or will be, a hard­ware over­hang. If some­one in 2019 could already run an AGI, on only $10,000 worth of AWS, if only they knew how, then the cost of com­pute is not rele­vant to the ques­tion of co­or­di­na­tion.

  • The num­ber of rele­vant ac­tors in­creases. If some­one builds an AGI in the next year, I am rea­son­ably con­fi­dent that that some­one will be Deep Mind. I ex­pect that in 15 years, if I knew that AGI would be de­vel­oped one year from then, it will be much less overde­ter­mined which group is go­ing to build it, be­cause there will be many more well funded AI teams with top tal­ent, and, most likely, none of them will have as strong a lead as Deep Mind cur­rently ap­pears to have.

    • This con­sid­er­a­tion sug­gests that co­or­di­na­tion gets harder over time. How­ever, this de­pends heav­ily on other fac­tors (like how ac­cepted AI safety memes are) that de­ter­mine how eas­ily Deep Mind could co­or­di­nate in­ter­nally.

If co­or­di­na­tion gets eas­ier over time, that’s prob­a­bly be­cause…

  • AI safety memes be­come more and more per­va­sive and gen­er­ally ac­cepted. It seems that co­or­di­na­tion is eas­ier in wor­lds where it is un­con­tro­ver­sial and com­mon knowl­edge that an un­al­igned AGI poses and ex­is­ten­tial risk, be­cause ev­ery­one agrees that they will lose big if any­one builds an AGI.

    • Over the past 15 years, the key ar­gu­ments of AI safety have gone from be­ing ex­tremely fringe, to a rea­son­ably re­garded (if some­what con­tro­ver­sial) po­si­tion, well in­side the over­ton win­dow. Will this pro­cess con­tinue? Will it be com­monly ac­cepted by ML re­searches in 2030, that ad­vanced AI poses and ex­is­ten­tial threat? Will it be com­monly ac­cepted by the lead­ers of na­tion-states?

    • What will the per­cep­tion of safety be in a world where there is an­other AGI win­ter? Sup­pose that nar­row ML proves to be ex­tremely use­ful in a large num­ber of fields, but there’s lots of hype about AGI be­ing right around the cor­ner, then that bub­ble bursts, and there is broad dis­in­ter­est in AGI again. What hap­pens to the per­cep­tion of AI safety? Is there a sense of “It looks like AI Align­ment wasn’t im­por­tant af­ter all”? How cau­tious will re­searchers be in de­vel­op­ing new AI tech­nolo­gies.

  • [Par­tial sub­point to the above con­sid­er­a­tion] In­di­vi­d­ual AI teams de­velop more se­ri­ous info se­cu­rity con­scious pro­cesses. If some team in Deep Mind dis­cov­ered AGI to­day, and the Deep Mind lead­er­ship opted to wait to in­sure safety be­fore de­ploy­ing it, I don’t know how long it would be un­til some rele­vant em­ploy­ees left to build AGI on their own, or some other group (such as a state ac­tor) stole their tech­nol­ogy and de­ployed it.

    • I don’t know if this is get­ting bet­ter or worse, over­time.

  • The tech­nolo­gies for main­tain­ing surveillance of would-be AGI de­vel­op­ers im­prove. Co­or­di­na­tion is made eas­ier by tech­nolo­gies that aid in en­force­ment. If surveillance tech­nol­ogy im­proves that seems like it would make co­or­di­na­tion eas­ier. As a spe­cial case, highly re­li­able lie de­tec­tion or mind read­ing tech­nolo­gies would be a game-changer for mak­ing co­or­di­na­tion eas­ier.

    • Is there a rea­son to think that offense will beat defense in this area? Surveillance could get harder over time if the tech­nol­ogy for de­tect­ing and defeat­ing surveillance out­paces the tech­nol­ogy for surveilling.

  • Se­cu­rity tech­nol­ogy im­proves. Similarly, im­prove­ments in com­puter se­cu­rity (and tra­di­tional info se­cu­rity), would make it eas­ier for ac­tors to vol­un­tar­ily de­lay de­ploy­ing ad­vanced AI tech­nolo­gies, be­cause they could trust that their com­peti­tors (other com­pa­nies and other na­tions), wouldn’t be able to steal their work.

    • I don’t know if this is plau­si­ble at all. My im­pres­sion is that the weak point of all se­cu­rity sys­tems is the peo­ple in­volved. What sort of ad­vance­ments would make the hu­man part of a se­cu­rity sys­tem more re­li­able?