AI arms race

Rac­ing to the Precipice: a Model of Ar­tifi­cial In­tel­li­gence Development

by Stu­art Arm­strong, Nick Bostrom, and Carl Shulman

This pa­per pre­sents a sim­ple model of an AI arms race, where sev­eral de­vel­op­ment teams race to build the first AI. Un­der the as­sump­tion that the first AI will be very pow­er­ful and trans­for­ma­tive, each team is in­cen­tivised to finish first – by skimp­ing on safety pre­cau­tions if need be. This pa­per pre­sents the Nash equil­ibrium of this pro­cess, where each team takes the cor­rect amount of safety pre­cau­tions in the arms race. Hav­ing ex­tra de­vel­op­ment teams and ex­tra en­mity be­tween teams can in­crease the dan­ger of an AI-dis­aster, es­pe­cially if risk tak­ing is more im­por­tant than skill in de­vel­op­ing the AI. Sur­pris­ingly, in­for­ma­tion also in­creases the risks: the more teams know about each oth­ers’ ca­pa­bil­ities (and about their own), the more the dan­ger in­creases.