Will the world’s elites navigate the creation of AI just fine?

One open ques­tion in AI risk strat­egy is: Can we trust the world’s elite de­ci­sion-mak­ers (here­after “elites”) to nav­i­gate the cre­ation of hu­man-level AI (and be­yond) just fine, with­out the kinds of spe­cial efforts that e.g. Bostrom and Yud­kowsky think are needed?

Some rea­sons for con­cern in­clude:

  • Other­wise smart peo­ple say un­rea­son­able things about AI safety.

  • Many peo­ple who be­lieved AI was around the cor­ner didn’t take safety very se­ri­ously.

  • Elites have failed to nav­i­gate many im­por­tant is­sues wisely (2008 fi­nan­cial crisis, cli­mate change, Iraq War, etc.), for a va­ri­ety of rea­sons.

  • AI may ar­rive rather sud­denly, leav­ing lit­tle time for prepa­ra­tion.

But if you were try­ing to ar­gue for hope, you might ar­gue along these lines (pre­sented for the sake of ar­gu­ment; I don’t ac­tu­ally en­dorse this ar­gu­ment):

  • If AI is pre­ceded by visi­ble sig­nals, elites are likely to take safety mea­sures. Effec­tive mea­sures were taken to ad­dress as­ter­oid risk. Large re­sources are de­voted to miti­gat­ing cli­mate change risks. Per­sonal and tribal self­ish­ness al­ign with AI risk-re­duc­tion in a way they may not al­ign on cli­mate change. Availa­bil­ity of in­for­ma­tion is in­creas­ing over time.

  • AI is likely to be pre­ceded by visi­ble sig­nals. Con­cep­tual in­sights of­ten take years of in­cre­men­tal tweak­ing. In vi­sion, speech, games, com­pres­sion, robotics, and other fields, perfor­mance curves are mostly smooth. “Hu­man-level perfor­mance at X” bench­marks in­fluence per­cep­tions and should be more ex­haus­tive and come more rapidly as AI ap­proaches. Re­cur­sive self-im­prove­ment ca­pa­bil­ities could be charted, and are likely to be AI-com­plete. If AI suc­ceeds, it will likely suc­ceed for rea­sons com­pre­hen­si­ble by the AI re­searchers of the time.

  • There­fore, safety mea­sures will likely be taken.

  • If safety mea­sures are taken, then elites will nav­i­gate the cre­ation of AI just fine. Cor­po­rate and gov­ern­ment lead­ers can use sim­ple heuris­tics (e.g. No­bel prizes) to ac­cess the up­per end of ex­pert opinion. AI de­signs with eas­ily tai­lored ten­dency to act may be the eas­iest to build. The use of early AIs to solve AI safety prob­lems cre­ates an at­trac­tor for “safe, pow­er­ful AI.” Arms races not in­sur­mountable.

The ba­sic struc­ture of this ‘ar­gu­ment for hope’ is due to Carl Shul­man, though he doesn’t nec­es­sar­ily en­dorse the de­tails. (Also, it’s just a rough ar­gu­ment, and as stated is not de­duc­tively valid.)

Per­son­ally, I am not very com­forted by this ar­gu­ment be­cause:

  • Elites of­ten fail to take effec­tive ac­tion de­spite plenty of warn­ing.

  • I think there’s a >10% chance AI will not be pre­ceded by visi­ble sig­nals.

  • I think the elites’ safety mea­sures will likely be in­suffi­cient.

Ob­vi­ously, there’s a lot more for me to spell out here, and some of it may be un­clear. The rea­son I’m post­ing these thoughts in such a rough state is so that MIRI can get some help on our re­search into this ques­tion.

In par­tic­u­lar, I’d like to know:

  • Which his­tor­i­cal events are analo­gous to AI risk in some im­por­tant ways? Pos­si­bil­ities in­clude: nu­clear weapons, cli­mate change, re­com­bi­nant DNA, nan­otech­nol­ogy, chlo­roflouro­car­bons, as­ter­oids, cy­bert­er­ror­ism, Span­ish flu, the 2008 fi­nan­cial crisis, and large wars.

  • What are some good re­sources (e.g. books) for in­ves­ti­gat­ing the rele­vance of these analo­gies to AI risk (for the pur­poses of illu­mi­nat­ing elites’ likely re­sponse to AI risk)?

  • What are some good stud­ies on elites’ de­ci­sion-mak­ing abil­ities in gen­eral?

  • Has the in­creas­ing availa­bil­ity of in­for­ma­tion in the past cen­tury no­tice­ably im­proved elite de­ci­sion-mak­ing?