A list of good heuristics that the case for AI x-risk fails

I think one rea­son ma­chine learn­ing re­searchers don’t think AI x-risk is a prob­lem is be­cause they haven’t given it the time of day. And on some level, they may be right in not do­ing so!

We all need to do meta-level rea­son­ing about what to spend our time and effort on. Even giv­ing an idea or ar­gu­ment the time of day re­quires it to cross a some­what high bar, if you value your time. Ul­ti­mately, in eval­u­at­ing whether it’s worth con­sid­er­ing a pu­ta­tive is­sue (like the ex­tinc­tion of hu­man­ity at the hands (graspers?) of a rogue AI), one must rely on heuris­tics; by giv­ing the ar­gu­ment the time of day, you’ve already con­ceded a sig­nifi­cant amount of re­sources to it! More­over, you risk priv­ileg­ing the hy­poth­e­sis or fal­ling vic­tim to Pas­cal’s Mug­ging.

Un­for­tu­nately, the case for x-risk from out-of-con­trol AI sys­tems seems to fail many pow­er­ful and ac­cu­rate heuris­tics. This can put pro­po­nents of this is­sue in a similar po­si­tion to flat-earth con­spir­acy the­o­rists at first glance. My goal here is to enu­mer­ate heuris­tics that ar­gu­ments for AI takeover sce­nar­ios fail.

Ul­ti­mately, I think ma­chine learn­ing re­searchers should not re­fuse to con­sider AI x-risk when pre­sented with a well-made case by a per­son they re­spect or have a per­sonal re­la­tion­ship with, but I’m am­biva­lent as to whether they have an obli­ga­tion to con­sider the case if they’ve only seen a few head­lines about Elon. I do find it a bit hard to un­der­stand how one doesn’t end up think­ing about the con­se­quences of su­per-hu­man AI, since it seems ob­vi­ously im­pact­ful and fas­ci­nat­ing. But I’m a very cu­ri­ous (read “dis­tractable”) per­son…


A list of heuris­tics that say not to worry about AI takeover sce­nar­ios:

  • Out­siders not ex­perts: This con­cern is be­ing voiced ex­clu­sively by non-ex­perts like Elon Musk, Steven Hawk­ing, and the talka­tive crazy guy next to you on the bus.

  • Lud­di­tism has a poor track record: For ev­ery new tech­nol­ogy, there’s been a pack of alarmist naysay­ers and dooms­day prophets. And then in­stead of fal­ling apart, the world got bet­ter.

  • ETA: No con­crete threat model: When some­one raises a hy­po­thet­i­cal con­cern, but can’t give you a good ex­pla­na­tion for how it could ac­tu­ally hap­pen, it’s much less likely to ac­tu­ally hap­pen. Is the pa­per­clip max­i­mizer the best you can do?

  • It’s straight out of sci­ence fic­tion: AI re­searchers didn’t come up with this con­cern, Hol­ly­wood did. Science fic­tion is con­structed based on en­ter­tain­ing premises, not re­al­is­tic ca­pa­bil­ities of tech­nolo­gies.

  • It’s not em­piri­cally testable: There’s no way to falsify the be­lief that AI will kill us all. It’s purely a mat­ter of faith. Such be­liefs don’t have good track records of match­ing re­al­ity.

  • It’s just too ex­treme: When­ever we hear an ex­treme pre­dic­tion, we should be sus­pi­cious. To the ex­tent that ex­treme changes hap­pen, they tend to be un­pre­dictable. While ex­treme pre­dic­tions some­times con­tain a seed of truth, re­al­ity tends to be more mun­dane and bor­ing.

  • It has no ground­ing in my per­sonal ex­pe­rience: When I train my AI sys­tems, they are dumb as door­knobs. You’re tel­ling me they’re go­ing to be smarter than me? In a few years? So smart that they can out­wit me, even though I con­trol the very sub­strate of their ex­is­tence?

  • It’s too far off: It’s too hard to pre­dict the fu­ture and we can’t re­ally hope to an­ti­ci­pate spe­cific prob­lems with fu­ture AI sys­tems; we’re sure to be sur­prised! We should wait un­til we can en­vi­sion more spe­cific is­sues, sce­nar­ios, and threats, not waste our time on what comes down to pure spec­u­la­tion.

I’m pretty sure this list in in­com­plete, and I plan to keep adding to it as I think of or hear new sug­ges­tions! Suggest away!!

Also, to be clear, I am writ­ing these de­scrip­tions from the per­spec­tive of some­one who has had very limited ex­po­sure to the ideas un­der­ly­ing con­cerns about AI takeover sce­nar­ios. I think a lot of these re­ac­tions in­di­cate sig­nifi­cant mi­s­un­der­stand­ings about what peo­ple work­ing on miti­gat­ing AI x-risk be­lieve, as well as mat­ters of fact (e.g. a num­ber of ex­perts have voiced con­cerns about AI x-risk, and a sig­nifi­cant por­tion of the re­search com­mu­nity seems to agree that these con­cerns are at least some­what plau­si­ble and im­por­tant).