The AI Timelines Scam

Link post

[epistemic sta­tus: that’s just my opinion, man. I have highly sug­ges­tive ev­i­dence, not de­duc­tive proof, for a be­lief I sincerely hold]

“If you see fraud and do not say fraud, you are a fraud.”Nasim Taleb

I was talk­ing with a col­league the other day about an AI or­ga­ni­za­tion that claims:

  1. AGI is prob­a­bly com­ing in the next 20 years.

  2. Many of the rea­sons we have for be­liev­ing this are se­cret.

  3. They’re se­cret be­cause if we told peo­ple about those rea­sons, they’d learn things that would let them make an AGI even sooner than they would oth­er­wise.

His re­sponse was (para­phras­ing): “Wow, that’s a re­ally good lie! A lie that can’t be dis­proven.”

I found this re­sponse re­fresh­ing, be­cause he im­me­di­ately jumped to the most likely con­clu­sion.

Near pre­dic­tions gen­er­ate more funding

Gen­er­ally, en­trepreneurs who are op­ti­mistic about their pro­ject get more fund­ing than ones who aren’t. AI is no ex­cep­tion. For a re­cent ex­am­ple, see the Hu­man Brain Pro­ject. The founder, Henry Makram, pre­dicted in 2009 that the pro­ject would suc­ceed in simu­lat­ing a hu­man brain by 2019, and the pro­ject was already widely con­sid­ered a failure by 2013. (See his TED talk, at 14:22)

The Hu­man Brain pro­ject got 1.3 billion Euros of fund­ing from the EU.

It’s not hard to see why this is. To jus­tify re­ceiv­ing large amounts of money, the leader must make a claim that the pro­ject is ac­tu­ally worth that much. And, AI pro­jects are more im­pact­ful if it is, in fact, pos­si­ble to de­velop AI soon. So, there is an eco­nomic pres­sure to­wards in­flat­ing es­ti­mates of the chance AI will be de­vel­oped soon.

Fear of an AI gap

The mis­sile gap was a lie by the US Air Force to jus­tify build­ing more nukes, by falsely claiming that the Soviet Union had more nukes than the US.

Similarly, there’s his­tor­i­cal prece­dent for an AI gap lie used to jus­tify more AI de­vel­op­ment. Fifth Gen­er­a­tion Com­puter Sys­tems was an am­bi­tious 1982 pro­ject by the Ja­panese gov­ern­ment (funded for $400 mil­lion in 1992, or $730 mil­lion in 2019 dol­lars) to cre­ate ar­tifi­cial in­tel­li­gence through mas­sively par­allel logic pro­gram­ming.

The pro­ject is widely con­sid­ered to have failed. From a 1992 New York Times ar­ti­cle:

A bold 10-year effort by Ja­pan to seize the lead in com­puter tech­nol­ogy is fiz­zling to a close, hav­ing failed to meet many of its am­bi­tious goals or to pro­duce tech­nol­ogy that Ja­pan’s com­puter in­dus­try wanted.


That at­ti­tude is a sharp con­trast to the pro­ject’s in­cep­tion, when it spread fear in the United States that the Ja­panese were go­ing to leapfrog the Amer­i­can com­puter in­dus­try. In re­sponse, a group of Amer­i­can com­pa­nies formed the Micro­elec­tron­ics and Com­puter Tech­nol­ogy Cor­po­ra­tion, a con­sor­tium in Austin, Tex., to co­op­er­ate on re­search. And the Defense Depart­ment, in part to meet the Ja­panese challenge, be­gan a huge long-term pro­gram to de­velop in­tel­li­gent sys­tems, in­clud­ing tanks that could nav­i­gate on their own.


The Fifth Gen­er­a­tion effort did not yield the break­throughs to make ma­chines truly in­tel­li­gent, some­thing that prob­a­bly could never have re­al­is­ti­cally been ex­pected any­way. Yet the pro­ject did suc­ceed in de­vel­op­ing pro­to­type com­put­ers that can perform some rea­son­ing func­tions at high speeds, in part by em­ploy­ing up to 1,000 pro­ces­sors in par­allel. The pro­ject also de­vel­oped ba­sic soft­ware to con­trol and pro­gram such com­put­ers. Ex­perts here said that some of these achieve­ments were tech­ni­cally im­pres­sive.


In his open­ing speech at the con­fer­ence here, Kazuhiro Fuchi, the di­rec­tor of the Fifth Gen­er­a­tion pro­ject, made an im­pas­sioned defense of his pro­gram.

“Ten years ago we faced crit­i­cism of be­ing too reck­less,” in set­ting too many am­bi­tious goals, he said, adding, “Now we see crit­i­cism from in­side and out­side the coun­try be­cause we have failed to achieve such grand goals.”

Out­siders, he said, ini­tially ex­ag­ger­ated the aims of the pro­ject, with the re­sult that the pro­gram now seems to have fallen short of its goals.

Some Amer­i­can com­puter sci­en­tists say pri­vately that some of their col­leagues did per­haps over­state the scope and threat of the Fifth Gen­er­a­tion pro­ject. Why? In or­der to coax more sup­port from the United States Govern­ment for com­puter sci­ence re­search.

(em­pha­sis mine)

This bears similar­ity to some con­ver­sa­tions on AI risk I’ve been party to in the past few years. The fear is that Others (Deep­Mind, China, who­ever) will de­velop AGI soon, so We have to de­velop AGI first in or­der to make sure it’s safe, be­cause Others won’t make sure it’s safe and We will. Also, We have to dis­cuss AGI strat­egy in pri­vate (and avoid pub­lic dis­cus­sion), so Others don’t get the wrong ideas. (Gen­er­ally, these claims have lit­tle em­piri­cal/​ra­tio­nal back­ing to them; they’re based on scary sto­ries, not his­tor­i­cally val­i­dated threat mod­els)

The claim that oth­ers will de­velop weapons and kill us with them by de­fault im­plies a moral claim to re­sources, and a moral claim to be jus­tified in mak­ing weapons in re­sponse. Such claims, if ex­ag­ger­ated, jus­tify claiming more re­sources and mak­ing more weapons. And they weaken a com­mu­nity’s ac­tual abil­ity to track and re­spond to real threats (as in The Boy Who Cried Wolf).

How does the AI field treat its crit­ics?

Hu­bert Dreyfus, prob­a­bly the most fa­mous his­tor­i­cal AI critic, pub­lished “Alchemy and Ar­tifi­cial In­tel­li­gence” in 1965, which ar­gued that the tech­niques pop­u­lar at the time were in­suffi­cient for AGI. Sub­se­quently, he was shunned by other AI re­searchers:

The pa­per “caused an up­roar”, ac­cord­ing to Pamela McCor­duck. The AI com­mu­nity’s re­sponse was de­ri­sive and per­sonal. Sey­mour Papert dis­missed one third of the pa­per as “gos­sip” and claimed that ev­ery quo­ta­tion was de­liber­ately taken out of con­text. Her­bert A. Si­mon ac­cused Dreyfus of play­ing “poli­tics” so that he could at­tach the pres­ti­gious RAND name to his ideas. Si­mon said, “what I re­sent about this was the RAND name at­tached to that garbage.”

Dreyfus, who taught at MIT, re­mem­bers that his col­leagues work­ing in AI “dared not be seen hav­ing lunch with me.” Joseph Weizen­baum, the au­thor of ELIZA, felt his col­leagues’ treat­ment of Dreyfus was un­pro­fes­sional and childish. Although he was an out­spo­ken critic of Dreyfus’ po­si­tions, he re­calls “I be­came the only mem­ber of the AI com­mu­nity to be seen eat­ing lunch with Dreyfus. And I de­liber­ately made it plain that theirs was not the way to treat a hu­man be­ing.”

This makes sense as anti-whistle­blower ac­tivity: os­tra­ciz­ing, dis­cred­it­ing, or pun­ish­ing peo­ple who break the con­spir­acy to the pub­lic. Does this still hap­pen in the AI field to­day?

Gary Mar­cus is a more re­cent AI re­searcher and critic. In 2012, he wrote:

Deep learn­ing is im­por­tant work, with im­me­di­ate prac­ti­cal ap­pli­ca­tions.


Real­is­ti­cally, deep learn­ing is only part of the larger challenge of build­ing in­tel­li­gent ma­chines. Such tech­niques lack ways of rep­re­sent­ing causal re­la­tion­ships (such as be­tween dis­eases and their symp­toms), and are likely to face challenges in ac­quiring ab­stract ideas like “sibling” or “iden­ti­cal to.” They have no ob­vi­ous ways of perform­ing log­i­cal in­fer­ences, and they are also still a long way from in­te­grat­ing ab­stract knowl­edge, such as in­for­ma­tion about what ob­jects are, what they are for, and how they are typ­i­cally used. The most pow­er­ful A.I. sys­tems … use tech­niques like deep learn­ing as just one el­e­ment in a very com­pli­cated en­sem­ble of tech­niques, rang­ing from the statis­ti­cal tech­nique of Bayesian in­fer­ence to de­duc­tive rea­son­ing.

In 2018, he tweeted an ar­ti­cle in which Yoshua Ben­gio (a deep learn­ing pi­o­neer) seemed to agree with these pre­vi­ous opinions. This tweet re­ceived a num­ber of mostly-crit­i­cal replies. Here’s one, by AI pro­fes­sor Zachary Lip­ton:

There’s a cou­ple prob­lems with this whole line of at­tack. 1) Say­ing it louder ≠ say­ing it first. You can’t claim credit for differ­en­ti­at­ing be­tween rea­son­ing and pat­tern recog­ni­tion. 2) Say­ing X doesn’t solve Y is pretty easy. But where are your con­crete solu­tions for Y?

The first crit­i­cism is es­sen­tially a claim that ev­ery­body knows that deep learn­ing can’t do rea­son­ing. But, this is es­sen­tially ad­mit­ting that Mar­cus is cor­rect, while still crit­i­ciz­ing him for say­ing it [ED NOTE: the phras­ing of this sen­tence is off (Lip­ton pub­li­cly agrees with Mar­cus on this point), and there is more con­text, see Lip­ton’s re­ply].

The sec­ond is a claim that Mar­cus shouldn’t crit­i­cize if he doesn’t have a solu­tion in hand. This policy de­ter­minis­ti­cally re­sults in the short AI timelines nar­ra­tive be­ing main­tained: to crit­i­cize the cur­rent nar­ra­tive, you must pre­sent your own solu­tion, which con­sti­tutes an­other nar­ra­tive for why AI might come soon.

Deep learn­ing pi­o­neer Yann LeCun’s re­sponse is similar:

Yoshua (and I, and oth­ers) have been say­ing this for a long time.
The differ­ence with you is that we are ac­tu­ally try­ing to do some­thing about it, not crit­i­cize peo­ple who don’t.

Again, the crit­i­cism is not that Mar­cus is wrong in say­ing deep learn­ing can’t do cer­tain forms of rea­son­ing, the crit­i­cism is that he isn’t pre­sent­ing an al­ter­na­tive solu­tion. (Of course, the claim could be cor­rect even if Mar­cus doesn’t have an al­ter­na­tive!)

Ap­par­ently, it’s con­sid­ered bad prac­tice in AI to crit­i­cize a pro­posal for mak­ing AGI with­out pre­sent­ing on al­ter­na­tive solu­tion. Clearly, such a policy causes large dis­tor­tions!

Here’s an­other re­sponse, by Steven Hansen (a re­search sci­en­tist at Deep­Mind):

Ideally, you’d be say­ing this through NeurIPS sub­mis­sions rather than New Yorker ar­ti­cles. A lot of the push-back you’re get­ting right now is due to the per­cep­tion that you haven’t been us­ing the ap­pro­pri­ate chan­nels to in­fluence the field.

That is: to crit­i­cize the field, you should go through the field, not through the press. This is stan­dard guild be­hav­ior. In the words of Adam Smith: “Peo­ple of the same trade sel­dom meet to­gether, even for mer­ri­ment and di­ver­sion, but the con­ver­sa­tion ends in a con­spir­acy against the pub­lic, or in some con­trivance to raise prices.”

(Also see Mar­cus’s medium ar­ti­cle on the Twit­ter thread, and on the limi­ta­tions of deep learn­ing)

[ED NOTE: I’m not say­ing these crit­ics on Twit­ter are pub­li­cly pro­mot­ing short AI timelines nar­ra­tives (in fact, some are pro­mot­ing the op­po­site), I’m say­ing that the norms by which they crit­i­cize Mar­cus re­sult in short AI timelines nar­ra­tives be­ing main­tained.]

Why model so­ciopoli­ti­cal dy­nam­ics?

This post has fo­cused on so­ciopolot­i­cal phe­nom­ena in­volved in the short AI timelines phe­nomenon. For this, I an­ti­ci­pate crit­i­cism along the lines of “why not just model the tech­ni­cal ar­gu­ments, rather than the cred­i­bil­ity of the peo­ple in­volved?” To which I pre-emp­tively re­ply:

  • No one can model the tech­ni­cal ar­gu­ments in iso­la­tion. Ba­sic facts, such as the ac­cu­racy of tech­ni­cal pa­pers on AI, or the fil­ter­ing pro­cesses de­ter­min­ing what you read and what you don’t, de­pend on so­ciopoli­ti­cal phe­nom­ena. This is far more true for peo­ple who don’t them­selves have AI ex­per­tise.

  • “When AGI will be de­vel­oped” isn’t just a tech­ni­cal ques­tion. It de­pends on what peo­ple ac­tu­ally choose to do (and what groups of peo­ple ac­tu­ally suc­ceed in ac­com­plish­ing), not just what can be done in the­ory. And so ba­sic ques­tions like “how good is the episte­mol­ogy of the AI field about AI timelines?” mat­ter di­rectly.

  • The so­ciopoli­ti­cal phe­nom­ena are ac­tively mak­ing tech­ni­cal dis­cus­sion harder. I’ve had a well-re­puted per­son in the AI risk space dis­cour­age me from writ­ing pub­li­cly about the tech­ni­cal ar­gu­ments, on the ba­sis that get­ting peo­ple to think through them might ac­cel­er­ate AI timelines (yes, re­ally).

Which is not to say that mod­el­ing such tech­ni­cal ar­gu­ments is not im­por­tant for fore­cast­ing AGI. I cer­tainly could have writ­ten a post eval­u­at­ing such ar­gu­ments, and I de­cided to write this post in­stead, in part be­cause I don’t have much to say on this is­sue that Gary Mar­cus hasn’t already said. (Of course, I’d have writ­ten a sub­stan­tially differ­ent post, or none at all, if I be­lieved the tech­ni­cal ar­gu­ments that AGI is likely to come soon had merit to them)

What I’m not saying

I’m not say­ing:

  1. That deep learn­ing isn’t a ma­jor AI ad­vance.

  2. That deep learn­ing won’t sub­stan­tially change the world in the next 20 years (through nar­row AI).

  3. That I’m cer­tain that AGI isn’t com­ing in the next 20 years.

  4. That AGI isn’t ex­is­ten­tially im­por­tant on long timescales.

  5. That it isn’t pos­si­ble that some AI re­searchers have asym­met­ric in­for­ma­tion in­di­cat­ing that AGI is com­ing in the next 20 years. (Un­likely, but pos­si­ble)

  6. That peo­ple who have tech­ni­cal ex­per­tise shouldn’t be eval­u­at­ing tech­ni­cal ar­gu­ments on their mer­its.

  7. That most of what’s go­ing on is peo­ple con­sciously ly­ing. (Rather, covert de­cep­tion hid­den from con­scious at­ten­tion (e.g. mo­ti­vated rea­son­ing) is per­va­sive; see The Elephant in the Brain)

  8. That many peo­ple aren’t sincerely con­fused on the is­sue.

I’m say­ing that there are sys­tem­atic so­ciopoli­ti­cal phe­nom­ena that cause dis­tor­tions in AI es­ti­mates, es­pe­cially to­wards shorter timelines. I’m say­ing that peo­ple are be­ing duped into be­liev­ing a lie. And at the point where 73% of tech ex­ec­u­tives say they be­lieve AGI will be de­vel­oped in the next 10 years, it’s a ma­jor one.

This has hap­pened be­fore. And, in all like­li­hood, this will hap­pen again.