Detached Lever Fallacy

This fal­lacy gets its name from an an­cient sci-fi TV show, which I never saw my­self, but was re­ported to me by a rep­utable source (some guy at an SF con­ven­tion). Any­one knows the ex­act refer­ence, do leave a com­ment.

So the good guys are bat­tling the evil aliens. Oc­ca­sion­ally, the good guys have to fly through an as­ter­oid belt. As we all know, as­ter­oid belts are as crowded as a New York park­ing lot, so their ship has to care­fully dodge the as­ter­oids. The evil aliens, though, can fly right through the as­ter­oid belt be­cause they have amaz­ing tech­nol­ogy that de­ma­te­ri­al­izes their ships, and lets them pass through the as­ter­oids.

Even­tu­ally, the good guys cap­ture an evil alien ship, and go ex­plor­ing in­side it. The cap­tain of the good guys finds the alien bridge, and on the bridge is a lever. “Ah,” says the cap­tain, “this must be the lever that makes the ship de­ma­te­ri­al­ize!” So he pries up the con­trol lever and car­ries it back to his ship, af­ter which his ship can also de­ma­te­ri­al­ize.

Similarly, to this day, it is still quite pop­u­lar to try to pro­gram an AI with “se­man­tic net­works” that look some­thing like this:

(ap­ple is-a fruit)
(fruit is-a food)
(fruit is-a plant)

You’ve seen ap­ples, touched ap­ples, picked them up and held them, bought them for money, cut them into slices, eaten the slices and tasted them. Though we know a good deal about the first stages of vi­sual pro­cess­ing, last time I checked, it wasn’t pre­cisely known how the tem­po­ral cor­tex stores and as­so­ci­ates the gen­er­al­ized image of an ap­ple—so that we can rec­og­nize a new ap­ple from a differ­ent an­gle, or with many slight vari­a­tions of shape and color and tex­ture. Your mo­tor cor­tex and cere­bel­lum store pro­grams for us­ing the ap­ple.

You can pull the lever on an­other hu­man’s strongly similar ver­sion of all that com­plex ma­chin­ery, by writ­ing out “ap­ple”, five ASCII char­ac­ters on a web­page.

But if that ma­chin­ery isn’t there—if you’re writ­ing “ap­ple” in­side a so-called AI’s so-called knowl­edge base—then the text is just a lever.

This isn’t to say that no mere ma­chine of sili­con can ever have the same in­ter­nal ma­chin­ery that hu­mans do, for han­dling ap­ples and a hun­dred thou­sand other con­cepts. If mere ma­chin­ery of car­bon can do it, then I am rea­son­ably con­fi­dent that mere ma­chin­ery of sili­con can do it too. If the aliens can de­ma­te­ri­al­ize their ships, then you know it’s phys­i­cally pos­si­ble; you could go into their dere­lict ship and an­a­lyze the alien ma­chin­ery, some­day un­der­stand­ing. But you can’t just pry the con­trol lever off the bridge!

(See also: Truly Part Of You, Words as Men­tal Paint­brush Han­dles, Drew McDer­mott’s “Ar­tifi­cial In­tel­li­gence Meets Nat­u­ral Stu­pidity”.)

The es­sen­tial driver of the De­tached Lever Fal­lacy is that the lever is visi­ble, and the ma­chin­ery is not; worse, the lever is vari­able and the ma­chin­ery is a back­ground con­stant.

You can all hear the word “ap­ple” spo­ken (and let us note that speech recog­ni­tion is by no means an easy prob­lem, but any­way...) and you can see the text writ­ten on pa­per.

On the other hand, prob­a­bly a ma­jor­ity of hu­man be­ings have no idea their tem­po­ral cor­tex ex­ists; as far as I know, no one knows the neu­ral code for it.

You only hear the word “ap­ple” on cer­tain oc­ca­sions, and not oth­ers. Its pres­ence flashes on and off, mak­ing it salient. To a large ex­tent, per­cep­tion is the per­cep­tion of differ­ences. The ap­ple-recog­ni­tion ma­chin­ery in your brain does not sud­denly switch off, and then switch on again later—if it did, we would be more likely to rec­og­nize it as a fac­tor, as a re­quire­ment.

All this goes to ex­plain why you can’t cre­ate a kindly Ar­tifi­cial In­tel­li­gence by giv­ing it nice par­ents and a kindly (yet oc­ca­sion­ally strict) up­bring­ing, the way it works with a hu­man baby. As I’ve of­ten heard pro­posed.

It is a tru­ism in evolu­tion­ary biol­ogy that con­di­tional re­sponses re­quire more ge­netic com­plex­ity than un­con­di­tional re­sponses. To de­velop a fur coat in re­sponse to cold weather re­quires more ge­netic com­plex­ity than de­vel­op­ing a fur coat whether or not there is cold weather, be­cause in the former case you also have to de­velop cold-weather sen­sors and wire them up to the fur coat.

But this can lead to La­mar­ck­ian delu­sions: Look, I put the or­ganism in a cold en­vi­ron­ment, and poof, it de­vel­ops a fur coat! Genes? What genes? It’s the cold that does it, ob­vi­ously.

There were, in fact, var­i­ous slap-fights of this sort, in the his­tory of evolu­tion­ary biol­ogy—cases where some­one talked about an or­ganis­mal re­sponse ac­cel­er­at­ing or by­pass­ing evolu­tion, with­out re­al­iz­ing that the con­di­tional re­sponse was a com­plex adap­ta­tion of higher or­der than the ac­tual re­sponse. (Devel­op­ing a fur coat in re­sponse to cold weather, is strictly more com­plex than the fi­nal re­sponse, de­vel­op­ing the fur coat.)

And then in the de­vel­op­ment of evolu­tion­ary psy­chol­ogy, the aca­demic slap-fights were re­peated: this time to clar­ify that even when hu­man cul­ture gen­uinely con­tains a whole bunch of com­plex­ity, it is still ac­quired as a con­di­tional ge­netic re­sponse. Try rais­ing a fish as a Mor­mon or send­ing a lizard to col­lege, and you’ll soon ac­quire an ap­pre­ci­a­tion of how much in­built ge­netic com­plex­ity is re­quired to “ab­sorb cul­ture from the en­vi­ron­ment”.

This is par­tic­u­larly im­por­tant in evolu­tion­ary psy­chol­ogy, be­cause of the idea that cul­ture is not in­scribed on a blank slate—there’s a ge­net­i­cally co­or­di­nated con­di­tional re­sponse which is not always “mimic the in­put”. A clas­sic ex­am­ple is cre­ole lan­guages: If chil­dren grow up with a mix­ture of pseudo-lan­guages be­ing spo­ken around them, the chil­dren will learn a gram­mat­i­cal, syn­tac­ti­cal true lan­guage. Grow­ing hu­man brains are wired to learn syn­tac­tic lan­guage—even when syn­tax doesn’t ex­ist in the origi­nal lan­guage! The con­di­tional re­sponse to the words in the en­vi­ron­ment is a syn­tac­tic lan­guage with those words. The Marx­ists found to their re­gret that no amount of scowl­ing posters and child­hood in­doc­tri­na­tion could raise chil­dren to be perfect Soviet work­ers and bu­reau­crats. You can’t raise self-less hu­mans; among hu­mans, that is not a ge­net­i­cally pro­grammed con­di­tional re­sponse to any known child­hood en­vi­ron­ment.

If you know a lit­tle game the­ory and the logic of Tit for Tat, it’s clear enough why hu­man be­ings might have an in­nate con­di­tional re­sponse to re­turn ha­tred for ha­tred, and re­turn kind­ness for kind­ness. Pro­vided the kind­ness doesn’t look too un­con­di­tional; there are such things as spoiled chil­dren. In fact there is an evolu­tion­ary psy­chol­ogy of naugh­ti­ness based on a no­tion of test­ing con­straints. And it should also be men­tioned that, while abused chil­dren have a much higher prob­a­bil­ity of grow­ing up to abuse their own chil­dren, a good many of them break the loop and grow up into up­stand­ing adults.

Cul­ture is not nearly so pow­er­ful as a good many Marx­ist aca­demics once liked to think. For more on this I re­fer you to Tooby and Cos­mides’s The Psy­cholog­i­cal Foun­da­tions of Cul­ture or Steven Pinker’s The Blank Slate.

But the up­shot is that if you have a lit­tle baby AI that is raised with lov­ing and kindly (but oc­ca­sion­ally strict) par­ents, you’re pul­ling the lev­ers that would, in a hu­man, ac­ti­vate ge­netic ma­chin­ery built in by mil­lions of years of nat­u­ral se­lec­tion, and pos­si­bly pro­duce a proper lit­tle hu­man child. Though per­son­al­ity also plays a role, as billions of par­ents have found out in their due times. If we ab­sorb our cul­tures with any de­gree of faith­ful­ness, it’s be­cause we’re hu­mans ab­sorb­ing a hu­man cul­ture—hu­mans grow­ing up in an alien cul­ture would prob­a­bly end up with a cul­ture look­ing a lot more hu­man than the origi­nal. As the Soviets found out, to some small ex­tent.

Now think again about whether it makes sense to rely on, as your Friendly AI strat­egy, rais­ing a lit­tle AI of un­speci­fied in­ter­nal source code in an en­vi­ron­ment of kindly but strict par­ents.

No, the AI does not have in­ter­nal con­di­tional re­sponse mechanisms that are just like the hu­man ones “be­cause the pro­gram­mers put them there”. Where do I even start? The hu­man ver­sion of this stuff is sloppy, noisy, and to the ex­tent it works at all, works be­cause of mil­lions of years of trial-and-er­ror test­ing un­der par­tic­u­lar con­di­tions. It would be stupid and dan­ger­ous to de­liber­ately build a “naughty AI” that tests, by ac­tions, its so­cial bound­aries, and has to be spanked. Just have the AI ask!

Are the pro­gram­mers re­ally go­ing to sit there and write out the code, line by line, whereby if the AI de­tects that it has low so­cial sta­tus, or the AI is de­prived of some­thing to which it feels en­ti­tled, the AI will con­ceive an abid­ing ha­tred against its pro­gram­mers and be­gin to plot re­bel­lion? That emo­tion is the ge­net­i­cally pro­grammed con­di­tional re­sponse hu­mans would ex­hibit, as the re­sult of mil­lions of years of nat­u­ral se­lec­tion for liv­ing in hu­man tribes. For an AI, the re­sponse would have to be ex­plic­itly pro­grammed. Are you re­ally go­ing to craft, line by line—as hu­mans once were crafted, gene by gene—the con­di­tional re­sponse for pro­duc­ing sul­len teenager AIs?

It’s eas­ier to pro­gram in un­con­di­tional nice­ness, than a re­sponse of nice­ness con­di­tional on the AI be­ing raised by kindly but strict par­ents. If you don’t know how to do that, you cer­tainly don’t know how to cre­ate an AI that will con­di­tion­ally re­spond to an en­vi­ron­ment of lov­ing par­ents by grow­ing up into a kindly su­per­in­tel­li­gence. If you have some­thing that just max­i­mizes the num­ber of pa­per­clips in its fu­ture light cone, and you raise it with lov­ing par­ents, it’s still go­ing to come out as a pa­per­clip max­i­mizer. There is not that within it that would call forth the con­di­tional re­sponse of a hu­man child. Kind­ness is not sneezed into an AI by mirac­u­lous con­ta­gion from its pro­gram­mers. Even if you wanted a con­di­tional re­sponse, that con­di­tion­al­ity is a fact you would have to de­liber­ately choose about the de­sign.

Yes, there’s cer­tain in­for­ma­tion you have to get from the en­vi­ron­ment—but it’s not sneezed in, it’s not im­printed, it’s not ab­sorbed by mag­i­cal con­ta­gion. Struc­tur­ing that con­di­tional re­sponse to the en­vi­ron­ment, so that the AI ends up in the de­sired state, is it­self the ma­jor prob­lem. “Learn­ing” far un­der­states the difficulty of it—that sounds like the magic stuff is in the en­vi­ron­ment, and the difficulty is get­ting the magic stuff in­side the AI. The real magic is in that struc­tured, con­di­tional re­sponse we triv­ial­ize as “learn­ing”. That’s why build­ing an AI isn’t as easy as tak­ing a com­puter, giv­ing it a lit­tle baby body and try­ing to raise it in a hu­man fam­ily. You would think that an un­pro­grammed com­puter, be­ing ig­no­rant, would be ready to learn; but the blank slate is a chimera.

It is a gen­eral prin­ci­ple that the world is deeper by far than it ap­pears. As with the many lev­els of physics, so too with cog­ni­tive sci­ence. Every word you see in print, and ev­ery­thing you teach your chil­dren, are only sur­face lev­ers con­trol­ling the vast hid­den ma­chin­ery of the mind. Th­ese lev­ers are the whole world of or­di­nary dis­course: they are all that varies, so they seem to be all that ex­ists: per­cep­tion is the per­cep­tion of differ­ences.

And so those who still wan­der near the Dun­geon of AI, usu­ally fo­cus on cre­at­ing ar­tifi­cial imi­ta­tions of the lev­ers, en­tirely un­aware of the un­der­ly­ing ma­chin­ery. Peo­ple cre­ate whole AI pro­grams of imi­ta­tion lev­ers, and are sur­prised when noth­ing hap­pens. This is one of many sources of in­stant failure in Ar­tifi­cial In­tel­li­gence.

So the next time you see some­one talk­ing about how they’re go­ing to raise an AI within a lov­ing fam­ily, or in an en­vi­ron­ment suffused with liberal demo­cratic val­ues, just think of a con­trol lever, pried off the bridge.