Artificial Addition

Sup­pose that hu­man be­ings had ab­solutely no idea how they performed ar­ith­metic. Imag­ine that hu­man be­ings had evolved, rather than hav­ing learned, the abil­ity to count sheep and add sheep. Peo­ple us­ing this built-in abil­ity have no idea how it worked, the way Aris­to­tle had no idea how his vi­sual cor­tex sup­ported his abil­ity to see things. Peano Arith­metic as we know it has not been in­vented. There are philoso­phers work­ing to for­mal­ize nu­mer­i­cal in­tu­itions, but they em­ploy no­ta­tions such as

Plus-Of(Seven, Six) = Thirteen

to for­mal­ize the in­tu­itively ob­vi­ous fact that when you add “seven” plus “six”, of course you get “thir­teen”.

In this world, pocket calcu­la­tors work by stor­ing a gi­ant lookup table of ar­ith­meti­cal facts, en­tered man­u­ally by a team of ex­pert Ar­tifi­cial Arith­meti­ci­ans, for start­ing val­ues that range be­tween zero and one hun­dred. While these calcu­la­tors may be helpful in a prag­matic sense, many philoso­phers ar­gue that they’re only simu­lat­ing ad­di­tion, rather than re­ally adding. No ma­chine can re­ally count—that’s why hu­mans have to count thir­teen sheep be­fore typ­ing “thir­teen” into the calcu­la­tor. Calcu­la­tors can re­cite back stored facts, but they can never know what the state­ments mean—if you type in “two hun­dred plus two hun­dred” the calcu­la­tor says “Er­ror: Ou­trange”, when it’s in­tu­itively ob­vi­ous, if you know what the words mean, that the an­swer is “four hun­dred”.

Philoso­phers, of course, are not so naive as to be taken in by these in­tu­itions. Num­bers are re­ally a purely for­mal sys­tem—the la­bel “thirty-seven” is mean­ingful, not be­cause of any in­her­ent prop­erty of the words them­selves, but be­cause the la­bel refers to thirty-seven sheep in the ex­ter­nal world. A num­ber is given this refer­en­tial prop­erty by its se­man­tic net­work of re­la­tions to other num­bers. That’s why, in com­puter pro­grams, the LISP to­ken for “thirty-seven” doesn’t need any in­ter­nal struc­ture—it’s only mean­ingful be­cause of refer­ence and re­la­tion, not some com­pu­ta­tional prop­erty of “thirty-seven” it­self.

No one has ever de­vel­oped an Ar­tifi­cial Gen­eral Arith­meti­cian, though of course there are plenty of do­main-spe­cific, nar­row Ar­tifi­cial Arith­meti­ci­ans that work on num­bers be­tween “twenty” and “thirty”, and so on. And if you look at how slow progress has been on num­bers in the range of “two hun­dred”, then it be­comes clear that we’re not go­ing to get Ar­tifi­cial Gen­eral Arith­metic any time soon. The best ex­perts in the field es­ti­mate it will be at least a hun­dred years be­fore calcu­la­tors can add as well as a hu­man twelve-year-old.

But not ev­ery­one agrees with this es­ti­mate, or with merely con­ven­tional be­liefs about Ar­tifi­cial Arith­metic. It’s com­mon to hear state­ments such as the fol­low­ing:

  • “It’s a fram­ing prob­lem—what ‘twenty-one plus’ equals de­pends on whether it’s ‘plus three’ or ‘plus four’. If we can just get enough ar­ith­meti­cal facts stored to cover the com­mon-sense truths that ev­ery­one knows, we’ll start to see real ad­di­tion in the net­work.”

  • “But you’ll never be able to pro­gram in that many ar­ith­meti­cal facts by hiring ex­perts to en­ter them man­u­ally. What we need is an Ar­tifi­cial Arith­meti­cian that can learn the vast net­work of re­la­tions be­tween num­bers that hu­mans ac­quire dur­ing their child­hood by ob­serv­ing sets of ap­ples.”

  • “No, what we re­ally need is an Ar­tifi­cial Arith­meti­cian that can un­der­stand nat­u­ral lan­guage, so that in­stead of hav­ing to be ex­plic­itly told that twenty-one plus six­teen equals thirty-seven, it can get the knowl­edge by ex­plor­ing the Web.”

  • “Frankly, it seems to me that you’re just try­ing to con­vince your­selves that you can solve the prob­lem. None of you re­ally know what ar­ith­metic is, so you’re flounder­ing around with these generic sorts of ar­gu­ments. ‘We need an AA that can learn X’, ‘We need an AA that can ex­tract X from the In­ter­net’. I mean, it sounds good, it sounds like you’re mak­ing progress, and it’s even good for pub­lic re­la­tions, be­cause ev­ery­one thinks they un­der­stand the pro­posed solu­tion—but it doesn’t re­ally get you any closer to gen­eral ad­di­tion, as op­posed to do­main-spe­cific ad­di­tion. Prob­a­bly we will never know the fun­da­men­tal na­ture of ar­ith­metic. The prob­lem is just too hard for hu­mans to solve.”

  • “That’s why we need to de­velop a gen­eral ar­ith­meti­cian the same way Na­ture did—evolu­tion.”

  • “Top-down ap­proaches have clearly failed to pro­duce ar­ith­metic. We need a bot­tom-up ap­proach, some way to make ar­ith­metic emerge. We have to ac­knowl­edge the ba­sic un­pre­dictabil­ity of com­plex sys­tems.”

  • “You’re all wrong. Past efforts to cre­ate ma­chine ar­ith­metic were fu­tile from the start, be­cause they just didn’t have enough com­put­ing power. If you look at how many trillions of synapses there are in the hu­man brain, it’s clear that calcu­la­tors don’t have lookup ta­bles any­where near that large. We need calcu­la­tors as pow­er­ful as a hu­man brain. Ac­cord­ing to Moore’s Law, this will oc­cur in the year 2031 on April 27 be­tween 4:00 and 4:30 in the morn­ing.”

  • “I be­lieve that ma­chine ar­ith­metic will be de­vel­oped when re­searchers scan each neu­ron of a com­plete hu­man brain into a com­puter, so that we can simu­late the biolog­i­cal cir­cuitry that performs ad­di­tion in hu­mans.”

  • “I don’t think we have to wait to scan a whole brain. Neu­ral net­works are just like the hu­man brain, and you can train them to do things with­out know­ing how they do them. We’ll cre­ate pro­grams that will do ar­ith­metic with­out we, our cre­ators, ever un­der­stand­ing how they do ar­ith­metic.”

  • “But Gödel’s The­o­rem shows that no for­mal sys­tem can ever cap­ture the ba­sic prop­er­ties of ar­ith­metic. Clas­si­cal physics is for­mal­iz­able, so to add two and two, the brain must take ad­van­tage of quan­tum physics.”

  • “Hey, if hu­man ar­ith­metic were sim­ple enough that we could re­pro­duce it in a com­puter, we wouldn’t be able to count high enough to build com­put­ers.”

  • “Haven’t you heard of John Searle’s Chi­nese Calcu­la­tor Ex­per­i­ment? Even if you did have a huge set of rules that would let you add ‘twenty-one’ and ‘six­teen’, just imag­ine trans­lat­ing all the words into Chi­nese, and you can see that there’s no gen­uine ad­di­tion go­ing on. There are no real num­bers any­where in the sys­tem, just la­bels that hu­mans use for num­bers...”

There is more than one moral to this parable, and I have told it with differ­ent morals in differ­ent con­texts. It illus­trates the idea of lev­els of or­ga­ni­za­tion, for ex­am­ple—a CPU can add two large num­bers be­cause the num­bers aren’t black-box opaque ob­jects, they’re or­dered struc­tures of 32 bits.

But for pur­poses of over­com­ing bias, let us draw two morals:

  • First, the dan­ger of be­liev­ing as­ser­tions you can’t re­gen­er­ate from your own knowl­edge.

  • Se­cond, the dan­ger of try­ing to dance around ba­sic con­fu­sions.

Lest any­one ac­cuse me of gen­er­al­iz­ing from fic­tional ev­i­dence, both les­sons may be drawn from the real his­tory of Ar­tifi­cial In­tel­li­gence as well.

The first dan­ger is the ob­ject-level prob­lem that the AA de­vices ran into: they func­tioned as tape recorders play­ing back “knowl­edge” gen­er­ated from out­side the sys­tem, us­ing a pro­cess they couldn’t cap­ture in­ter­nally. A hu­man could tell the AA de­vice that “twenty-one plus six­teen equals thirty-seven”, and the AA de­vices could record this sen­tence and play it back, or even pat­tern-match “twenty-one plus six­teen” to out­put “thirty-seven!”, but the AA de­vices couldn’t gen­er­ate such knowl­edge for them­selves.

Which is strongly rem­i­nis­cent of be­liev­ing a physi­cist who tells you “Light is waves”, record­ing the fas­ci­nat­ing words and play­ing them back when some­one asks “What is light made of?”, with­out be­ing able to gen­er­ate the knowl­edge for your­self. More on this theme to­mor­row.

The sec­ond moral is the meta-level dan­ger that con­sumed the Ar­tifi­cial Arith­metic re­searchers and opinionated by­stan­ders—the dan­ger of danc­ing around con­fus­ing gaps in your knowl­edge. The ten­dency to do just about any­thing ex­cept grit your teeth and buckle down and fill in the damn gap.

Whether you say, “It is emer­gent!”, or whether you say, “It is un­know­able!”, in nei­ther case are you ac­knowl­edg­ing that there is a ba­sic in­sight re­quired which is pos­sess­able, but un­pos­sessed by you.

How can you know when you’ll have a new ba­sic in­sight? And there’s no way to get one ex­cept by bang­ing your head against the prob­lem, learn­ing ev­ery­thing you can about it, study­ing it from as many an­gles as pos­si­ble, per­haps for years. It’s not a pur­suit that academia is set up to per­mit, when you need to pub­lish at least one pa­per per month. It’s cer­tainly not some­thing that ven­ture cap­i­tal­ists will fund. You want to ei­ther go ahead and build the sys­tem now, or give up and do some­thing else in­stead.

Look at the com­ments above: none are aimed at set­ting out on a quest for the miss­ing in­sight which would make num­bers no longer mys­te­ri­ous, make “twenty-seven” more than a black box. None of the com­menters re­al­ized that their difficul­ties arose from ig­no­rance or con­fu­sion in their own minds, rather than an in­her­ent prop­erty of ar­ith­metic. They were not try­ing to achieve a state where the con­fus­ing thing ceased to be con­fus­ing.

If you read Judea Pearl’s “Prob­a­bil­is­tic Rea­son­ing in In­tel­li­gent Sys­tems: Net­works of Plau­si­ble In­fer­ence” then you will see that the ba­sic in­sight be­hind graph­i­cal mod­els is in­dis­pens­able to prob­lems that re­quire it. (It’s not some­thing that fits on a T-Shirt, I’m afraid, so you’ll have to go and read the book your­self. I haven’t seen any on­line pop­u­lariza­tions of Bayesian net­works that ad­e­quately con­vey the rea­sons be­hind the prin­ci­ples, or the im­por­tance of the math be­ing ex­actly the way it is, but Pearl’s book is won­der­ful.) There were once dozens of “non-mono­tonic log­ics” awk­wardly try­ing to cap­ture in­tu­itions such as “If my bur­glar alarm goes off, there was prob­a­bly a bur­glar, but if I then learn that there was a small earth­quake near my home, there was prob­a­bly not a bur­glar.” With the graph­i­cal-model in­sight in hand, you can give a math­e­mat­i­cal ex­pla­na­tion of ex­actly why first-or­der logic has the wrong prop­er­ties for the job, and ex­press the cor­rect solu­tion in a com­pact way that cap­tures all the com­mon-sense de­tails in one el­e­gant swoop. Un­til you have that in­sight, you’ll go on patch­ing the logic here, patch­ing it there, adding more and more hacks to force it into cor­re­spon­dence with ev­ery­thing that seems “ob­vi­ously true”.

You won’t know the Ar­tifi­cial Arith­metic prob­lem is un­solv­able with­out its key. If you don’t know the rules, you don’t know the rule that says you need to know the rules to do any­thing. And so there will be all sorts of clever ideas that seem like they might work, like build­ing an Ar­tifi­cial Arith­meti­cian that can read nat­u­ral lan­guage and down­load mil­lions of ar­ith­meti­cal as­ser­tions from the In­ter­net.

And yet some­how the clever ideas never work. Some­how it always turns out that you “couldn’t see any rea­son it wouldn’t work” be­cause you were ig­no­rant of the ob­sta­cles, not be­cause no ob­sta­cles ex­isted. Like shoot­ing blind­folded at a dis­tant tar­get—you can fire blind shot af­ter blind shot, cry­ing, “You can’t prove to me that I won’t hit the cen­ter!” But un­til you take off the blind­fold, you’re not even in the aiming game. When “no one can prove to you” that your pre­cious idea isn’t right, it means you don’t have enough in­for­ma­tion to strike a small tar­get in a vast an­swer space. Un­til you know your idea will work, it won’t.

From the his­tory of pre­vi­ous key in­sights in Ar­tifi­cial In­tel­li­gence, and the grand messes which were pro­posed prior to those in­sights, I de­rive an im­por­tant real-life les­son: When the ba­sic prob­lem is your ig­no­rance, clever strate­gies for by­pass­ing your ig­no­rance lead to shoot­ing your­self in the foot.