Belief in Intelligence

Since I am so un­cer­tain of Kas­parov’s moves, what is the em­piri­cal con­tent of my be­lief that “Kas­parov is a highly in­tel­li­gent chess player”? What real-world ex­pe­rience does my be­lief tell me to an­ti­ci­pate? Is it a clev­erly masked form of to­tal ig­no­rance?

To sharpen the dilemma, sup­pose Kas­parov plays against some mere chess grand­mas­ter Mr. G, who’s not in the run­ning for world cham­pion. My own abil­ity is far too low to dis­t­in­guish be­tween these lev­els of chess skill. When I try to guess Kas­parov’s move, or Mr. G’s next move, all I can do is try to guess “the best chess move” us­ing my own mea­ger knowl­edge of chess. Then I would pro­duce ex­actly the same pre­dic­tion for Kas­parov’s move or Mr. G’s move in any par­tic­u­lar chess po­si­tion. So what is the em­piri­cal con­tent of my be­lief that “Kas­parov is a bet­ter chess player than Mr. G”?

The em­piri­cal con­tent of my be­lief is the testable, falsifi­able pre­dic­tion that the fi­nal chess po­si­tion will oc­cupy the class of chess po­si­tions that are wins for Kas­parov, rather than drawn games or wins for Mr. G. (Count­ing res­ig­na­tion as a le­gal move that leads to a chess po­si­tion clas­sified as a loss.) The de­gree to which I think Kas­parov is a “bet­ter player” is re­flected in the amount of prob­a­bil­ity mass I con­cen­trate into the “Kas­parov wins” class of out­comes, ver­sus the “drawn game” and “Mr. G wins” class of out­comes. Th­ese classes are ex­tremely vague in the sense that they re­fer to vast spaces of pos­si­ble chess po­si­tions—but “Kas­parov wins” is more spe­cific than max­i­mum en­tropy, be­cause it can be definitely falsified by a vast set of chess po­si­tions.

The out­come of Kas­parov’s game is pre­dictable be­cause I know, and un­der­stand, Kas­parov’s goals. Within the con­fines of the chess board, I know Kas­parov’s mo­ti­va­tions—I know his suc­cess crite­rion, his util­ity func­tion, his tar­get as an op­ti­miza­tion pro­cess. I know where Kas­parov is ul­ti­mately try­ing to steer the fu­ture and I an­ti­ci­pate he is pow­er­ful enough to get there, al­though I don’t an­ti­ci­pate much about how Kas­parov is go­ing to do it.

Imag­ine that I’m vis­it­ing a dis­tant city, and a lo­cal friend vol­un­teers to drive me to the air­port. I don’t know the neigh­bor­hood. Each time my friend ap­proaches a street in­ter­sec­tion, I don’t know whether my friend will turn left, turn right, or con­tinue straight ahead. I can’t pre­dict my friend’s move even as we ap­proach each in­di­vi­d­ual in­ter­sec­tion—let alone, pre­dict the whole se­quence of moves in ad­vance.

Yet I can pre­dict the re­sult of my friend’s un­pre­dictable ac­tions: we will ar­rive at the air­port. Even if my friend’s house were lo­cated el­se­where in the city, so that my friend made a com­pletely differ­ent se­quence of turns, I would just as con­fi­dently pre­dict our ar­rival at the air­port. I can pre­dict this long in ad­vance, be­fore I even get into the car. My flight de­parts soon, and there’s no time to waste; I wouldn’t get into the car in the first place, if I couldn’t con­fi­dently pre­dict that the car would travel to the air­port along an un­pre­dictable path­way.

Isn’t this a re­mark­able situ­a­tion to be in, from a sci­en­tific per­spec­tive? I can pre­dict the out­come of a pro­cess, with­out be­ing able to pre­dict any of the in­ter­me­di­ate steps of the pro­cess.

How is this even pos­si­ble? Or­di­nar­ily one pre­dicts by imag­in­ing the pre­sent and then run­ning the vi­su­al­iza­tion for­ward in time. If you want a pre­cise model of the So­lar Sys­tem, one that takes into ac­count plane­tary per­tur­ba­tions, you must start with a model of all ma­jor ob­jects and run that model for­ward in time, step by step.

Some­times sim­pler prob­lems have a closed-form solu­tion, where calcu­lat­ing the fu­ture at time T takes the same amount of work re­gard­less of T. A coin rests on a table, and af­ter each minute, the coin turns over. The coin starts out show­ing heads. What face will it show a hun­dred min­utes later? Ob­vi­ously you did not an­swer this ques­tion by vi­su­al­iz­ing a hun­dred in­ter­ven­ing steps. You used a closed-form solu­tion that worked to pre­dict the out­come, and would also work to pre­dict any of the in­ter­ven­ing steps.

But when my friend drives me to the air­port, I can pre­dict the out­come suc­cess­fully us­ing a strange model that won’t work to pre­dict any of the in­ter­me­di­ate steps. My model doesn’t even re­quire me to in­put the ini­tial con­di­tions—I don’t need to know where we start out in the city!

I do need to know some­thing about my friend. I must know that my friend wants me to make my flight. I must credit that my friend is a good enough plan­ner to suc­cess­fully drive me to the air­port (if he wants to). Th­ese are prop­er­ties of my friend’s ini­tial state—prop­er­ties which let me pre­dict the fi­nal des­ti­na­tion, though not any in­ter­me­di­ate turns.

I must also credit that my friend knows enough about the city to drive suc­cess­fully. This may be re­garded as a re­la­tion be­tween my friend and the city; hence, a prop­erty of both. But an ex­tremely ab­stract prop­erty, which does not re­quire any spe­cific knowl­edge about ei­ther the city, or about my friend’s knowl­edge about the city.

This is one way of view­ing the sub­ject mat­ter to which I’ve de­voted my life—these re­mark­able situ­a­tions which place us in such an odd epistemic po­si­tions. And my work, in a sense, can be viewed as un­rav­el­ing the ex­act form of that strange ab­stract knowl­edge we can pos­sess; whereby, not know­ing the ac­tions, we can jus­tifi­ably know the con­se­quence.

“In­tel­li­gence” is too nar­row a term to de­scribe these re­mark­able situ­a­tions in full gen­er­al­ity. I would say rather “op­ti­miza­tion pro­cess”. A similar situ­a­tion ac­com­pa­nies the study of biolog­i­cal nat­u­ral se­lec­tion, for ex­am­ple; we can’t pre­dict the ex­act form of the next or­ganism ob­served.

But my own spe­cialty is the kind of op­ti­miza­tion pro­cess called “in­tel­li­gence”; and even nar­rower, a par­tic­u­lar kind of in­tel­li­gence called “Friendly Ar­tifi­cial In­tel­li­gence”—of which, I hope, I will be able to ob­tain es­pe­cially pre­cise ab­stract knowl­edge.