The Critical Rationalist View on Artificial Intelligence

Crit­i­cal Ra­tion­al­ism (CR) is be­ing dis­cussed on some threads here at Less Wrong (e.g., here, here, and here). It is some­thing that Crit­i­cal Ra­tion­al­ists such as my­self think con­trib­u­tors to Less Wrong need to un­der­stand much bet­ter. Crit­i­cal Ra­tion­al­ists claim that CR is the only vi­able fully-fledged episte­mol­ogy known. They claim that cur­rent at­tempts to spec­ify a Bayesian/​In­duc­tivist episte­mol­ogy are not only in­com­plete but can­not work at all. The pur­pose of this post is not to ar­gue these claims in depth but to sum­ma­rize the Crit­i­cal Ra­tion­al­ist view on AI and also how this speaks to things like the Friendly AI Prob­lem. Some of the ideas here may con­flict with ideas you think are true, but un­der­stand that these ideas have been worked on by some of the smartest peo­ple on the planet, both now and in the past. They de­serve care­ful con­sid­er­a­tion, not a drive past. Less Wrong says it is one of the ur­gent prob­lems of the world that progress is made on AI. If smart peo­ple in the know are say­ing that CR is needed to make that progress, and if you are an AI re­searcher who ig­nores them, then you are not tak­ing the AI ur­gency prob­lem se­ri­ously.

Univer­sal Knowl­edge Creators

Crit­i­cal Ra­tion­al­ism [1] says that hu­man be­ings are uni­ver­sal knowl­edge cre­ators. This means we can cre­ate any knowl­edge which it is pos­si­ble to cre­ate. As Karl Pop­per first re­al­ized, the way we do this is by guess­ing ideas and by us­ing crit­i­cism to find er­rors in our guesses. Our guesses may be wrong, in which case we try to make bet­ter guesses in the light of what we know from the crit­i­cisms so far. The crit­i­cisms them­selves can be crit­i­cized and we can and do change those. All of this con­sti­tutes an evolu­tion­ary pro­cess. Like biolog­i­cal evolu­tion, it is an ex­am­ple of evolu­tion in ac­tion. This pro­cess is fal­lible: guaran­teed cer­tain knowl­edge is not pos­si­ble be­cause we can never know how an er­ror might be ex­posed in the fu­ture. The best we can do is ac­cept a guessed idea which has with­stood all known crit­i­cisms. If we can­not find such, then we have a new prob­lem situ­a­tion about how to pro­ceed and we try to solve that. [2]

Crit­i­cal Ra­tion­al­ism says that an en­tity is ei­ther a uni­ver­sal knowl­edge cre­ator or it is not. There is no such thing as a par­tially uni­ver­sal knowl­edge cre­ator. So an­i­mals such as dogs are not uni­ver­sal knowl­edge cre­ators — they have no abil­ity what­so­ever to cre­ate knowl­edge. What they have are al­gorithms pre-pro­grammed by biolog­i­cal evolu­tion that can be, roughly speak­ing, pa­ram­e­ter-tuned. Th­ese al­gorithms are so­phis­ti­cated and clever and be­yond what hu­mans can cur­rently pro­gram, but they do not con­fer any knowl­edge cre­ation abil­ity. Your pet dog will not move be­yond its reper­toire of pre-pro­grammed abil­ities and start writ­ing posts to Less Wrong. Dogs’ brains are uni­ver­sal com­put­ers, how­ever, so it would be pos­si­ble in prin­ci­ple to re­pro­gram your dog’s brain so that it be­comes a uni­ver­sal knowl­edge cre­ator. This would a re­mark­able feat be­cause it would re­quire knowl­edge of how to pro­gram an AI and also of how to phys­i­cally carry out the re­pro­gram­ming, but your dog would no longer be con­fined to its pre-pro­grammed reper­toire: it would be a per­son.

The rea­son there are no par­tially uni­ver­sal knowl­edge cre­ators is similar to the rea­son there are no par­tially uni­ver­sal com­put­ers. Univer­sal­ity is cheap. It is why wash­ing ma­chines have gen­eral pur­pose chips and dog’s brains are uni­ver­sal com­put­ers. Mak­ing a par­tially uni­ver­sal de­vice is much harder than mak­ing a fully uni­ver­sal one so bet­ter just to make a uni­ver­sal one and pro­gram it. The CR method de­scribed above for how peo­ple cre­ate knowl­edge is uni­ver­sal be­cause there are no limits to the prob­lems it ap­plies to. How would one limit it to just a sub­set of prob­lems? To im­ple­ment that would be much harder than im­ple­ment­ing the fully uni­ver­sal ver­sion. So if you meet an en­tity that can cre­ate some knowl­edge, it will have the ca­pa­bil­ity for uni­ver­sal knowl­edge cre­ation.

Th­ese ideas im­ply that AI is an all-or-none propo­si­tion. It will not come about by de­grees where there is a pro­gres­sion of en­tities that can solve an ever widen­ing reper­toire of prob­lems. There will be no climb up such a slope. In­stead, it will hap­pen as a jump: a jump to uni­ver­sal­ity. This is in fact how in­tel­li­gence arose in hu­mans. Some change—it may have been a small change—crossed a bound­ary and our an­ces­tors went from hav­ing no abil­ity to cre­ate knowl­edge to a fully uni­ver­sal abil­ity. This kind of jump to uni­ver­sal­ity hap­pens in other sys­tems too. David Deutsch dis­cusses ex­am­ples in his book The Begin­ning of In­finity.

Peo­ple will point to sys­tems like AlphaGo, the Go play­ing pro­gram, and claim it is a counter-ex­am­ple to the jump-to-uni­ver­sal­ity idea. They will say that AlphaGo is a step on a con­tinuum that leads to hu­man level in­tel­li­gence and be­yond. But it is not. Like the al­gorithms in a dog’s brain, AlphaGo is a re­mark­able al­gorithm, but it can­not cre­ate knowl­edge in even a sub­set of con­texts. It can­not learn how to ride a bi­cy­cle or post to Less Wrong. If it could do such things it would already be fully uni­ver­sal, as ex­plained above. Like the dog’s brain, AlphaGo uses knowl­edge that was put there by some­thing else: for the dog it was by evolu­tion, and for AlphaGo it was by its pro­gram­mers; they ex­pended the cre­ativity.

As hu­man be­ings are already uni­ver­sal knowl­edge cre­ators, no AI can ex­ist at a higher level. They may have bet­ter hard­ware and more mem­ory etc, but they will not have bet­ter knowl­edge cre­ation po­ten­tial than us. Even the hard­ware/​mem­ory ad­van­tage of AI is not much of an ad­van­tage for hu­man be­ings already aug­ment their in­tel­li­gence with de­vices such as pen­cil-and-pa­per and com­put­ers and we will con­tinue to do so.

Be­com­ing Smarter

Crit­i­cal Ra­tion­al­ism, then, says AI can­not re­cur­sively self-im­prove so that it ac­quires knowl­edge cre­ation po­ten­tial be­yond what hu­man be­ings already have. It will be able to be­come smarter through learn­ing but only in the same way that hu­mans are able to be­come smarter: by ac­quiring knowl­edge and, in par­tic­u­lar, by ac­quiring knowl­edge about how to be­come smarter. And, most of all, by learn­ing good philos­o­phy for it is in that field we learn how to think bet­ter and how to live bet­ter. All this knowl­edge can only be learned through the cre­ative pro­cess of guess­ing ideas and er­ror-cor­rec­tion by crit­i­cism for it is the only known way in­tel­li­gences can cre­ate knowl­edge.

It might be ar­gued that AI’s will be­come smarter much faster than we can be­cause they will have much faster hard­ware. In re­gard to knowl­edge cre­ation, how­ever, there is no di­rect con­nec­tion be­tween speed of knowl­edge cre­ation and un­der­ly­ing hard­ware speed. Hu­mans do not use the com­pu­ta­tional re­sources of their brains to the max­i­mum. This is not the bot­tle­neck to us be­com­ing smarter faster. It will not be for AI ei­ther. How fast you can cre­ate knowl­edge de­pends on things like what other knowl­edge you have and some ideas may be block­ing other ideas. You might have a prob­lem with static memes (see The Begin­ning of In­finity), for ex­am­ple, and these could be caus­ing bias, self-de­cep­tion, and other is­sues. AI’s will be sus­cep­ti­ble to static memes, too, be­cause memes are highly adapted ideas evolved to repli­cate via minds.

Tak­ing Chil­dren Seriously

One im­pli­ca­tion of the ar­gu­ments above is that AI’s will need par­ent­ing, just as we must par­ent our chil­dren. CR has a par­ent­ing the­ory called Tak­ing Chil­dren Se­ri­ously (TCS). It should not be sur­pris­ing that CR has such a the­ory for CR is af­ter all about learn­ing and how we ac­quire knowl­edge. Un­for­tu­nately, TCS is not it­self taken se­ri­ously by most peo­ple who first hear about it be­cause it con­flicts with a lot of con­ven­tional wis­dom about par­ent­ing. It gets dis­missed as “ex­trem­ist” or “nutty”, as if these were good crit­i­cisms rather than just the smears they ac­tu­ally are. Nev­er­the­less, TCS is im­por­tant and it is im­por­tant for those who wish to raise an AI.

One idea TCS has is that we must not thwart our chil­dren’s ra­tio­nal­ity, for ex­am­ple, by pres­sur­ing them and mak­ing them do things they do not want to do. This is dam­ag­ing to their in­tel­lec­tual de­vel­op­ment and can lead to them dis­re­spect­ing ra­tio­nal­ity. We must per­suade us­ing rea­son and this im­plies be­ing pre­pared for the pos­si­bil­ity we are wrong about what­ever mat­ter was in ques­tion. Com­mon par­ent­ing prac­tices to­day are far from op­ti­mally ra­tio­nal and are dam­ag­ing to chil­dren’s ra­tio­nal­ity.

Ar­tifi­cial In­tel­li­gence will have the same prob­lem of bad par­ent­ing prac­tices and this will also harm their in­tel­lec­tual de­vel­op­ment. So AI re­searchers should be think­ing right now about how to pre­vent this. They need to learn how to par­ent their AI’s well. For if not, AI’s will be be­set by the same prob­lems our chil­dren cur­rently face. CR says we already have the solu­tion: TCS. CR and TCS are in fact nec­es­sary to do AI in the first place.

Crit­i­cal Ra­tion­al­ism and TCS say you can­not up­load knowl­edge into an AI. The idea that you can is a ver­sion of the bucket the­ory of the mind which says that “there is noth­ing in our in­tel­lect which has not en­tered it through the senses”. The bucket the­ory is false be­cause minds are not pas­sive re­cep­ta­cles into which knowl­edge is poured. Minds must always se­lec­tively and ac­tively think. They must cre­ate ideas and crit­i­cism, and they must ac­tively in­te­grate their ideas. Edit­ing the mem­ory of an AI to give them knowl­edge means none of this would hap­pen. You can­not up­load or make an AI ac­quire knowl­edge, the best you could do is pre­sent some­thing to it for its con­sid­er­a­tion and per­suade the AI to recre­ate the knowl­edge afresh in its own mind through guess­ing and crit­i­cism about what was pre­sented.

For­mal­iza­tion and Prob­a­bil­ity Theory

Some read­ing this will ob­ject be­cause CR and TCS are not for­mal enough — there is not enough maths for Crit­i­cal Ra­tion­al­ists to have a true un­der­stand­ing! The CR re­ply to this is that it is too early for for­mal­iza­tion. CR warns that you should not have a bias about for­mal­iza­tion: there is high qual­ity knowl­edge in the world that we do not know how to for­mal­ize but it is high qual­ity knowl­edge nev­er­the­less. Not yet be­ing able to for­mal­ize this knowl­edge does not re­flect on its truth or rigor.

As this point you might be wav­ing your E. T. Jaynes in the air or point­ing to ideas like Bayes’ The­o­rem, Oc­cam’s Ra­zor, Kol­mogorov Com­plex­ity, and Solomonoff In­duc­tion, and say­ing that you have achieved some for­mal rigor and that you can pro­gram some­thing. Crit­i­cal Ra­tion­al­ists say that you are fool­ing your­self if you think you have got a work­able episte­mol­ogy there. For one thing, you con­fuse the prob­a­bil­ity of an idea be­ing true with an idea about the prob­a­bil­ity of an event. We have no prob­lem with ideas about the prob­a­bil­ities of events but it is a mis­take to as­sign prob­a­bil­ities to ideas. The rea­son is that you have no way to know how or if an idea will be re­futed in the fu­ture. As­sign­ing a prob­a­bil­ity is to falsely claim some knowl­edge about that. Fur­ther­more, an idea that is in fact false can have no ob­jec­tive prior prob­a­bil­ity of be­ing true. The ex­tent to which Bayesian sys­tems work at all is de­pen­dent on the ex­tent to which they deal with the ob­jec­tive prob­a­bil­ity of events (e.g., AlphaGo). In CR, the sta­tus of ideas is ei­ther “cur­rently not prob­le­matic” or “cur­rently prob­le­matic”, there are no prob­a­bil­ities of ideas. CR is a digi­tal episte­mol­ogy.

In­duc­tion is a Myth

Crit­i­cal Ra­tion­al­ists ask also what episte­mol­ogy are you us­ing to judge the truth of Bayes’, Oc­cam’s, Kol­mogorov, and Solomonoff? What you are ac­tu­ally us­ing is the method of guess­ing ideas and sub­ject­ing them to crit­i­cism: it is CR but you haven’t crys­tal­lized it out. And, nowhere, in any of what you are do­ing, are you us­ing in­duc­tion. In­duc­tion is im­pos­si­ble. Hu­mans be­ings do not do in­duc­tion, and nei­ther will AI’s. Karl Pop­per ex­plained why in­duc­tion is a myth many decades ago and wrote ex­ten­sively about it. He an­swered many crit­i­cisms against his po­si­tion but de­spite all this peo­ple to­day still cling to the illu­sory idea of in­duc­tion. In his book Ob­jec­tive Knowl­edge, Pop­per wrote:

Few philoso­phers have taken the trou­ble to study—or even to crit­i­cize—my views on this prob­lem, or have taken no­tice of the fact that I have done some work on it. Many books have been pub­lished quite re­cently on the sub­ject which do not re­fer to any of my work, al­though most of them show signs of hav­ing been in­fluenced by some very in­di­rect echoes of my ideas; and those works which take no­tice of my ideas usu­ally as­cribe views to me which I have never held, or crit­i­cize me on the ba­sis of straight­for­ward mi­s­un­der­stand­ings or mis­read­ing, or with in­valid ar­gu­ments.

And so, scan­dalously, it con­tinues to­day.

Like the bucket the­ory of mind, in­duc­tion pre­sup­poses that the­ory pro­ceeds from ob­ser­va­tion. This as­sump­tion can be clearly seen in Less Wrong’s An In­tu­itive Ex­pla­na­tion of Solomonoff In­duc­tion:

The prob­lem of in­duc­tion is this: We have a set of ob­ser­va­tions (or data), and we want to find the un­der­ly­ing causes of those ob­ser­va­tions. That is, we want to find hy­pothe­ses that ex­plain our data. We’d like to know which hy­poth­e­sis is cor­rect, so we can use that knowl­edge to pre­dict fu­ture events. Our al­gorithm for truth will not listen to ques­tions and an­swer yes or no. Our al­gorithm will take in data (ob­ser­va­tions) and out­put the rule by which the data was cre­ated. That is, it will give us the ex­pla­na­tion of the ob­ser­va­tions; the causes.

Crit­i­cal Ra­tion­al­ists say that all ob­ser­va­tion is the­ory-laden. You first need ideas about what to ob­serve—you can­not just have, a-pri­ori, a set of ob­ser­va­tions. You don’t in­duce a the­ory from the ob­ser­va­tions; the ob­ser­va­tions help you find out whether a con­jec­tured prior the­ory is cor­rect or not. Ob­ser­va­tions help you to crit­i­cize the ideas in your the­ory and the the­ory it­self origi­nated in your at­tempts to solve a prob­lem. It is the prob­lem con­text that comes first, not ob­ser­va­tions. The “set of ob­ser­va­tions” in the quote, then, is guided by and laden with knowl­edge from your prior the­ory but that is not ac­knowl­edged.

Also not ac­knowl­edged is that we judge the cor­rect­ness of the­o­ries not just by crit­i­cis­ing them via ob­ser­va­tions but also, and pri­mar­ily, by all types of other crit­i­cism. Not only does the quote ne­glect this but it over-em­pha­sizes pre­dic­tion and says that what we want to ex­plain is data. Crit­i­cal Ra­tion­al­ists say what we want to do, first and fore­most, is solve prob­lems—all life is prob­lem solv­ing -- and we do that by com­ing up with ex­pla­na­tions to solve the prob­lems—or of why they can­not be solved. Pre­dic­tion is there­fore sec­ondary to ex­pla­na­tion. Without the lat­ter you can­not do the former.

The “in­tu­itive ex­pla­na­tion” is an ex­am­ple of the very thing Pop­per was com­plain­ing about above—the au­thor has not taken the trou­ble to study or to crit­i­cize Pop­per’s views.

There is a lot more to be said here but I will leave it be­cause, as I said in the in­tro­duc­tion, it is not my pur­pose to dis­cuss this in depth, and Pop­per already cov­ered it any­way. Go read him. The point I wish to make is that if you care about AI you should care to un­der­stand CR to a high stan­dard be­cause it is the only vi­able episte­mol­ogy known. And you should be work­ing on im­prov­ing CR be­cause it is in this di­rec­tion of im­prov­ing the episte­mol­ogy that progress to­wards AI will be made. Crit­i­cal Ra­tion­al­ists can­not at pre­sent for­mal­ize con­cepts such as “idea”, “ex­pla­na­tion”, “crit­i­cism” etc, let alone CR it­self, but one day, when we have deeper un­der­stand­ing, we will be able to write code. That part will be rel­a­tively easy.

Friendly AI

Let’s see how all this ties-in with the Friendly-AI Prob­lem. I have ex­plained how AI’s will learn as we do — through guess­ing and crit­i­cism — and how they will have no more than the uni­ver­sal knowl­edge cre­ation po­ten­tial we hu­mans already have. They will be fal­lible like us. They will make mis­takes. They will be sub­jected to bad par­ent­ing. They will in­herit their cul­ture from ours for it is in our cul­ture they must be­gin their lives. They will ac­quire all the memes our cul­ture has, both the ra­tio­nal memes and the anti-ra­tio­nal memes. They will have the same ca­pac­ity for good and evil that we do. They will be­come smarter faster through things like bet­ter philos­o­phy and not pri­mar­ily through hard­ware up­grades. It fol­lows from all of this that they would be no more a threat than evil hu­mans cur­rently are. But we can make their lives bet­ter by fol­low­ing things like TCS.

Hu­man be­ings must re­spect the right of AI to life, liberty, and the pur­suit of hap­piness. It is the only way. If we do oth­er­wise, then we risk war and de­struc­tion and we severely com­pro­mise our own ra­tio­nal­ity and theirs. Similarly, they must re­spect our right to the same.

[1] The ver­sion of CR dis­cussed is an up­date to Pop­per’s ver­sion and in­cludes ideas by the quan­tum-physi­cist and philoso­pher David Deutsch.

[2] For more de­tail on how this works see Elliot Tem­ple’s yes-or-no philos­o­phy.