Leaving LessWrong for a more rational life

You are un­likely to see me post­ing here again, af­ter to­day. There is a say­ing here that poli­tics is the mind-kil­ler. My hereti­cal re­al­iza­tion lately is that philos­o­phy, as gen­er­ally prac­ticed, can also be mind-kil­ling.

As many of you know I am, or was run­ning a twice-monthly Ra­tion­al­ity: AI to Zom­bies read­ing group. One of the bits I de­sired to in­clude in each read­ing group post was a col­lec­tion of con­trast­ing views. To re­search such views I’ve found my­self listen­ing dur­ing my com­mute to talks given by other thinkers in the field, e.g. Nick Bostrom, An­ders Sand­berg, and Ray Kurzweil, and peo­ple I feel are do­ing “ide­olog­i­cally al­igned” work, like Aubrey de Grey, Chris­tine Peter­son, and Robert Fre­itas. Some of these were talks I had seen be­fore, or gen­er­ally views I had been ex­posed to in the past. But look­ing through the lens of learn­ing and ap­ply­ing ra­tio­nal­ity, I came to a sur­pris­ing (to me) con­clu­sion: it was philo­soph­i­cal thinkers that demon­strated the largest and most costly mis­takes. On the other hand, de Grey and oth­ers who are pri­mar­ily work­ing on the sci­en­tific and/​or en­g­ineer­ing challenges of sin­gu­lar­ity and tran­shu­man­ist tech­nolo­gies were far less likely to sub­ject them­selves to epistem­atic mis­takes of sig­nifi­cant con­se­quences.

Philos­o­phy as the anti-sci­ence...

What sort of mis­takes? Most of­ten rea­son­ing by anal­ogy. To cite a spe­cific ex­am­ple, one of the core un­der­ly­ing as­sump­tion of sin­gu­lar­ity in­ter­pre­ta­tion of su­per-in­tel­li­gence is that just as a chim­panzee would be un­able to pre­dict what a hu­man in­tel­li­gence would do or how we would make de­ci­sions (aside: how would we know? Were any chimps con­sulted?), we would be equally in­ept in the face of a su­per-in­tel­li­gence. This ar­gu­ment is, how­ever, non­sense. The hu­man ca­pac­ity for ab­stract rea­son­ing over math­e­mat­i­cal mod­els is in prin­ci­ple a fully gen­eral in­tel­li­gent be­havi­our, as the sci­en­tific rev­olu­tion has shown: there is no as­pect of the nat­u­ral world which has re­mained be­yond the reach of hu­man un­der­stand­ing, once a suffi­cient amount of ev­i­dence is available. The wave-par­ti­cle du­al­ity of quan­tum physics, or the 11-di­men­sional space of string the­ory may defy hu­man in­tu­ition, i.e. our built-in in­tel­li­gence. But we have proven our­selves perfectly ca­pa­ble of un­der­stand­ing the log­i­cal im­pli­ca­tions of mod­els which em­ploy them. We may not be able to build in­tu­ition for how a su­per-in­tel­li­gence thinks. Maybe—that’s not proven ei­ther. But even if that is so, we will be able to rea­son about its in­tel­li­gent be­havi­our in ad­vance, just like string the­o­rists are able to rea­son about 11-di­men­sional space-time with­out us­ing their evolu­tion­ar­ily de­rived in­tu­itions at all.

This post is not about the sin­gu­lar­ity na­ture of su­per-in­tel­li­gence—that was merely my choice of an illus­tra­tive ex­am­ple of a cat­e­gory of mis­takes that are too of­ten made by those with a philo­soph­i­cal back­ground rather than the em­piri­cal sci­ences: the rea­son­ing by anal­ogy in­stead of the build­ing and an­a­lyz­ing of pre­dic­tive mod­els. The fun­da­men­tal mis­take here is that rea­son­ing by anal­ogy is not in it­self a suffi­cient ex­pla­na­tion for a nat­u­ral phe­nomenon, be­cause it says noth­ing about the con­text sen­si­tivity or in­sen­si­tivity of the origi­nal ex­am­ple and un­der what con­di­tions it may or may not hold true in a differ­ent situ­a­tion.

A suc­cess­ful physi­cist or biol­o­gist or com­puter en­g­ineer would have ap­proached the prob­lem differ­ently. A core part of be­ing suc­cess­ful in these ar­eas is know­ing when it is that you have in­suffi­cient in­for­ma­tion to draw con­clu­sions. If you don’t know what you don’t know, then you can’t know when you might be wrong. To be an effec­tive ra­tio­nal­ist, it is of­ten not im­por­tant to an­swer “what is the calcu­lated prob­a­bil­ity of that out­come?” The bet­ter first ques­tion is “what is the un­cer­tainty in my calcu­lated prob­a­bil­ity of that out­come?” If the un­cer­tainty is too high, then the data sup­ports no con­clu­sions. And the way you re­duce un­cer­tainty is that you build mod­els for the do­main in ques­tion and em­piri­cally test them.

The lens that sees its own flaws...

Com­ing back to LessWrong and the se­quences. In the pref­ace to Ra­tion­al­ity, Eliezer Yud­kowsky says his biggest re­gret is that he did not make the ma­te­rial in the se­quences more prac­ti­cal. The prob­lem is in fact deeper than that. The art of ra­tio­nal­ity is the art of truth seek­ing, and em­piri­cism is part and par­cel es­sen­tial to truth seek­ing. There’s lip ser­vice done to em­piri­cism through­out, but in all the “ap­plied” se­quences re­lat­ing to quan­tum physics and ar­tifi­cial in­tel­li­gence it ap­pears to be for­got­ten. We get in­stead defini­tive con­clu­sions drawn from thought ex­per­i­ments only. It is per­haps not sur­pris­ing that these se­quences seem the most con­tro­ver­sial.

I have for a long time been con­cerned that those se­quences in par­tic­u­lar pro­mote some un­grounded con­clu­sions. I had thought that while an­noy­ing this was per­haps a one-off mis­take that was fix­able. Re­cently I have re­al­ized that the un­der­ly­ing cause runs much deeper: what is taught by the se­quences is a form of flawed truth-seek­ing (thought ex­per­i­ments fa­vored over real world ex­per­i­ments) which in­evitably re­sults in er­rors, and the er­rors I take is­sue with in the se­quences are merely ex­am­ples of this phe­nomenon.

And these er­rors have con­se­quences. Every sin­gle day, 100,000 peo­ple die of pre­ventable causes, and ev­ery day we con­tinue to risk ex­tinc­tion of the hu­man race at un­ac­cept­ably high odds. There is work that could be done now to alle­vi­ate both of these is­sues. But within the LessWrong com­mu­nity there is ac­tu­ally out­right hos­tility to work that has a rea­son­able chance of alle­vi­at­ing suffer­ing (e.g. ar­tifi­cial gen­eral in­tel­li­gence ap­plied to molec­u­lar man­u­fac­tur­ing and life-sci­ence re­search) due to con­cerns ar­rived at by flawed rea­son­ing.

I now re­gard the se­quences as a memetic haz­ard, one which may at the end of the day be do­ing more harm than good. One should work to de­velop one’s own ra­tio­nal­ity, but I now fear that the ap­proach taken by the LessWrong com­mu­nity as a con­tinu­a­tion of the se­quences may re­sult in more harm than good. The anti-hu­man­i­tar­ian be­hav­iors I ob­serve in this com­mu­nity are not the re­sult of ini­tial con­di­tions but the pro­cess it­self.

What next?

How do we fix this? I don’t know. On a per­sonal level, I am no longer sure en­gage­ment with such a com­mu­nity is a net benefit. I ex­pect this to be my last post to LessWrong. It may hap­pen that I check back in from time to time, but for the most part I in­tend to try not to. I wish you all the best.

A note about effec­tive al­tru­ism…

One shin­ing light of good­ness in this com­mu­nity is the fo­cus on effec­tive al­tru­ism—do­ing the most good to the most peo­ple as mea­sured by some ob­jec­tive means. This is a no­ble goal, and the cor­rect goal for a ra­tio­nal­ist who wants to con­tribute to char­ity. Un­for­tu­nately it too has been poi­soned by in­cor­rect modes of thought.

Ex­is­ten­tial risk re­duc­tion, the ar­gu­ment goes, trumps all forms of char­i­ta­ble work be­cause re­duc­ing the chance of ex­tinc­tion by even a small amount has far more ex­pected util­ity than would ac­com­plish­ing all other char­i­ta­ble works com­bined. The prob­lem lies in the like­li­hood of ex­tinc­tion, and the ac­tions se­lected in re­duc­ing ex­is­ten­tial risk. There is so much un­cer­tainty re­gard­ing what we know, and so much un­cer­tainty re­gard­ing what we don’t know that it is im­pos­si­ble to de­ter­mine with any ac­cu­racy the ex­pected risk of, say, un­friendly ar­tifi­cial in­tel­li­gence cre­at­ing per­pet­ual sub­op­ti­mal out­comes, or what effect char­i­ta­ble work in the area (e.g. MIRI) is have to re­duce that risk, if any.

This is best ex­plored by an ex­am­ple of ex­is­ten­tial risk done right. As­teroid and cometary im­pacts is per­haps the cat­e­gory of ex­ter­nal (not-hu­man-caused) ex­is­ten­tial risk which we know the most about, and have done the most to miti­gate. When it was rec­og­nized that im­pactors were a risk to be taken se­ri­ously, we rec­og­nized what we did not know about the phe­nomenon: what were the or­bits and masses of Earth-cross­ing as­ter­oids? We built telescopes to find out. What is the ma­te­rial com­po­si­tion of these ob­jects? We built space probes and col­lected me­te­orite sam­ples to find out. How dam­ag­ing an im­pact would there be for var­i­ous ma­te­rial prop­er­ties, speeds, and in­ci­dence an­gles? We built high-speed pro­jec­tile test ranges to find out. What could be done to change the course of an as­ter­oid found to be on col­li­sion course? We have ex­e­cuted at least one im­pact probe and will mon­i­tor the effect that had on the comet’s or­bit, and have on the draw­ing board probes that will use grav­i­ta­tional mechanisms to move their tar­get. In short, we iden­ti­fied what it is that we don’t know and sought to re­solve those un­cer­tain­ties.

How then might one ap­proach an ex­is­ten­tial risk like un­friendly ar­tifi­cial in­tel­li­gence? By iden­ti­fy­ing what it is we don’t know about the phe­nomenon, and seek­ing to ex­per­i­men­tally re­solve that un­cer­tainty. What rele­vant facts do we not know about (un­friendly) ar­tifi­cial in­tel­li­gence? Well, much of our un­cer­tainty about the ac­tions of an un­friendly AI could be re­solved if we were to know more about how such agents con­struct their thought mod­els, and re­lat­edly what lan­guage were used to con­struct their goal sys­tems. We could also stand to benefit from know­ing more prac­ti­cal in­for­ma­tion (ex­per­i­men­tal data) about in what ways AI box­ing works and in what ways it does not, and how much that is de­pen­dent on the struc­ture of the AI it­self. Thank­fully there is an in­sti­tu­tion that is do­ing that kind of work: the Fu­ture of Life in­sti­tute (not MIRI).

Where should I send my char­i­ta­ble dona­tions?

Aubrey de Grey’s SENS Re­search Foun­da­tion.

100% of my char­i­ta­ble dona­tions are go­ing to SENS. Why they do not get more play in the effec­tive al­tru­ism com­mu­nity is be­yond me.

If you feel you want to spread your money around, here are some non-prof­its which have I have vet­ted for do­ing re­li­able, ev­i­dence-based work on sin­gu­lar­ity tech­nolo­gies and ex­is­ten­tial risk:

  • Robert Fre­itas and Ralph Merkle’s In­sti­tute for Molec­u­lar Man­u­fac­tur­ing does re­search on molec­u­lar nan­otech­nol­ogy. They are the only group that work on the long-term Drexlar­ian vi­sion of molec­u­lar ma­chines, and pub­lish their re­search on­line.

  • Fu­ture of Life In­sti­tute is the only ex­is­ten­tial-risk AI or­ga­ni­za­tion which is ac­tu­ally do­ing mean­ingful ev­i­dence-based re­search into ar­tifi­cial in­tel­li­gence.

  • B612 Foun­da­tion is a non-profit seek­ing to launch a space­craft with the ca­pa­bil­ity to de­tect, to the ex­tent pos­si­ble, ALL Earth-cross­ing as­ter­oids.

I wish I could recom­mend a skep­ti­cism, em­piri­cism, and ra­tio­nal­ity pro­mot­ing in­sti­tute. Un­for­tu­nately I am not aware of an or­ga­ni­za­tion which does not suffer from the flaws I iden­ti­fied above.

Ad­den­dum re­gard­ing un­finished business

I will no longer be run­ning the Ra­tion­al­ity: From AI to Zom­bies read­ing group as I am no longer in good con­science able or will­ing to host it, or par­ti­ci­pate in this site, even from my typ­i­cally con­trar­ian point of view. Nev­er­the­less, I am enough of a liber­tar­ian that I feel it is not my role to put up road­blocks to oth­ers who wish to delve into the ma­te­rial as it is pre­sented. So if some­one wants to take over the role of or­ga­niz­ing these read­ing groups, I would be happy to hand over the reigns to that per­son. If you think that per­son should be you, please leave a re­ply in an­other thread, not here.

EDIT: Ob­vi­ously I’ll stick around long enough to an­swer ques­tions be­low :)