Philosophical differences

[Many peo­ple have been com­plain­ing about the lack of new con­tent on LessWrong lately, so I thought I’d cross-post my lat­est blog post here in dis­cus­sion. Feel free to cri­tique the con­tent as much as you like, but please do keep in mind that I wrote this for my per­sonal blog and not with LW in mind speci­fi­cally, so some parts might not be up to LW stan­dards, whereas oth­ers might be ob­vi­ous to ev­ery­one here. In other words...well, be gen­tle]

---------------------------

You know what’s scarier than hav­ing en­emy sol­diers at your bor­der?

Hav­ing sleeper agents within your bor­ders.

Enemy sol­diers are malev­olent, but they are at least visi­bly malev­olent. You can see what they’re do­ing; you can fight back against them or set up defenses to stop them. Sleeper agents on the other hand are malev­olent and in­visi­ble. They are a threat and you don’t know that they’re a threat. So when a sleeper agent de­cides that it’s time to wake up and smell the gun­pow­der, not only will you be un­able to stop them, but they’ll be in a po­si­tion to do far more dam­age than a lone sol­dier ever could. A sin­gle well-placed sleeper agent can take down an en­tire power grid, or bring a key sup­ply route to a grind­ing halt, or – in the worst case – kill thou­sands with an act of ter­ror­ism, all with­out the slight­est warn­ing.

Okay, so imag­ine that your coun­try is in wartime, and that a small group of vigilant cit­i­zens has un­cov­ered an en­emy sleeper cell in your city. They’ve shown you con­vinc­ing ev­i­dence for the ex­is­tence of the cell, and demon­strated that the cell is ac­tively plan­ning to com­mit some large-scale act of vi­o­lence – per­haps not im­mi­nently, but cer­tainly in the near-to-mid-fu­ture. Worse, the cell seems to have even more ne­far­i­ous plots in the offing, pos­si­bly in­volv­ing nu­clear or biolog­i­cal weapons.

Now imag­ine that when you go to in­ves­ti­gate fur­ther, you find to your sur­prise and frus­tra­tion that no one seems to be par­tic­u­larly con­cerned about any of this. Oh sure, they ac­knowl­edge that in the­ory a sleeper cell could do some dam­age, and that the whole mat­ter is prob­a­bly wor­thy of fur­ther study. But by and large they just hear you out and then shrug and go about their day. And when you, alarmed, point out that this is not just a the­ory – that you have proof that a real sleeper cell is ac­tu­ally op­er­at­ing and mak­ing plans right now – they still re­main re­mark­ably blase. You show them the ev­i­dence, but they ei­ther don’t find it con­vinc­ing, or sim­ply mi­s­un­der­stand it at a very ba­sic level (“A wire­tap? But sleeper agents use cel­l­phones, and cel­l­phones are wire­less!”). Some peo­ple listen but dis­miss the idea out of hand, claiming that sleeper cell at­tacks are “some­thing that only hap­pen in the movies”. Strangest of all, at least to your mind, are the peo­ple who ac­knowl­edge that the ev­i­dence is con­vinc­ing, but say they still aren’t con­cerned be­cause the cell isn’t plan­ning to com­mit any acts of vi­o­lence im­mi­nently, and there­fore won’t be a threat for a while. In the end, all of your at­tempts to raise the alarm are to no avail, and you’re left feel­ing kind of dou­bly scared – scared first be­cause you know the sleeper cell is out there, plot­ting some heinous act, and scared sec­ond be­cause you know you won’t be able to con­vince any­one of that fact be­fore it’s too late to do any­thing about it.

This is roughly how I feel about AI risk.

You see, I think ar­tifi­cial in­tel­li­gence is prob­a­bly the most sig­nifi­cant ex­is­ten­tial threat fac­ing hu­man­ity right now. This, to put it mildly, is some­thing of a fringe po­si­tion in most in­tel­lec­tual cir­cles (al­though that’s be­com­ing less and less true as time goes on), and I’ll grant that it sounds kind of ab­surd. But re­gard­less of whether or not you think I’m right to be scared of AI, you can imag­ine how the fact that AI risk is re­ally hard to ex­plain would make me even more scared about it. Threats like nu­clear war or an as­ter­oid im­pact, while ter­rify­ing, at least have the virtue of be­ing sim­ple to un­der­stand – it’s not ex­actly hard to sell peo­ple on the no­tion that a 2km hunk of rock col­lid­ing with the planet might be a bad thing. As a re­sult peo­ple are aware of these threats and take them (sort of) se­ri­ously, and var­i­ous or­ga­ni­za­tions are (sort of) tak­ing steps to stop them.

AI is differ­ent, though. AI is more like the sleeper agents I de­scribed above – fright­en­ingly in­visi­ble. The idea that AI could be a sig­nifi­cant risk is not re­ally on many peo­ple’s radar at the mo­ment, and worse, it’s an idea that re­sists at­tempts to put it on more peo­ple’s radar, be­cause it’s so bloody con­fus­ing a topic even at the best of times. Our civ­i­liza­tion is effec­tively blind to this threat, and mean­while AI re­search is mak­ing progress all the time. We’re on the Ti­tanic steam­ing through the North At­lantic, un­aware that there’s an ice­berg out there with our name on it – and the cap­tain is or­der­ing full-speed ahead.

(That’s right, not one but two om­i­nous metaphors. Can you see that I’m se­ri­ous?)

But I’m get­ting ahead of my­self. I should prob­a­bly back up a bit and ex­plain where I’m com­ing from.

Ar­tifi­cial in­tel­li­gence has been in the news lately. In par­tic­u­lar, var­i­ous big names like Elon Musk, Bill Gates, and Stephen Hawk­ing have all been sound­ing the alarm in re­gards to AI, de­scribing it as the great­est threat that our species faces in the 21st cen­tury. They (and oth­ers) think it could spell the end of hu­man­ity – Musk said, “If I had to guess what our biggest ex­is­ten­tial threat is, it’s prob­a­bly [AI]”, and Gates said, “I…don’t un­der­stand why some peo­ple are not con­cerned [about AI]”.

Of course, oth­ers are not so con­vinced – ma­chine learn­ing ex­pert An­drew Ng said that “I don’t work on not turn­ing AI evil to­day for the same rea­son I don’t worry about the prob­lem of over­pop­u­la­tion on the planet Mars”.

In this case I hap­pen to agree with the Musks and Gates of the world – I think AI is a tremen­dous threat that we need to fo­cus much of our at­ten­tion on it in the fu­ture. In fact I’ve thought this for sev­eral years, and I’m kind of glad that the big-name in­tel­lec­tu­als are fi­nally catch­ing up.

Why do I think this? Well, that’s a com­pli­cated sub­ject. It’s a topic I could prob­a­bly spend a dozen blog posts on and still not get to the bot­tom of. And maybe I should spend those dozen-or-so blog posts on it at some point – it could be worth it. But for now I’m kind of left with this big in­fer­en­tial gap that I can’t eas­ily cross. It would take a lot of ex­plain­ing to ex­plain my po­si­tion in de­tail. So in­stead of talk­ing about AI risk per se in this post, I thought I’d go off in a more meta-di­rec­tion – as I so of­ten do – and talk about philo­soph­i­cal differ­ences in gen­eral. I figured if I couldn’t make the case for AI be­ing a threat, I could at least make the case for mak­ing the case for AI be­ing a threat.

(If you’re still con­fused, and still won­der­ing what the whole deal is with this AI risk thing, you can read a not-too-ter­rible pop­u­lar in­tro­duc­tion to the sub­ject here, or check out Nick Bostrom’s TED Talk on the topic. Bostrom also has a best­sel­ling book out called Su­per­in­tel­li­gence. The one sen­tence sum­mary of the prob­lem would be: how do we get a su­per­in­tel­li­gent en­tity to want what we want it to want?)

(Trust me, this is much much harder than it sounds)

So: why then am I so meta-con­cerned about AI risk? After all, based on the pre­vi­ous cou­ple para­graphs it seems like the topic ac­tu­ally has pretty de­cent aware­ness: there are pop­u­lar in­ter­net ar­ti­cles and TED talks and celebrity in­tel­lec­tual en­dorse­ments and even best­sel­ling books! And it’s true, there’s no doubt that a ton of progress has been made lately. But we still have a very long way to go. If you had seen the same num­ber of on­line dis­cus­sions about AI that I’ve seen, you might share my de­spair. Such dis­cus­sions are filled with replies that be­tray a fun­da­men­tal mi­s­un­der­stand­ing of the prob­lem at a very ba­sic level. I con­stantly see peo­ple say­ing things like “Won’t the AI just figure out what we want?”, or “If the AI gets dan­ger­ous why can’t we just un­plug it?”, or “The AI can’t have free will like hu­mans, it just fol­lows its pro­gram­ming”, or “lol so you’re scared of Skynet?”, or “Why not just pro­gram it to max­i­mize hap­piness?”.

Hav­ing read a lot about AI, these mi­s­un­der­stand­ings are frus­trat­ing to me. This is not that un­usual, of course – pretty much any com­plex topic is go­ing to have peo­ple mi­s­un­der­stand­ing it, and mi­s­un­der­stand­ings of­ten frus­trate me. But there is some­thing unique about the con­fu­sions that sur­round AI, and that’s the ex­tent to which the con­fu­sions are philo­soph­i­cal in na­ture.

Why philo­soph­i­cal? Well, ar­tifi­cial in­tel­li­gence and philos­o­phy might seem very dis­tinct at first glance, but look closer and you’ll see that they’re con­nected to one an­other at a very deep level. Take al­most any topic of in­ter­est to philoso­phers – free will, con­scious­ness, episte­mol­ogy, de­ci­sion the­ory, metaethics – and you’ll find an AI re­searcher look­ing into the same ques­tions. In fact I would go fur­ther and say that those AI re­searchers are usu­ally do­ing a bet­ter job of ap­proach­ing the ques­tions. Daniel Den­net said that “AI makes philos­o­phy hon­est”, and I think there’s a lot of truth to that idea. You can’t write fuzzy, ill-defined con­cepts into com­puter code. Think­ing in terms of hav­ing to pro­gram some­thing that ac­tu­ally works takes your head out of the philo­soph­i­cal clouds, and puts you in a mind­set of ac­tu­ally an­swer­ing ques­tions.

All of which is well and good. But the prob­lem with look­ing at philos­o­phy through the lens of AI is that it’s a two-way street – it means that when you try to in­tro­duce some­one to the con­cepts of AI and AI risk, they’re go­ing to be haul­ing all of their philo­soph­i­cal bag­gage along with them.

And make no mis­take, there’s a lot of bag­gage. Philos­o­phy is a dis­ci­pline that’s no­to­ri­ous for many things, but prob­a­bly first among them is a lack of con­sen­sus (I wouldn’t be sur­prised if there’s not even a con­sen­sus among philoso­phers about how much con­sen­sus there is among philoso­phers). And the re­sult of this lack of con­sen­sus has been a kind of grab-bag ap­proach to philos­o­phy among the gen­eral pub­lic – peo­ple see that even the ex­perts are di­vided, and think that that means they can just choose what­ever philo­soph­i­cal po­si­tion they want.

Want. That’s the key word here. Peo­ple treat philo­soph­i­cal be­liefs not as things that are ei­ther true or false, but as choices – things to be se­lected based on their per­sonal prefer­ences, like pick­ing out a new set of cur­tains. They say “I pre­fer to be­lieve in a soul”, or “I don’t like the idea that we’re all just atoms mov­ing around”. And why shouldn’t they say things like that? There’s no one to con­tra­dict them, no philoso­pher out there who can say “ac­tu­ally, we set­tled this ques­tion a while ago and here’s the an­swer”, be­cause philos­o­phy doesn’t set­tle things. It’s just not set up to do that. Of course, to be fair peo­ple seem to treat a lot of their non-philo­soph­i­cal be­liefs as choices as well (which frus­trates me to no end) but the prob­lem is par­tic­u­larly pro­nounced in philos­o­phy. And the re­sult is that peo­ple wind up run­ning around with a lot of bad philos­o­phy in their heads.

(Oh, and if that last sen­tence both­ered you, if you’d rather I said some­thing less judg­men­tal like “philos­o­phy I dis­agree with” or “philos­o­phy I don’t per­son­ally hap­pen to hold”, well – the no­tion that there’s no such thing as bad philos­o­phy is ex­actly the kind of bad philos­o­phy I’m talk­ing about)

(he said, only 80% se­ri­ously)

Any­way, I find this whole situ­a­tion pretty con­cern­ing. Be­cause if you had said to me that in or­der to con­vince peo­ple of the sig­nifi­cance of the AI threat, all we had to do was ex­plain to them some sci­ence, I would say: no prob­lem. We can do that. Our so­ciety has got­ten pretty good at ex­plain­ing sci­ence; so far the Great Di­dac­tic Pro­ject has been far more suc­cess­ful than it had any right to be. We may not have got­ten ex­plain­ing sci­ence down to a sci­ence, but we’re at least mak­ing progress. I my­self have been known to ex­plain sci­en­tific con­cepts to peo­ple ev­ery now and again, and fancy my­self not half-bad at it.

Philos­o­phy, though? Differ­ent story. Ex­plain­ing philos­o­phy is re­ally, re­ally hard. It’s hard enough that when I en­counter some­one who has philo­soph­i­cal views I con­sider to be ut­terly wrong or deeply con­fused, I usu­ally don’t even bother try­ing to ex­plain my­self – even if it’s some­one I oth­er­wise have a great deal of re­spect for! In­stead I just dis­en­gage from the con­ver­sa­tion. The times I’ve done oth­er­wise, with a few no­table ex­cep­tions, have only ended in frus­tra­tion – there’s just too much of a gap to cross in one con­ver­sa­tion. And up un­til now that hasn’t re­ally both­ered me. After all, if we’re be­ing hon­est, most philo­soph­i­cal views that peo­ple hold aren’t that im­por­tant in grand scheme of things. Peo­ple don’t re­ally use their philo­soph­i­cal views to in­form their ac­tions – in fact, prob­a­bly the main thing that peo­ple use philos­o­phy for is to sound im­pres­sive at par­ties.

AI risk, though, has im­pressed upon me an ur­gency in re­gards to philos­o­phy that I’ve never felt be­fore. All of a sud­den it’s im­por­tant that ev­ery­one have sen­si­ble no­tions of free will or con­scious­ness; all of a sud­den I can’t let peo­ple get away with be­ing ut­terly con­fused about metaethics.

All of a sud­den, in other words, philos­o­phy mat­ters.

I’m not sure what to do about this. I mean, I guess I could just quit com­plain­ing, buckle down, and do the hard work of get­ting bet­ter at ex­plain­ing philos­o­phy. It’s difficult, sure, but it’s not in­finitely difficult. I could write blogs posts and talk to peo­ple at par­ties, and see what works and what doesn’t, and maybe grad­u­ally start chang­ing a few peo­ple’s minds. But this would be a long and difficult pro­cess, and in the end I’d prob­a­bly only be able to af­fect – what, a few dozen peo­ple? A hun­dred?

And it would be frus­trat­ing. Ar­gu­ments about philos­o­phy are so hard pre­cisely be­cause the ques­tions be­ing de­bated are foun­da­tional. Philo­soph­i­cal be­liefs form the bedrock upon which all other be­liefs are built; they are the premises from which all ar­gu­ments start. As such it’s hard enough to even no­tice that they’re there, let alone be­gin to ques­tion them. And when you do no­tice them, they of­ten seem too self-ev­i­dent to be worth stat­ing.

Take math, for ex­am­ple – do you think the num­ber 5 ex­ists, as a num­ber?

Yes? Okay, how about 700? 3 billion? Do you think it’s ob­vi­ous that num­bers just keep ex­ist­ing, even when they get re­ally big?

Well, guess what – some philoso­phers de­bate this!

It’s ac­tu­ally sur­pris­ingly hard to find an un­con­tro­ver­sial po­si­tion in philos­o­phy. Pretty much ev­ery­thing is de­bated. And of course this usu­ally doesn’t mat­ter – you don’t need philos­o­phy to fill out a tax re­turn or drive the kids to school, af­ter all. But when you hold some foun­da­tional be­liefs that seem self-ev­i­dent, and you’re in a dis­cus­sion with some­one else who holds differ­ent foun­da­tional be­liefs, which they also think are self-ev­i­dent, prob­lems start to arise. Philo­soph­i­cal de­bates usu­ally con­sist of lit­tle more than two peo­ple talk­ing past one an­other, with each won­der­ing how the other could be so stupid as to not un­der­stand the sheer ob­vi­ous­ness of what they’re say­ing. And the an­noy­ing this is, both par­ti­ci­pants are cor­rect – in their own frame­work, their po­si­tions prob­a­bly are ob­vi­ous. The prob­lem is, we don’t all share the same frame­work, and in a set­ting like that frus­tra­tion is the de­fault, not the ex­cep­tion.

This is not to say that all efforts to dis­cuss philos­o­phy are doomed, of course. Peo­ple do some­times have pro­duc­tive philo­soph­i­cal dis­cus­sions, and the odd per­son even man­ages to change their mind, oc­ca­sion­ally. But to do this takes a lot of effort. And when I say a lot of effort, I mean a lot of effort. To make progress philo­soph­i­cally you have to be will­ing to adopt a kind of ex­treme epistemic hu­mil­ity, where your in­tu­itions count for very lit­tle. In fact, far from treat­ing your in­tu­itions as un­ques­tion­able givens, as most peo­ple do, you need to be treat­ing them as things to be care­fully ex­am­ined and scru­ti­nized with acute skep­ti­cism and even wari­ness. Your re­ac­tion to some­one hav­ing a differ­ing in­tu­ition from you should not be “I’m right and they’re wrong”, but rather “Huh, where does my in­tu­ition come from? Is it just a fea­ture­less feel­ing or can I break it down fur­ther and ex­plain it to other peo­ple? Does it ac­cord with my other in­tu­itions? Why does per­son X have a differ­ent in­tu­ition, any­way?” And most im­por­tantly, you should be ask­ing “Do I en­dorse or re­ject this in­tu­ition?”. In fact, you could prob­a­bly say that the whole his­tory of philos­o­phy has been lit­tle more than an at­tempt by peo­ple to at­tain re­flec­tive equil­ibrium among their differ­ent in­tu­itions – which of course can’t hap­pen with­out the will­ing­ness to dis­card cer­tain in­tu­itions along the way when they con­flict with oth­ers.

I guess what I’m try­ing to say is: when you’re dis­cussing philos­o­phy with some­one and you have a dis­agree­ment, your fore­most goal should be to try to find out ex­actly where your in­tu­itions differ. And once you iden­tify that, from there the im­me­di­ate next step should be to zoom in on your in­tu­itions – to figure out the source and con­tent of the in­tu­ition as much as pos­si­ble. In­tu­itions aren’t blank struc­ture­less feel­ings, as much as it might seem like they are. With enough in­tro­spec­tion in­tu­itions can be ex­pli­cated and elu­ci­dated upon, and de­scribed in some de­tail. They can even be passed on to other peo­ple, as­sum­ing at least some kind of ba­sic com­mon episte­molog­i­cal frame­work, which I do think all hu­mans share (yes, even ob­jec­tive-re­al­ity-deny­ing post­mod­ernists).

Any­way, this whole con­cept of zoom­ing in on in­tu­itions seems like an im­por­tant one to me, and one that hasn’t been em­pha­sized enough in the in­tel­lec­tual cir­cles I travel in. When some­one doesn’t agree with some ba­sic foun­da­tional be­lief that you have, you can’t just throw up your hands in de­spair – you have to per­se­vere and figure out why they don’t agree. And this takes effort, which most peo­ple aren’t will­ing to ex­pend when they already see their de­bate op­po­nent as some­one who’s be­ing willfully stupid any­way. But – need­less to say – no one thinks of their po­si­tions as be­ing a re­sult of willful stu­pidity. Pretty much ev­ery­one holds be­liefs that seem ob­vi­ous within the frame­work of their own wor­ld­view. So if you want to change some­one’s mind with re­spect to some philo­soph­i­cal ques­tion or an­other, you’re go­ing to have to dig deep and en­gage with their wor­ld­view. And this is a difficult thing to do.

Hence, the philo­soph­i­cal quag­mire that we find our so­ciety to be in.

It strikes me that im­prov­ing our abil­ity to ex­plain and dis­cuss philos­o­phy amongst one an­other should be of paramount im­por­tance to most in­tel­lec­tu­ally se­ri­ous peo­ple. This ap­plies to AI risk, of course, but also to many ev­ery­day top­ics that we all dis­cuss: fem­i­nism, geopoli­tics, en­vi­ron­men­tal­ism, what have you – pretty much ev­ery­thing we talk about grounds out to philos­o­phy even­tu­ally, if you go deep enough or meta enough. And to the ex­tent that we can’t dis­cuss philos­o­phy pro­duc­tively right now, we can’t make progress on many of these im­por­tant is­sues.

I think philoso­phers should – to some ex­tent – be ashamed of the state of their field right now. When you com­pare philos­o­phy to sci­ence it’s clear that sci­ence has made great strides in ex­plain­ing the con­tents of its find­ings to the gen­eral pub­lic, whereas philos­o­phy has not. Philoso­phers seem to treat their field as be­ing al­most in­con­se­quen­tial, as if what­ever they con­clude at some level won’t mat­ter. But this clearly isn’t true – we need vastly im­proved dis­cus­sion norms when it comes to philos­o­phy, and we need far greater effort on the part of philoso­phers when it comes to ex­plain­ing philos­o­phy, and we need these things right now. Re­gard­less of what you think about AI, the 21st cen­tury will clearly be fraught with difficult philo­soph­i­cal prob­lems – from ge­netic en­g­ineer­ing to the eth­i­cal treat­ment of an­i­mals to the prob­lem of what to do with global poverty, it’s ob­vi­ous that we will soon need philo­soph­i­cal an­swers, not just philo­soph­i­cal ques­tions. Im­prove­ments in tech­nol­ogy mean im­prove­ments in ca­pa­bil­ity, and that means that things which were once merely thought ex­per­i­ments will be lifted into the realm of real ex­per­i­ments.

I think the prob­lem that hu­man­ity faces in the 21st cen­tury is an un­prece­dented one. We’re faced with the task of ac­tu­ally solv­ing philos­o­phy, not just do­ing philos­o­phy. And if I’m right about AI, then we have ex­actly one try to get it right. If we don’t, well..

Well, then the fate of hu­man­ity may liter­ally hang in the bal­ance.