On saving the world

This is the fi­nal post in my pro­duc­tivity se­quence.

The first post de­scribed what I achieved. The next three posts de­scribe how. This post de­scribes why, ex­plain­ing the sources of my pas­sion and the cir­cum­stances that con­vinced a young Nate to try and save the world. Within, you will find no sug­ges­tions, no tech­niques to em­u­late, no new ideas to pon­der. This is a ra­tio­nal­ist com­ing-of-age story. With luck, you may find it in­spiring. Re­gard­less, I hope you can learn from my mis­takes.

Never fear, I’ll be back to busi­ness soon — there’s lots of study­ing to do. But be­fore then, there’s a story to tell, a memo­rial to what I left be­hind.

I was raised Catholic. On my eighth birth­day, hav­ing re­ceived my first com­mu­nion about a year prior, I ca­su­ally asked my priest how to reaf­firm my faith and do some­thing for the Lord. The mem­ory is fuzzy, but I think I donated a chunk of al­lowance money and made a pub­lic con­fes­sion at the fol­low­ing mass.

A bunch of the grownups made a big deal out of it, as grownups are like to do. “Faith of a child”, and all that. This con­fused me, es­pe­cially when I re­al­ized that what I had done was rare. I wasn’t try­ing to get pats on the head, I was ap­peal­ing to the Lord of the Heav­ens and the Earth. Were we all on the same page, here? This was the cre­ator. He was in­finitely vir­tu­ous, and he had told us what to do.

And yet, ev­ery­one was con­tent to re­cite hymns once a week and donate for the re­con­struc­tion of the church. What about the rest of the world, the sick, the dy­ing? Where were the pros­ely­tiz­ers, the mis­sion­ary op­por­tu­ni­ties? Why was ev­ery­one just sit­ting around?

On that day, I be­came ac­quainted with civ­i­liza­tional in­ad­e­quacy. I re­al­ized you could hand a room full of peo­ple the literal word of God, and they’d still strug­gle to pay at­ten­tion for an hour ev­ery week­end.

This didn’t shake my faith, mind you. It didn’t even oc­cur to me that the grownups might not ac­tu­ally be­lieve their tales. No, what I learned that day was that there are a lot of peo­ple who hold be­liefs they aren’t will­ing to act upon.

Even­tu­ally, my faith faded. The dis­trust re­mained.

Gain­ing Confidence

I grew up in a small village, pop­u­la­tion ~1200. My early ed­u­ca­tion took place in a one-room schoolhouse. The lo­cal towns even­tu­ally rol­led all their school dis­tricts into one, but even then, my grad­u­at­ing class barely broke 50 peo­ple. It wasn’t difficult to ex­cel.

Ages twelve and thir­teen were rough — that was right af­ter they merged school dis­tricts, and those were the years I was first put a few grades ahead in math classes. I was awk­ward and un­der­con­fi­dent. I felt es­tranged and lonely, and it was easy to get shoe­horned into the “smart kid” stereo­type by all the new stu­dents.

Even­tu­ally, though, I de­cided that the stereo­type was bo­gus. Any­one in­tel­li­gent should be able to es­cape such pi­geon­hol­ing. In fact, I con­cluded that any­one with real smarts should be able to find their way out of any mess. I ob­served the con­fi­dence pos­sessed by my peers, even those who seemed to have no rea­son for con­fi­dence. I no­ticed the ease with which they en­gaged in so­cial in­ter­ac­tions. I de­cided I could em­u­late these.

I faked con­fi­dence, and it soon be­came real. I found that my so­cial limi­ta­tions had been largely psy­cholog­i­cal, and that the ma­jor­ity of my class­mates were more than will­ing to be friends. I learned how to get good grades with­out alienat­ing my peers. It helped that I tended to buck au­thor­ity (I was no “teacher’s pet”) and that I en­joyed teach­ing oth­ers. I had a knack for pin­point­ing mi­s­un­der­stand­ings and was of­ten able to teach bet­ter than the teach­ers could — as a peer, I could com­mu­ni­cate on a differ­ent level.

I started do­ing very well for my­self. I got ex­cel­lent grades with min­i­mal effort. I over­came my so­cial anx­ieties. I had a few close friends and was on good terms with most ev­ery­one else. I par­ti­ci­pated in a num­ber of ex­tra cir­cu­lars where I held high sta­tus. As you may imag­ine, I grew quite ar­ro­gant.

In ret­ro­spect, my ac­com­plish­ments were hardly im­pres­sive. At the time, though, it felt like ev­ery­one else wasn’t even try­ing. It be­came ap­par­ent that if I wanted some­thing done right, I’d have to do it my­self.

Shat­tered Illusions

Up un­til the age of four­teen I had this grow­ing in­tu­ition that you can’t trust oth­ers to ac­tu­ally get things done. This be­lief didn’t be­come ex­plicit un­til the end of ninth grade, when I learned how the gov­ern­ment of the United States of Amer­ica ac­tu­ally works.

Allow me to provide a few pieces of con­text.

For one thing, I was learn­ing to pro­gram com­put­ers at the time. I had been pro­gram­ming for maybe a year and a half, and I was start­ing to form con­cepts of el­e­gance and min­i­mal­ism. I had a be­lief that the best de­sign is a small de­sign, a de­sign forced by na­ture at ev­ery step along the way, a de­sign that re­quires no ar­bi­trary choices.

For an­other thing, my re­li­gion had died not with a bang, but with a whim­per. I’d com­part­men­tal­ized it, and it had slowly with­ered away. I didn’t Believe any more, but I didn’t mind that oth­ers did. It was a happy fan­tasy, a so­cial tool. Just as chil­dren are al­lowed to be­lieve in Santa Claus, grownups were al­lowed to be­lieve in Gods.

The gov­ern­ment, though, was a differ­ent mat­ter all to­gether. I as­sumed that a lot of very smart peo­ple had put a lot of effort into its de­sign — that’s what the “Found­ing Fathers” meme im­plied, any­way. But maybe it wasn’t even that. Maybe I just pos­sessed an un­spo­ken, un­challenged be­lief that the grownups knew what they were do­ing, at least at the very high­est lev­els. This was the very fabric of so­ciety it­self: surely it was metic­u­lously cal­ibrated to max­i­mize hu­man virtue, to pro­tect us from cir­cum­stance and evil.

When I was fi­nally told how the US gov­ern­ment worked, I couldn’t be­lieve my ears. It was a mess. An ar­bi­trary, clunky mon­stros­ity full of loop­holes a child could abuse. I could think of a dozen im­prove­ments off the top of my head.

To give you an idea of how my teenaged mind worked, it was im­me­di­ately clear to me that any first-or­der “im­prove­ments” sug­gested by naïve ninth-graders would have un­in­tended nega­tive con­se­quences. There­fore, im­prove­ment num­ber one in­volved re­design­ing the sys­tem to make it easy to test many differ­ent im­prove­ments in par­allel, adding ma­chin­ery to adopt the im­prove­ments that were ac­tu­ally shown to work.

Yet even these sim­ple ideas were ab­sent in the ac­tual sys­tem. Cor­rup­tion and in­effi­ciency ran ram­pant. Worse, my peers didn’t seem par­tic­u­larly per­turbed: they took the sys­tem as a given, and merely mem­o­rized the ma­chin­ery for long enough to pass a test. Even the grownups were ap­a­thetic: they dick­ered over who should have power within the sys­tem, never sug­gest­ing we should al­ter the sys­tem it­self.

My child­hood illu­sions fell to pieces. I re­al­ized that noth­ing was metic­u­lously man­aged, that the smartest peo­ple weren’t in con­trol, mak­ing sure that ev­ery­thing was op­ti­mal. All the world prob­lems, the sick­nesses and the in­jus­tices and the death: these weren’t nec­es­sary evils, they were a product of ne­glect. The most im­por­tant sys­tem of all was poorly co­or­di­nated, bloated, and out­dated — and no­body seemed to care.

De­cid­ing to Save the World

This is the con­text in which I de­cided to save the world. I wasn’t as young and stupid as you might think — I didn’t be­lieve I was go­ing to save the world. I just de­cided to. The world is big, and I was small. I knew that, in all like­li­hood, I’d strug­gle in­effec­tu­ally for decades and achieve only a bit­ter, cyn­i­cal adult­hood.

But the vast ma­jor­ity of my peers hadn’t made it as far as I had. Even though a few were sym­pa­thetic, there was sim­ply no way we could change things. It was out­side of our con­trol.

The adults were worse. They smiled, they nod­ded, they com­mended my crit­i­cal think­ing skills. Then they went back to what they were do­ing. A few of them took the time to in­form me that it’s great to want to change the world and all, but even­tu­ally I’d re­al­ize that the best way to do that was to set­tle down and be a teacher, or run a church, or just be kind to oth­ers.

I wasn’t sur­prised. I already knew it was rare for peo­ple to ac­tu­ally try and fix things.

I had youth­ful ideal­ism, I had big am­bi­tions, but I knew full well that I didn’t ac­tu­ally have a chance. I knew that I wouldn’t be able to sin­gle-hand­edly re­design the so­cial con­tract, but I also knew that if ev­ery­one who made it as far as I did gave up just be­cause chang­ing the world is im­pos­si­ble, then the world would never change.

If ev­ery­body was cowed by the sim­ple fact that they can’t suc­ceed, then that one-in-a-mil­lion per­son who can suc­ceed would never take their shot.

So I was sure as hell go­ing to take mine.

Broad­en­ing Scope

Mere im­pos­si­bil­ity was never a hur­dle: The Phan­tom Tol­l­booth saw to that at a young age. When grownups say you can’t do some­thing, what they mean is that they can’t do it. I spent time de­vis­ing strate­gies to get lev­er­age and push gov­ern­ments out of their stag­nant state and into some­thing ca­pa­ble of growth.

In 2005, a teacher to whom I’d ranted in­tro­duced me to an­other im­por­tant book: Ish­mael. It wasn’t the ideas that stuck with me — I dis­agreed with a few at the time, and I now dis­agree with most. No, what this book gave me was scope. This au­thor, too, wished to save the world, and the breadth of his ideas ex­ceeded my own. This book gave me no an­swers, but it gave me bet­ter questions

Why merely hone the gov­ern­ment, in­stead of re­design­ing it al­to­gether?

More im­por­tantly, What sort of world are you aiming for?

“So you want to be an ideal­ist?“, the book asked. “Very well, but what is your ideal?

I re­fo­cused, look­ing to fully define the ideals I strove for in a hu­man so­cial sys­tem. I knew I wouldn’t be able to in­sti­tute any solu­tion di­rectly, but I also knew that push­ing gov­ern­ments would be much eas­ier if I had some­thing to push them to­wards.

After all, the Com­mu­nist Man­i­festo changed the world, once.

This be­came my new goal: dis­till an ideal so­cial struc­ture for hu­mans. The prob­lem was in­sur­mountable, of course, but this was hardly a de­ter­rence. I was bright enough to un­der­stand tru­isms like “no one sys­tem will work for ev­ery­body” and “you’re not perfect enough to get this right”, but these were no trou­ble. I didn’t need to di­rectly spec­ify an ideal so­cial struc­ture: a meta-struc­ture, an im­perfect sys­tem that ratch­ets to­wards perfec­tion, a sys­tem that is op­ti­mal in the limit, would be fine by me.

From my van­tage point, old ideas like com­mu­nism and democ­racy soon seemed laugh­able. In­ter­est­ing ideas in their time, per­haps, but ob­vi­ously doomed to failure. It’s easy to build a utopia when you imag­ine that peo­ple will set aside their greed and over­come their ap­a­thy. But those aren’t sys­tems for peo­ple: Peo­ple are greedy, and peo­ple are ap­a­thetic. I wanted some­thing that worked — nay, thrived — when pop­u­lated by ac­tual hu­mans, with all their flaws.

I de­voted time and effort to re­search and study. This was dan­ger­ous, as there was no feed­back loop. As soon as I stepped be­yond the achieve­ments of his­tory, there was no way to ac­tu­ally test any­thing I came up with. Many times, I set­tled on one idea for a few months, mul­ling it over, declar­ing it perfect. Time and again, I later found a fatal flaw, a piece of faulty rea­son­ing, and the whole thing came tum­bling down. After many cy­cles, I no­ticed that the flaws were usu­ally visi­ble in ad­vance. I be­came cog­nizant of the fact that I’d been gloss­ing over them, ig­nor­ing them, ex­plain­ing them away.

I learned not to trust my own de­crees of perfec­tion. I started mon­i­tor­ing my thought pro­cesses very closely. I learned to no­tice the lit­tle ghosts of doubt, to ad­dress them ear­lier and more thor­oughly. (I be­came a staunch athe­ist, un­sur­pris­ingly.) This was, per­haps, the be­gin­ning of my ra­tio­nal­ist train­ing. Un­for­tu­nately, it was all self-di­rected. Some­how, it never oc­curred to me to read liter­a­ture on how to think bet­ter. I didn’t have much trust in psy­cholog­i­cal liter­a­ture, any­way, and I was ar­ro­gant.

Com­mu­ni­ca­tion Failures

It was dur­ing this pe­riod that I ex­plic­itly de­cided not to pur­sue math. I rea­soned that in or­der to ac­tu­ally save the world, I’d need to fo­cus on charisma, poli­ti­cal con­nec­tions, and a solid un­der­stand­ing of the ma­chin­ery un­der­ly­ing the world’s ma­jor gov­ern­ments. Upon grad­u­at­ing high school, I de­cided to go to a col­lege in Wash­ing­ton D.C. and study poli­ti­cal sci­ence. I dou­ble ma­jored in Com­puter Science as a fal­lback plan, a way to ac­tu­ally make money as needed (and be­cause I loved it).

I went into my Poly Sci de­gree ex­pect­ing to learn about the me­chan­ics of so­ciety. Amus­ingly enough, I didn’t know that “Eco­nomics” was a field. We didn’t have any econ classes in my tiny high school, and no­body had seen fit to tell me about it. I ex­pected “Poli­ti­cal Science” to teach me the work­ings of na­tions in­clud­ing the world econ­omy, but quickly re­al­ized that it’s about the ac­tual poli­ti­ci­ans, the so­cial pea­cock­ing, the façades. For­tu­nately, a re­quired In­tro to Econ class soon reme­died the situ­a­tion, and I quickly changed my ma­jor to Eco­nomics.

My ideas ex­pe­rienced sig­nifi­cant re­fine­ment as I re­ceived for­mal train­ing. Un­for­tu­nately, no­body would listen to them.

It’s not that they were dis­missed as childish ideal­ism: I had grad­u­ated to larger prob­lems. I’d been think­ing long and hard about the prob­lem for a few years, and I’d had some in­ter­est­ing in­sights. But when I tried to ex­plain them to peo­ple, al­most ev­ery­one had im­me­di­ate ad­verse re­ac­tions.

I an­ti­ci­pated crit­i­cism, and rel­ished the prospect. My ideas were in des­per­ate need of an out­side challenger. But the re­ac­tions of oth­ers were far worse than I an­ti­ci­pated.

No­body found flaws in my logic. No­body challenged my bold claims. In­stead, they sim­ply failed to un­der­stand. They got stuck three or four points be­fore the in­ter­est­ing points, and could go no fur­ther. I learned that most peo­ple don’t un­der­stand ba­sic eco­nomics or game the­ory. Many oth­ers were en­trenched in blue­greens­man­ship and re­flex­ively treated my sug­ges­tions as at­tacks. Aspiring poli­ti­ci­ans balked at the claim that Democ­racy, while per­haps an im­por­tant step in our cul­tural evolu­tion, can’t pos­si­bly be the end of the line. Still oth­ers in­sisted that it’s use­less to dis­cuss ideals, be­cause they can never be achieved.

In short, I found my­self on the far side of a wide in­fer­en­tial gap.

I learned that many peo­ple, af­ter fal­ling into the gap, were in­ca­pable of climb­ing out, no mat­ter how slowly I walked them through the in­ter­ven­ing steps. They had already passed judge­ment on the con­clu­sion, and re­jected my at­tempts to root out their mis­con­cep­tions, be­com­ing im­pa­tient be­fore ac­tu­ally listen­ing. I grew very cau­tious with who I shared my ideas with, wor­ry­ing that ex­pos­ing them too quickly or in the wrong fash­ion would be a per­ma­nent set­back.

I had a small few friends who knew enough eco­nomics and other sub­jects to fol­low along and who wouldn’t dis­card un­couth ideas out­right. I be­gan to value these peo­ple highly, as they were among the few who could ac­tu­ally put pres­sure on me, ex­pose flaws in my rea­son­ing, and help me come up with solu­tions.

Even­tu­ally, I had a few in­sights that I’ve yet to find in the liter­a­ture, a few ideas that I still ac­tu­ally be­lieve are im­por­tant. You’ll ex­cuse me if I don’t men­tion them here: there is a lot of in­fer­en­tial dis­tance. Per­haps one day I’ll write a se­quence.

Even then, I could see no easy path to pub­lic sup­port. Most peo­ple lacked the knowl­edge to un­der­stand my claims with­out effort, and lacked the in­cen­tive to put in the effort for some un­proven boy.

Phase Two

For­tu­nately, I had other tricks up my sleeve.

I at­tempted three differ­ent tech star­tups. Two of them failed. The last was healthier, but we shut it down be­cause the ex­pected gains were lower than an in­dus­try salary. In the in­terim, I honed my pro­gram­ming skills and se­cured an in­dus­try job (I’m a soft­ware en­g­ineer at Google).

By the time I grad­u­ated, my ideas were largely re­fined and sta­ble. I had set­tled upon a solid meta so­cial sys­tem as an ideal to strive for, and I’m still fairly con­fi­dent that it’s a good one — one where the de­sign is forced by na­ture at ev­ery step, one that re­quires no ar­bi­trary choices, one that ratch­ets to­wards op­ti­mal­ity. And even if the ideal was not perfect, the mod­ern world is in­sane enough that even a small step to­wards a bet­ter-co­or­di­nated so­ciety would yield gi­gan­tic benefits.

The prob­lem changed from one of re­fin­ing ideas to one of con­vinc­ing oth­ers.

It was clear that I couldn’t spread my ideas by merely stat­ing them, due to the in­fer­en­tial dis­tance, so I started work­ing on two in­di­rect ap­proaches in the hours af­ter work.

The first was a book, which went back to my roots: sim­ple, low-cost ideas for how to change the cur­rent sys­tem of gov­ern­ment in small ways that could have large pay­offs. The goal of this pro­ject was to shake peo­ple from the blue-green mind­set, to con­vince them that we should stop bick­er­ing within the frame­work and con­sider mod­ify­ing the frame­work it­self. This book was meant to the be first in a se­ries, in which I’d slowly build to­wards more rad­i­cal sug­ges­tions.

The sec­ond pro­ject was de­signed to put peo­ple in a more ra­tio­nal frame of mind. I wanted peo­ple who could look past the la­bels and see the things, peo­ple who don’t just mem­o­rize how the world works but see it as muta­ble, as some­thing they can ac­tu­ally change. I wanted peo­ple that I could pull out of in­fer­en­tial gaps, in case they fell into mine.

Upon in­tro­spec­tion, I re­al­ized that much of my abil­ity came from a spe­cific out­look on the world that I had at a young age. I had a knack for un­der­stand­ing what the teach­ers were try­ing to teach me, for rec­og­niz­ing and dis­card­ing the cruft in their state­ments. I saw many fel­low stu­dents putting stock in his­tor­i­cal ac­ci­dents of ex­pla­na­tion where I found it easy to grasp the un­der­ly­ing con­cepts and drop the bag­gage. This abil­ity to cull the cruft is im­por­tant to un­der­stand­ing my grand de­signs.

This rea­son­ing (and a few other de­sires, in­clud­ing a per­pet­ual fas­ci­na­tion with math and physics) led me to cre­ate sim­plifience, a web­site that pro­motes such a mind­set.

It never made it to the point where I was com­fortable pub­li­ciz­ing it, but that hardly mat­ters any­more. In ret­ro­spect, it’s an un­finished jum­ble of ra­tio­nal­ity train­ing, math ex­pla­na­tions, and sci­ence en­thu­si­asm. It’s im­por­tant in one key re­spect:

As I was writ­ing sim­plifience, I did a lot of re­search for it. Dur­ing this re­search, I kept stum­bling upon web ar­ti­cles on this one web­site that ar­tic­u­lated what I was try­ing to ex­press, only bet­ter. That web­site was LessWrong, and those ar­ti­cles were the Se­quences.

It took me an em­bar­rass­ingly long time to ac­tu­ally pay at­ten­tion. In fact, if you go to sim­plifience.com, you can watch as the ar­ti­cles grow more and more in­fluenced by the se­quences. My ex­po­sure to them was patchy, cen­tered around ideas that I’d already had. It took me a while to re­al­ize that I should read the rest of them, that I might learn new things that ex­tended the ideas I’d figured out on my own.

It seemed like a good way to learn how to think bet­ter, to learn from some­one who had had similar in­sights. I didn’t even con­sider the pos­si­bil­ity that this au­thor, too, had some grand agenda. The idea that Eliezer’s agenda could be more press­ing than my own never even crossed my mind.

At this point, you may be able to em­pathize with how I felt when I first re­al­ized the im­por­tance of an in­tel­li­gence ex­plo­sion.


It was like get­ting ten years worth of wind knocked out of me.

I saw some­thing fa­mil­iar in the se­quences — the wind­ing, metic­u­lous ex­pla­na­tions of some­one strug­gling to bridge an in­fer­en­tial gap. I rec­og­nized the need to cover sub­jects that looked com­pletely tan­gen­tial to the ac­tual point, just to get peo­ple to the level where they wouldn’t re­ject the main ideas out-of-hand. I no­ticed the peo­ple fal­ling to the side, de­bat­ing is­sues two or three steps be­fore the ac­tual in­ter­est­ing prob­lems. It was this fa­mil­iar pat­tern, above all else, that made me ac­tu­ally pay at­ten­tion.

Every­thing clicked. I was already thor­oughly con­vinced of civ­i­liza­tional in­ad­e­quacy. I had long since con­cluded that there’s not much that can hold a strong in­tel­li­gence down. I had a sort of vague idea that an AI would seek out “good” val­ues, but such illu­sions were eas­ily dis­pel­led — I was a moral rel­a­tivist. And the stakes were as high as stakes go. Ar­tifi­cial in­tel­li­gence was a prob­lem more press­ing than my own.

The re­al­iza­tion shook me to my core. It wasn’t even the in­tel­li­gence ex­plo­sion idea that scared me, it was the rev­e­la­tion of a fatal flaw at the foun­da­tion of my be­liefs. Poorly de­signed gov­ern­ments had awo­ken my fear that so­ciety can’t han­dle co­or­di­na­tion prob­lems, but I never — not once in nearly a decade — stopped to con­sider whether de­sign­ing bet­ter so­cial sys­tems was ac­tu­ally the best way to op­ti­mize the world.

I pro­fessed a de­sire to save the world, but had mi­s­un­der­stood the play­ing field so badly that ex­is­ten­tial risk had never even crossed my mind. Some­how, I had missed the most im­por­tant prob­lems, and they should have been ob­vi­ous. Some­thing was very wrong.

It was time to halt, melt, and catch fire.

This was one of the most difficult things I’ve done.

I was more care­ful, the sec­ond time around. The Se­quences shook my foun­da­tions and brought the whole tower crash­ing down, but what I would build in its place was by no means a fore­gone con­clu­sion.

I had been blind to all ex­is­ten­tial risks, not just AI risk, and there was a pos­si­bil­ity that I had missed other fea­tures of the prob­lem space as well. I was well aware of the fact that, hav­ing been in­tro­duced to AI risk by Eliezer’s writ­ings, I was bi­ased to­wards his view­point. I didn’t want to make the same mis­take twice, to jump for the sec­ond big prob­lem that crossed my path just be­cause it was larger than the first. I had to start from scratch, rea­son­ing from the be­gin­ning. I knew I must watch out for con­junc­tion fal­la­cies caused by nice nar­ra­tives, ar­gu­ments made from high stakes (Pas­cal’s mug­ging), putting too much stock on in­side views, and so on. I had to figure out how to ac­tu­ally save the world.

It took me a long time to de­pro­gram, to get back to neu­tral. I con­sid­ered care­fully, ac­count­ing for my bi­ases as best I could. I read a lot. I weighed the ev­i­dence. The pro­cess took many months.

By July of 2013, I came to agree with MIRI’s con­clu­sions.


Writ­ing it all out like this, I re­al­ize that I’ve failed to con­vey the feel­ing of it all. Depend­ing upon whether you be­lieve that I was ac­tu­ally able to come up with bet­ter ways to struc­ture peo­ple, you may feel that I’m ei­ther pretty ac­com­plished or ex­tremely de­luded. Per­haps both.

Really, though, it’s nei­ther. This raw story, which omits de­tails from the rest of my life, paints a strange pic­ture in­deed. The in­ten­sity is dis­til­led.

I was not a zealot, in prac­tice. My at­tempts to save the world didn’t bleed much into the rest of my life. I learned early on that this wasn’t the sort of thing that most peo­ple en­joyed dis­cussing, and I was wary of in­fer­en­tial gaps. My work was done par­allel to an oth­er­wise nor­mal life. Only a se­lect few peo­ple were privy to my goals, my con­clu­sions. The whole thing of­ten felt dis­con­nected from re­al­ity, just some un­usual hobby. The ma­jor­ity of my friends, if they read this, will be sur­prised.

There are many holes in this sum­mary, too. It fails to cap­ture the dark spots. It omits the feel­ings of un­cer­tainty and hel­pless­ness, the cy­cles of guilt at be­ing un­pro­duc­tive fol­lowed by lin­ger­ing de­pres­sion, the wa­ver­ing be­tween staunch ideal­ism and a con­vic­tion that my goals were noth­ing but a com­fortable fan­tasy. It skips over the year I burned out, writ­ing the whole idea off, study­ing abroad and build­ing my­self a healthier men­tal state be­fore re­turn­ing and pick­ing ev­ery­thing back up.

Noth­ing in this sum­mary de­scribes the con­stant doubt about whether I was pur­su­ing the best path or merely the eas­iest one. I’ve failed to men­tion my com­plete failure to net­work and my spec­tac­u­lar in­abil­ity to find peo­ple who would ac­tu­ally take me se­ri­ously. It’s hard to con­vey the fear that I was just pre­tend­ing I wanted to save the world, just act­ing like I was try­ing, be­cause that’s the nar­ra­tive that I wanted. How could some­one ‘smart’ ac­tu­ally fail to find pow­er­ful friends if they were re­ally try­ing for nine years?

I claim no glory: the jour­ney was messy, and it was poorly ex­e­cuted. I tell the story in part be­cause peo­ple have asked me where my pas­sion comes from and how I be­came al­igned with MIRI’s mis­sion. Mostly, though, I tell the story be­cause it feels like some­thing I have to tell be­fore mov­ing on. It feels al­most dishon­est to try to save the world in this new way with­out at least ac­knowl­edg­ing that I walked an­other path, once.

The source of my passion

So to those of you won­der­ing where my pas­sion comes from, I an­swer this: it has always been there. It was a small flame, when I was young, and it was fed by a deep mis­trust in so­ciety’s ca­pa­bil­ities and a strong be­lief that if any­one can mat­ter then I had bet­ter try.

From my per­spec­tive, I’ve been ded­i­cat­ing my en­ergy to­wards ‘sav­ing the world’ since first I re­al­ized that the world was in need of sav­ing. This pas­sion was not re­cently kin­dled, it was merely redi­rected.

There was a burst of pro­duc­tivity these past few months, af­ter I re­fo­cused my efforts. I was given a new path, and on it the analo­gous ob­sta­cles have already been sur­mounted. MIRI has already spent years pro­mot­ing that ra­tio­nal state of mind, bridg­ing its in­fer­en­tial gap, find­ing peo­ple who can ac­tu­ally work on solv­ing the prob­lem in­stead of ar­gu­ing about whether there is a prob­lem to be solved. This was in­vi­go­rat­ing, like skip­ping ahead ten years in terms of where I wanted to be.

Alongside that, I felt a burn­ing need to catch up. I was late to the party, and I had been fool­ish for a very long time. I was ter­rified that I wouldn’t ac­tu­ally be able to help — that, af­ter all my work, the most I’d be able to do to solve the big prob­lems was earn to give. I’d have done it, be­cause the ac­tual goal is to save the world, not to satisfy Nate. But the idea scared me, and the de­sire to keep ac­tively work­ing on the big prob­lems drove me for­ward.

In a way, too, ev­ery­thing got eas­ier — I needed only to be­come good at logic and de­ci­sion the­ory, to read a bunch of math text­books, a task that was triv­ially mea­surable and joyfully easy com­pared to try­ing to con­vince the en­tire world to em­brace strange, un­pol­ished ideas.

All these fac­tors con­tributed to my re­cent pro­duc­tivity. But the pas­sion, the fer­vor, the de­sire to op­ti­mize the fu­ture — that has been there for a long time. Peo­ple some­times ask where I get my pas­sion from, and I find it hard to an­swer.

We hold the en­tire fu­ture of the uni­verse in our hands. Is that not jus­tifi­ca­tion enough?

I learned a long time ago that most peo­ple are con­tent to ac­cept the way things are. Every­one wants the world to change, but most are cowed by the fact that they can’t change it them­selves.

But if the chance that one per­son can save the world is one in a mil­lion, then there had bet­ter be a mil­lion peo­ple try­ing.

It is this knowl­edge — that the world will only be saved by peo­ple who ac­tu­ally try to save it — that drives me.

I still have these strange ideas, this pet in­fer­en­tial gap that I hope to bridge one day. It still hurts, that things im­por­tant to me were su­per­seded, but they were su­per­seded, and it is bet­ter to know than to re­main in the dark.

When I was four­teen, I saw many hor­rors laid out be­fore us: war, cor­rup­tion, en­vi­ron­men­tal de­struc­tion, and the silent tragedies of au­to­mo­bile ac­ci­dents, court­room in­jus­tices, and death by dis­ease and ag­ing. All around me, I saw a so­ciety that couldn’t co­or­di­nate, full of peo­ple re­signed to un­nec­es­sary fates.

I was told to set­tle for mak­ing a small differ­ence. I re­solved to do the op­po­site.

I made a promise to my­self. I didn’t promise to fix gov­ern­ments: that was a means to an end, a con­ve­nient solu­tion for some­one who didn’t know how to look fur­ther out. I didn’t promise to change the world, ei­ther: ev­ery lit­tle thing is a change, and not all changes are good. No, I promised to save the world.

That promise still stands.

The world sure as hell isn’t go­ing to save it­self.