Self-sacrifice is a scarce resource

“I just solved the trol­ley prob­lem.… See, the trol­ley prob­lem forces you to choose be­tween two ver­sions of let­ting other peo­ple die, but the ac­tual solu­tion is very sim­ple. You sac­ri­fice your­self.”

- The Good Place

High school: Naive morality

When I was a teenager, I had a very sim­ple, naive view of moral­ity. I thought that the right thing to do was to make oth­ers happy. I also had a naive view of how this was ac­com­plished—I spent my time dig­ging through trash cans to sort out the re­cy­clables, pick­ing up lit­ter on the streets, read­ing the Com­mu­nist Man­i­festo, go­ing to protests, you know, high school kind of stuff. I also poured my heart into my dance group which was al­most en­tirely com­prised of dis­ad­van­taged stu­dents—mainly poor, de­vel­op­men­tally dis­abled, or severely de­pressed, though we had all sorts. They were good peo­ple, for the most part, and I liked many of them sim­ply as friends, but I prob­a­bly also had some sort of in­tel­li­gentsia sav­ior com­plex go­ing on with the amount of effort I put into that group.

The mo­ment of reck­on­ing for my naive moral­ity came when I started dat­ing a de­pressed, trau­ma­tized, and un­be­liev­ably pedan­tic boy with a su­pe­ri­or­ity com­plex even big­ger than his voice was loud. I didn’t like him. I think there was a time when I thought I loved him, but I always knew I didn’t like him. He was deeply un­kind, and it was like there was noth­ing real in­side of him. But my naive moral­ity told me that dat­ing him was the right thing to do, be­cause he liked me, and be­cause maybe if I gave enough of my­self I could fix him, and then he would be kind to oth­ers like he was to me. Need­less to say this did not work. I am much worse off for the choices I made at that time, with one effect be­ing that I have trou­ble dis­t­in­guish­ing be­tween giv­ing too much of my­self and just giv­ing ba­sic hu­man de­cency.

And even if it were true that pour­ing all of my love and good­ness out for a bro­ken per­son could make them whole again, what good would it be? There are mil­lions of sad peo­ple in the world, and with that method I would only be able to save a few at most (or in re­al­ity, one, be­cause of how badly pour­ing kind­ness into a black hole burns you out). If you re­ally want to make peo­ple’s lives bet­ter, that is, if you re­ally care about hu­man flour­ish­ing, you can’t give your whole self to save one per­son. You only have one self to give.

Effec­tive al­tru­ism, my early days

When I first moved to the Bay, right af­ter col­lege, I lived with five other peo­ple in what could per­haps prac­ti­cally but cer­tainly not legally be called a four-bed­room apart­ment. Four of the oth­ers were my age, and three of us (in­clud­ing me) were ve­gan. The pre­vi­ous ten­ants had left be­hind a large box of oat­meal and a gal­lon of cin­na­mon, so that was most of what I ate, though I some­times bought a jar of peanut but­ter to spice things up or mooched food off of our one adult house­mate. I was pretty young and pretty new to EA and I didn’t think it was morally per­mis­si­ble to spend money, and many of my house­mates seemed to think like­wise. Crazy-burnout-guy work was ba­si­cally the only thing we did—var­i­ously for CEA, CHAI, GiveWell, LessWrong, and an EA startup. My room­mate would be gone when I woke up and not back from work yet when I fell asleep, and there was work hap­pen­ing at ba­si­cally all hours. One time my room­mate and I asked Habryka if he wanted to read Luke’s re­port on con­scious­ness with us on Fri­day night and he told us he would be busy; when we asked with what he said he’d be work­ing.

One day I met some Aus­tralian guys who had been there in the re­ally early days of EA, who told us about eat­ing out of the garbage (re­ally!) and sleep­ing seven to a hal­lway or some­thing ridicu­lous like that, so that they could donate fully 100% of their earn­ings to global poverty. And then I felt bad about my­self, be­cause even though I was ve­gan, liv­ing in a ten­e­ment, half-starv­ing my­self, and work­ing for an EA org, I could have been do­ing more.

It was a long and com­plex pro­cess to get from there to where I am now, but suffice it to say I now re­al­ize that be­ing mis­er­able and half-starv­ing is not an ideal way to set one­self up for any kind of pro­duc­tive work, world-sav­ing or oth­er­wise.

You can’t make a policy out of self-sacrifice

I want to cir­cle back to the quote at the be­gin­ning of this post. (Don’t worry, there won’t be any spoilers for The Good Place). It’s sup­posed to be a touch­ing mo­ment, and in some ways it is, but it’s also frus­trat­ing. Whether or not self-sac­ri­fice was cor­rect in that situ­a­tion misses the point; the prob­lem is that self-sac­ri­fice can­not be the an­swer to the trol­ley prob­lem.

Let’s say, for sim­plic­ity’s sake, that me jump­ing in front of the trol­ley will stop it. So I do that, and boom, six lives saved. But if the trol­ley prob­lem is a metaphor for any real-world prob­lem, there are mil­lions of trol­leys hurtling down mil­lions of tracks, and whether you jump in front of one of those trol­leys your­self or not, mil­lions of peo­ple are still go­ing to die. You still need to come up with a policy-level an­swer for the prob­lem, and the fact re­mains that the policy that will re­sult in the fewest deaths is switch­ing tracks to kill one per­son in­stead of five. You can’t jump in front of a mil­lion trol­leys.

There may be times when self-sac­ri­fice is the best of sev­eral bad op­tions. Like, if you’re in a crash­ing air­plane with Eliezer Yud­kowsky and Scott Alexan­der (or sub­sti­tute your morally im­por­tant figures of choice) and there are only two parachutes, then sure, there’s prob­a­bly a good ar­gu­ment to be made for let­ting them have the parachutes. But the point I want to make is, you can’t make a policy out of self-sac­ri­fice. Be­cause there’s only one of you, and there’s only so much of you that can be given, and it’s not nearly com­men­su­rate with the amount of ill in the world.


I am not at­tempt­ing to ar­gue that, in do­ing your best to do the right thing, you will never have to make de­ci­sions that are painful for you. I know many a per­son work­ing on AI safety who, if the world were differ­ent, would have loved noth­ing more than to be a physi­cist. I’m glad for my work in the Bay, but I also re­gret not liv­ing nearer to my par­ents as they grow older. We all make sac­ri­fices at the al­tar of op­por­tu­nity cost, but that’s true for ev­ery­one, whether they’re try­ing to do the right thing or not.

The key thing is that those AI safety re­searchers are not mak­ing them­selves mis­er­able with their choices, and nei­ther am I. We en­joy our work and our lives, even if there are other things we might have en­joyed that we’ve had to give up for var­i­ous rea­sons. Choos­ing the path of least re­gret doesn’t mean you’ll have no re­grets on the path you go down.

The differ­ence, as I see it, is that the “self-sac­ri­fices” I talked about ear­lier in the post made my life strictly worse. I would have been strictly bet­ter off if I hadn’t poured kind­ness into some­one I hated, or if I hadn’t lived in a dark con­verted cafe with a night­mare shower and tried to sub­sist off of stale oat­meal with no salt.

You’ll most likely have to make sac­ri­fices if you’re aiming at any­thing worth­while, but be care­ful not to fol­low poli­cies that de­plete the core of your­self. You won’t be very good at achiev­ing your goals if you’re burnt out, trau­ma­tized, or dead. Self-sac­ri­fice is gen­er­ally thought of as vir­tu­ous, in the col­lo­quial sense of the word, but moral­ities that ad­vo­cate it are un­likely to lead you where you want to go.

Self-sac­ri­fice is a scarce re­source.