Prolegomena to a Theory of Fun

Eliezer Yudkowsky17 Dec 2008 23:33 UTC
61 points
52 comments6 min readLW linkArchive

Followup to: Joy in the Merely Good

Raise the topic of cryonics, uploading, or just medically extended lifespan/​healthspan, and some bioconservative neo-Luddite is bound to ask, in portentous tones:

“But what will people do all day?”

They don’t try to actually answer the question. That is not a bioethicist’s role, in the scheme of things. They’re just there to collect credit for the Deep Wisdom of asking the question. It’s enough to imply that the question is unanswerable, and therefore, we should all drop dead.

That doesn’t mean it’s a bad question.

It’s not an easy question to answer, either. The primary experimental result in hedonic psychology—the study of happiness—is that people don’t know what makes them happy.

And there are many exciting results in this new field, which go a long way toward explaining the emptiness of classical Utopias. But it’s worth remembering that human hedonic psychology is not enough for us to consider, if we’re asking whether a million-year lifespan could be worth living.

Fun Theory, then, is the field of knowledge that would deal in questions like:

  • “How much fun is there in the universe?”

  • “Will we ever run out of fun?”

  • “Are we having fun yet?”

  • “Could we be having more fun?”

One major set of experimental results in hedonic psychology has to do with overestimating the impact of life events on happiness. Six months after the event, lottery winners aren’t as happy as they expected to be, and quadriplegics aren’t as sad. A parent who loses a child isn’t as sad as they think they’ll be, a few years later. If you look at one moment snapshotted out of their lives a few years later, that moment isn’t likely to be about the lost child. Maybe they’re playing with one of their surviving children on a swing. Maybe they’re just listening to a nice song on the radio.

When people are asked to imagine how happy or sad an event will make them, they anchor on the moment of first receiving the news, rather than realistically imagining the process of daily life years later.

Consider what the Christians made of their Heaven, meant to be literally eternal. Endless rest, the glorious presence of God, and occasionally—in the more clueless sort of sermon—golden streets and diamond buildings. Is this eudaimonia? It doesn’t even seem very hedonic.

As someone who said his share of prayers back in his Orthodox Jewish childhood upbringing, I can personally testify that praising God is an enormously boring activity, even if you’re still young enough to truly believe in God. The part about praising God is there as an applause light that no one is allowed to contradict: it’s something theists believe they should enjoy, even though, if you ran them through an fMRI machine, you probably wouldn’t find their pleasure centers lighting up much.

Ideology is one major wellspring of flawed Utopias, containing things that the imaginer believes should be enjoyed, rather than things that would actually be enjoyable.

And eternal rest? What could possibly be more boring than eternal rest?

But to an exhausted, poverty-stricken medieval peasant, the Christian Heaven sounds like good news in the moment of being first informed: You can lay down the plow and rest! Forever! Never to work again!

It’d get boring after… what, a week? A day? An hour?

Heaven is not configured as a nice place to live. It is rather memetically optimized to be a nice place for an exhausted peasant to imagine. It’s not like some Christians actually got a chance to live in various Heavens, and voted on how well they liked it after a year, and then they kept the best one. The Paradise that survived was the one that was retold, not lived.

Timothy Feriss observed, “Living like a millionaire requires doing interesting things and not just owning enviable things.” Golden streets and diamond walls would fade swiftly into the background, once obtained —but so long as you don’t actually have gold, it stays desirable.

And there’s two lessons required to get past such failures; and these lessons are in some sense opposite to one another.

The first lesson is that humans are terrible judges of what will actually make them happy, in the real world and the living moments. Daniel Gilbert’s Stumbling on Happiness is the most famous popular introduction to the research.

We need to be ready to correct for such biases—the world that is fun to live in, may not be the world that sounds good when spoken into our ears.

And the second lesson is that there’s nothing in the universe out of which to construct Fun Theory, except that which we want for ourselves or prefer to become.

If, in fact, you don’t like praying, then there’s no higher God than yourself to tell you that you should enjoy it. We sometimes do things we don’t like, but that’s still our own choice. There’s no outside force to scold us for making the wrong decision.

This is something for transhumanists to keep in mind—not because we’re tempted to pray, of course, but because there are so many other logical-sounding solutions we wouldn’t really want.

The transhumanist philosopher David Pearce is an advocate of what he calls the Hedonistic Imperative: The eudaimonic life is the one that is as pleasurable as possible. So even happiness attained through drugs is good? Yes, in fact: Pearce’s motto is “Better Living Through Chemistry”.

Or similarly: When giving a small informal talk once on the Stanford campus, I raised the topic of Fun Theory in the post-talk mingling. And someone there said that his ultimate objective was to experience delta pleasure. That’s “delta” as in the Dirac delta—roughly, an infinitely high spike (that happens to be integrable). “Why?” I asked. He said, “Because that means I win.”

(I replied, “How about if you get two times delta pleasure? Do you win twice as hard?”)

In the transhumanist lexicon, “orgasmium” refers to simplified brains that are just pleasure centers experiencing huge amounts of stimulation—a happiness counter containing a large number, plus whatever the minimum surrounding framework to experience it. You can imagine a whole galaxy tiled with orgasmium. Would this be a good thing?

And the vertigo-inducing thought is this—if you would prefer not to become orgasmium, then why should you?

Mind you, there are many reasons why something that sounds unpreferred at first glance, might be worth a closer look. That was the first lesson. Many Christians think they want to go to Heaven.

But when it comes to the question, “Don’t I have to want to be as happy as possible?” then the answer is simply “No. If you don’t prefer it, why go there?”

There’s nothing except such preferences out of which to construct Fun Theory—a second look is still a look, and must still be constructed out of preferences at some level.

In the era of my foolish youth, when I went into an affective death spiral around intelligence, I thought that the mysterious “right” thing that any superintelligence would inevitably do, would be to upgrade every nearby mind to superintelligence as fast as possible. Intelligence was good; therefore, more intelligence was better.

Somewhat later I imagined the scenario of unlimited computing power, so that no matter how smart you got, you were still just as far from infinity as ever. That got me thinking about a journey rather than a destination, and allowed me to think “What rate of intelligence increase would be fun?”

But the real break came when I naturalized my understanding of morality, and value stopped being a mysterious attribute of unknown origins.

Then if there was no outside light in the sky to order me to do things—

The thought occurred to me that I didn’t actually want to bloat up immediately into a superintelligence, or have my world transformed instantaneously and completely into something incomprehensible. I’d prefer to have it happen gradually, with time to stop and smell the flowers along the way.

It felt like a very guilty thought, but—

But there was nothing higher to override this preference.

In which case, if the Friendly AI project succeeded, there would be a day after the Singularity to wake up to, and myself to wake up to it.

You may not see why this would be a vertigo-inducing concept. Pretend you’re Eliezer2003 who has spent the last seven years talking about how it’s forbidden to try to look beyond the Singularity—because the AI is smarter than you, and if you knew what it would do, you would have to be that smart yourself—

—but what if you don’t want the world to be made suddenly incomprehensible? Then there might be something to understand, that next morning, because you don’t actually want to wake up in an incomprehensible world, any more than you actually want to suddenly be a superintelligence, or turn into orgasmium.

I can only analogize the experience to a theist who’s suddenly told that they can know the mind of God, and it turns out to be only twenty lines of Python.

You may find it hard to sympathize. Well, Eliezer1996, who originally made the mistake, was smart but methodologically inept, as I’ve mentioned a few times.

Still, expect to see some outraged comments on this very blog post, from commenters who think that it’s selfish and immoral, and above all a failure of imagination, to talk about human-level minds still running around the day after the Singularity.

That’s the frame of mind I used to occupy—that the things I wanted were selfish, and that I shouldn’t think about them too much, or at all, because I would need to sacrifice them for something higher.

People who talk about an existential pit of meaninglessness in a universe devoid of meaning—I’m pretty sure they don’t understand morality in naturalistic terms. There is vertigo involved, but it’s not the vertigo of meaninglessness.

More like a theist who is frightened that someday God will order him to murder children, and then he realizes that there is no God and his fear of being ordered to murder children was morality. It’s a strange relief, mixed with the realization that you’ve been very silly, as the last remnant of outrage at your own selfishness fades away.

So the first step toward Fun Theory is that, so far as I can tell, it looks basically okay to make our future light cone—all the galaxies that we can get our hands on—into a place that is fun rather than not fun.

We don’t need to transform the universe into something we feel dutifully obligated to create, but isn’t really much fun—in the same way that a Christian would feel dutifully obliged to enjoy heaven—or that some strange folk think that creating orgasmium is, logically, the rightest thing to do.

Fun is okay. It’s allowed. It doesn’t get any better than fun.

And then we can turn our attention to the question of what is fun, and how to have it.

Part of The Fun Theory Sequence

Next post: “High Challenge”

(start of sequence)

Eliezer Yudkowsky17 Dec 2008 23:33 UTC
61 points
52 comments6 min readLW linkArchive