You can value other things than actions themselves and the pleasure they bring. Maybe you like choosing and planning and acting. Maybe you like solving complex problems. Maybe you like variety (laughter and food and discovery and music, rather than the best out of four) and complexity. Maybe you like growing and learning. Maybe you like creating things. Maybe you like overcoming obstacles. Maybe you like exchanging ideas with other minds and bulding relationships and helping. Maybe you like intricate webs of feelings that aren’t all pleasant but are made more enjoyable because they’re bittersweet or frightening or weird.
So you’ll select activities that maximize that and your zillion other values, not just activities that maximize pleasure. But there’s no particular reason why current activities should be optimal. The pleasure of scratching an itch is unique, and somewhat complex. I wouldn’t trade it for, say, a more intense copy of the pleasure of eating a potato chip—I already have that one. But I would trade it for a different, more intense, more complex pleasure.
But I would trade it for a different, more intense, more complex pleasure.
Which is my thesis: outside of denying change, the only two dimension under which is significant to find an improvement are intensity and complexity. However your position appears to me not to be reflectively consistent: after all, you could consider the complexity and intensity of your whole human experience and be seduced by a different, more complex, more intense type of existence.
As I parse it, yes! It’d have to be done by gradual change (I don’t want to die and be replaced by someone cooler), but yeah, that’s what I want. Where am I inconsistent?
Hmm, don’t we disagree about connotations? You seem to think “orgasmium” covers that kind of being that has varied, complex, intense fun, and does things instead of just feeling like it does them. Do you?
No I don’t. My reply to muflax was not related from this one. I don’t have a name for this state of affair, it doesn’t even seem to have an official name. Cooler existence is ok for me
So what is it you are asking for, then? Merely that you don’t just get the fun, but the things that cause the fun as well?
If so, then that’s a pretty straightforward point that any FAI would respect. (Even though I tend to think it’s mistaken, or at least, awfully inefficient.)
I’m curious. You’re alone in the universe, will always be. After centuries of hard work, you’ve finished your Theory of Everything and Then Some. Your mind strains under the ecstasy of your discovery. You’re now in a room with two buttons. Button A resets you to your mindstate thirty seconds ago, lets you experience those thirty seconds of extreme bliss, then repeats, for five minutes. Button B entertains you for five minutes with mildly amusing videos of cats dressed as anime characters. Which button do you press?
You are not resolving my curiosity! (Well, you’re telling me I haven’t completely misunderstood your position, which is something.) I’d like to know your reasons for your reasons for choosing A (your reasons for choosing A are as you say obvious).
My intuitions say that A is wrong on the same level that replacing all kinds of fun by a big orgasm is sad. This is connected to the intuition that overcoming things is good (and you can’t cheat by tying the “Behold! I, human, have vanquished this!” emotion to eating a potato chip), and that choosing and planning and acting are fun. I’m not done unpacking them.
(Come to think of it, I’m not sure I press B, though I probably do. But if B gives me a small, pleasant, new insight (say, a new way to prove Pythagoras’s theorem) instead, I press it.)
I choose A because A is the most fun thing to do. I find it really hard to get any deeper reasons than that. I’ll try to elaborate anyway and hope it doesn’t get too messy. It would be really weird for humans to have that large a value dissonance, so there’s probably some confusion around here somewhere.
A couple of months ago, I even would’ve agreed with you and would’ve opposed straight orgasmium. But I just can’t find any argument against it. I tried tracing my enjoyment of activities, but I couldn’t find nuanced, complex fun there. It’s all composed of few (probably <5) components. I enjoy solving complex problems, but that’s because it’s one of the few ways I know to trigger the sensation of enjoying-to-learn. It’s the only reason I even bother with programming anymore. If I could trigger the same learning experience by reading a dictionary, I’d get back to studying Latin. It seems very much possible to just straight maximize all components and reach perfect orgasmium. (I strongly suspect they are all bounded, as I already have found my personal Best Movie[1] and Best Song[2] and have for years not been able to improve one bit on that enjoyment, only match it.)
If I could self-modify to eat chips like Light (Death Note spoiler), I totally would. Why is that sad or cheating? Who are you wronging? What code are you breaking? If it’s a way to maximize fun, then so be it. It doesn’t match common tropes, it doesn’t have much status and you don’t look clever for doing it, but if it works, then that’s that. Not my fault that the universe has obvious cheat codes built in. Screw the dungeon, just get the cheese already.
I strongly suspect that I don’t actually care that my values are fulfilled outside of my experience. I see no reason why anyone would. The only explanation that even begins to make sense to me is memetic hijacking, as Tim Tyler often proposes. Claiming to oppose hedonistic optimization might easily be evolutionary beneficial. It looks similar to normal altruistic trade-offs. (Of course this doesn’t discredit these more complicated values. A rational agent is free to value whatever it wants, regardless how it got these values. But it would explain why some agents have them and others don’t.)
(I’m assuming the least convenient world where there aren’t any choices around that are more enjoyable than A, where being locked in a loop of pressing A doesn’t prevent me from fulfilling some other desires and so on. As I already have a Theory of Everything and am alone, I can’t off the top of my head think of any value that would be left out anyway.)
I tried tracing my enjoyment of activities, but I couldn’t find nuanced, complex fun there.
You like G. R. R. Martin and you don’t have complex fun?! You said you have depression, this is probably linked (boo for flat affect). I do have complex fun. Like, just yesterday I ate a peach and a pastry and compared everything about the texture and the taste and the temperature, had an uncontrollable urge to run my fingers over every surface in the house, and thought about reducing qualia. (And, uh, I already eat crisps like Light, but like him I would enjoy it more if it were part of an intricate world-conquering plot.)
If it’s a way to maximize fun, then so be it.
Clippy would say “If it’s a way to maximize paperclips, then so be it. ” I don’t care only about fun, I care about a zillion things only a few of which fall under the usual concept of fun. I want the dungeon. You don’t go in the cave of trials for the items, you go for the challenge and the made-more-valuable-by-the-challenge items. I don’t want to stare at a “You win” screen forever.
And I don’t actually know why! All I can say is “Becoming orgasmium? That sounds pointless. I’m not going to renounce complexity and variety to make my value system more elegant, it’s just not worth it”.
I do know why I value things outside my own experience. (Will shoot you and get tortured for it rather than let you press the blow-up-the-world-when-we’re-both-dead button, for example.) There’s no reason optimization processes should optimize their input sequences only. Clippy doesn’t want to overestimate the number of paperclips in the world.
You like G. R. R. Martin and you don’t have complex fun?!
GRRM is one of the few authors who writes fiction that doesn’t insult my intelligence. It’s not that I prefer his work to, say, Naruto. It’s just that his is the only I can enjoy at all. I remember reading much more trash in my childhood and found them equally enjoyable. If I try reading them now, I just keep on rolling my eyes and raging.
You said you have depression, this is probably linked (boo for flat affect).
I don’t think so. I don’t feel depressed right now (luckily, for months). I sometimes did have a fairly flat affect; that ain’t now. The degradation effect I described seems fairly common. Lots of experiments in happiness studies show that set levels adjust ruthlessly. Also, Daoists and Buddhists, among many others, have advocated simplicity in favor of complexity and it seems to work.
(Which is the reason I intentionally don’t learn to cook so I can still enjoy really cheap and simple food. I don’t want to fall into the same trap my obsession with fiction has gotten me into.)
Clippy would say “If it’s a way to maximize paperclips, then so be it.”
Yes, and Clippy would be right. So? I don’t see how this is a counter-argument. This reminds me of AIXI agents wireheading themselves. I don’t think that’s a failure mode. It’s a perfect valid action and if anything shows that they actually are clever. Of course it sucks for the programmers who wanted to get other things done. Similarly, evolution might’ve taken care to prevent this happening in humans for the same reasons, although it failed in my case. (Pure speculation.)
I do know why I value things outside my own experience.
Why? Honest question, I just get “404” if I ask myself that.
My perspective is that there are rewards and actions to get these rewards. I think that the rewards are the important thing. They are not instruments to get to my “real” values, they are my values. I was set up to favor specific pathways to these rewards for reasons that are not my own (but those of my genes, memes or other influences). I, too, sometimes feel bad for bypassing them, but that is a very obvious security feature and I’m much more willing to consider this aversion to be in error than the bypass.
For example, D&D has many ways to get Infinite Wish Spells. They completely break the game and prevent all the intended benefits of the game, so it’s a bad idea for the players to use them as shortcuts. It’s much better to go through long, dangerous dungeons than to just ignore everyone else and ascend to godhood and take over the whole universe. But that’s because the one who is making the decisions is the player, who wants to socialize and boast and so on. If you lived in D&D, this intuition, this aversion to do things that weren’t “intended”, would not serve you well. Now you should go for the game-breaker.
The most obvious anti-hedonism feature is boredom. I agree with you that
I don’t want to stare at a “You win” screen forever.
But I consider this boredom to be a bad feature! I wish I could just stare at “you win” instead of having to jump through all the hoops every time, in increasing difficulty, with dwindling supply, as has already happened. I’ve watched >90% of every anime even remotely interesting to me and can’t rarely get any real enjoyment out of them anymore because this stupid boredom is adjusting my happiness set level down. I’d much prefer to just get The Perfect Entertainment and have it stay that way, forever. Simplicity and elegance are good things.
I notice that this discussion makes me feel resigned and sad, so I will adjust down my confidence that this is right. Can you try to elaborate on why you value external things? Why you think a holodeck is bad, except for the emotions that come up?
(Besides, just pay me more than the 10 bucks Dave offered and I’m not pressing anything. I’m very pain-averse, so no need to go through such extremes. ;))
The degradation effect I described seems fairly common. Lots of experiments in happiness studies show that set levels adjust ruthlessly.
New fun gets old. We want variety over time as well as space. Doesn’t affect complexity or external referents.
Yes, and Clippy would be right.
I meant that fun-maximizers don’t have more power to move me than paperclips-maximizers.
I do know why I value things outside my own experience.
Why? Honest question, I just get “404” if I ask myself that.
Just that there’s no particular reason for my values to be only about my experience, like there’s no reason for them to be only about parts of the universe that are green.
My perspective is that there are rewards and actions to get these rewards. I think that the rewards are the important thing. They are not instruments to get to my “real” values, they are my values.
Yeah, I understand that. I don’t know why I disagree. I value the reward, but I also value many things about the action.
As an example: in Judaism, you want to set obstacles for yourself and to overcome them (there even is a bunch of rules to prevent this from going overboard). There are vegan Orthodow Jews who complain that keeping kosher is too easy for them and who decide some vegetables count as milk and others as meat and they don’t mix them. Chosen fetters like that appeal to me like mad. It’s a kind of fun, but, I expect, one completely alien to you.
I was set up to favor specific pathways to these rewards for reasons that are not my own (but those of my genes, memes or other influences).
I have to ask where your own reasons come from, causally speaking.
Now you should go for the game-breaker.
Agree we should break the rules to get what we want, disagree about what we want.
But I consider this boredom to be a bad feature!
So you want classical orgasmium, not complex fun with no referent (like falling in love with a chatbot). Glad that’s clear.
Simplicity and elegance are good things.
Yes. But not so good that I’d renounce humor and challenge and pride and aesthetics and freedom and truth and more. Same reason that I can’t decide to burn down an orphanage—I care about making a point about choosing my own values and relativity of morality, but I care about human life more.
I notice that this discussion makes me feel resigned and sad, so I will adjust down my confidence that this is right.
I’ve been adjusting in the opposite direction as well. Truly we are an Aumann cage match made in heaven.
Can you try to elaborate on why you value external things?
As I said earlier, why wouldn’t I? I value non-green things.
My brain sucks. It can’t represent the joys and great deeds and wisdom in a single human life (except the one it’s being). Unless other people are clever chatbots, there are more great things in a day of their lives than in my brain in its highest bliss. It sounds just odd that this should be worthless. (Also phrased as: Every morning, I weigh myself. If it’s less than Earth, I save the world.)
Also, not sure what happens to the value of suicide if you value only your subjective experience. Isn’t it undefined?
Not directly related, but values are over 4D, not 3D. (Which is why it’s not completely stupid to care about paths, not just endpoints.)
Why you think a holodeck is bad, except for the emotions that come up?
Same reason I think that if I could never have anything more pleasant than ice cream (but I’d live forever and get as much complex and varied fun as I want and nothing horribly bad would be happening), it’d be bad. It’s missing stuff.
(Besides, just pay me more than the 10 bucks Dave offered and I’m not pressing anything. I’m very pain-averse, so no need to go through such extremes. ;))
Shooting you would be a last resort. I like you humans, I don’t wanna kill you.
Been thinking more and noticed that I’m confused about how “terminal values” actually work.
It seems like my underlying model of preferences is eliminativist. (Relevant caricature.) Because the decision making process uses (projected and real) rewards to decide between actions, it is only these rewards that actually matter, not the patterns that triggered them. As such, there aren’t complex values and wireheading is a fairly obvious optimization.
To take the position of a self-modifying AI, I might look at my source code and
find the final decision making function that takes a list of possible actions
and their expected utility. It then returns the action with the maximum utility.
It is obvious to me that this function does not “care” about the actions, but
only about the utility. I might then be tempted to modify it such that, for
example, the list always contains a maximum utility dummy action (aka I wirehead
myself). This is clearly what this function “wants”.
But that’s not what “I” want. At the least, I should include the function that
rates the actions, too. Now I might modify it so that it simply rates every
action as optimal, but that’s taking the perspective of the function that picks
the action, not the one that rates it! The rating function actually cares about internal criteria (its terminal values) and circumventing this would be wrong.
The problem then becomes how to find out what those terminal values are and which of those to optimize for. (As humans are hypocritical and revealed preferences often match neither professed nor introspected preferences.) Picking the choosing function as an optimization target is much easier and always consistent.
I’m not confident that this view is right, but I can’t quite reduce preferences in any other consistent way. I checked the Neuroscience of Desire again, but I don’t see how you can extract caring about referents from that. In other words, it’s all just neurons firing. What these neurons optimize is being triggered, not some external state of the world. (Wireheading solution: let’s just trigger them directly.)
For now, I’m retracting my endorsement of wireheading until I have a better understanding of the issue. (I will also try to not blow up any world as I might still need it.)
I was set up to favor specific pathways to these rewards for reasons that are not my own (but those of my genes, memes or other influences).
I have to ask where your own reasons come from, causally speaking.
Good point. I can’t just disown all reasons or “I” become a rock, which doesn’t appeal to me, identity-wise. I like minimalist identities the most, so I retain pleasure = good, but not reproductive success, for example. In other words, I keep the basic mechanism that evolution gave me to achieve goals, I ignore the meta-goal of reproductive success it had.
I’m not happy with this argument, but I find extended versions that care about externals just as implausible. The choice between both seems arbitrary, so I go with the simpler one for now.
Also, not sure what happens to the value of suicide if you value only your subjective experience. Isn’t it undefined?
Yes. Death itself has fairly close to 0 utility to me, but I don’t like dying (because of the pain and shame it causes me, mostly), so I’m normally against suicide.
Can you try to elaborate on why you value external things?
As I said earlier, why wouldn’t I? I value non-green things.
Ok, fair. I can’t provide a better case for even why “pleasure” is good, but “pain” ain’t. It just feels that way to me. That’s just how the algorithm works. I’m just surprised that this difference in perceived values exists. If I further add MrMind’s stated values, either terminal value acquisition is fairly shaky and random in humans or easy to manipulate or very hard to introspect on, despite the appearance to the contrary.
A thought experiment. Imagine “reality” disappears suddenly and you wake up in Omega’s Simulation Chamber. Omega explains that all your life has been a simulation of the wallpaper kind. There weren’t any other minds, only ELIZA-style chatbots (but more sophisticated). Would this make you sad?
I don’t get a particularly bad response from that, maybe only slight disappointment because I was mistaken about the state of the world. I take that as weak evidence that I don’t care much about referents. But maybe I just have shitty relationships with people and nothing much to lose, so I’ll try improving in that regard first, to make that intuition more reliable. (That’s gotta take me some time.)
ETA:
The degradation effect I described seems fairly common. Lots of experiments in happiness studies show that set levels adjust ruthlessly.
New fun gets old. We want variety over time as well as space. Doesn’t affect complexity or external referents.
What about sustainability? What if we run out of interesting complexity?
Been thinking more and noticed that I’m confused about how “terminal values” actually work.
It seems like my underlying model of preferences is eliminativist. (Relevant caricature.) Because the decision making process uses (projected and real) rewards to decide between actions, it is only these rewards that actually matter, not the patterns that triggered them. As such, there aren’t complex values and wireheading is a fairly obvious optimization.
To take the position of a self-modifying AI, I might look at my source code and
find the final decision making function that takes a list of possible actions
and their expected utility. It then returns the action with the maximum utility.
It is obvious to me that this function does not “care” about the actions, but
only about the utility. I might then be tempted to modify it such that, for
example, the list always contains a maximum utility dummy action (aka I wirehead
myself). This is clearly what this function “wants”.
But that’s not what “I” want. At the least, I should include the function that
rates the actions, too. Now I might modify it so that it simply rates every
action as optimal, but that’s taking the perspective of the function that picks
the action, not the one that rates it! The rating function actually cares about internal criteria (its terminal values) and circumventing this would be wrong.
The problem then becomes how to find out what those terminal values are and which of those to optimize for. (As humans are hypocritical and revealed preferences often match neither professed nor introspected preferences.) Picking the choosing function as an optimization target is much easier and almost consistent.
I’m not confident that this view is right, but I can’t quite reduce preferences in any other consistent way. I checked the Neuroscience of Desire again, but I don’t see how you can extract caring about referents from that. In other words, it’s all just neurons firing. What these neurons optimize is being triggered, not some external state of the world. (Wireheading solution: let’s just trigger them directly.)
For now, I’m retracting my endorsement of wireheading until I have a better understanding of the issue. (I will also try to not blow up any world as I might still need it.)
I think we are approaching the core of the problem raised by my post. Maybe this thread warrants an entrirely new discussion post (I’ve still 14 karma point to put in the line ;)), if you and Muflax agree to start afresh.
Yeah, I think we’ve got each others’ positions here (MrMind wants to stay in meatspace, muflax wants to move to a holodeck, MixedNuts wants to move to an expensive simulation and also to know with so many handles start with ‘m’). That’s a better starting point for debate than bluntly stating one of your conclusions. Is there a way to co-write posts? (Also, what does muflax think? What about the rest of LW?)
Sure, no problem. Both of you can feel free to contact me via mail or Jabber (IRC, too.) if that helps, or just keep the discussion going.
I get the impression both of you are arguing essentially the Humans’ side in Three Worlds Collide, which is just outright insane to me. You are defending pain and boredom and challenge! Me, I’m SuperHappy all the way. I find the Baby Eaters easier to sympathize with than the Humans!
muflax wants to move to a holodeck
Nit-picking: I’m not fully endorsing a holodeck. In a laboratory metaphor, I want the cheese. Whether the best way to get there is through the labyrinth (your way) or around it (the holodeck) doesn’t matter to me. I suspect a bypass is easier, but I’m not sure. I’m qualifying my position so I don’t get forced into a more fanatic position through argument than the evidence warrants. :)
I’m defending boredom and challenge, undecided on pain so far. MrMind wants to retain a configuration that has those abilities but didn’t say these abilities mattered in themselves, though that’s a likely implication.
Well, obviously you’d rather fall in love with a real person than with wallpaper if the wallpaper is more expensive. That ain’t likely; a system to write arbitrary input to your brain is easier than the galaxy-spanning civilizations I want.
Nitpicking: I was not stating bluntly my conclusion, but my preferences.
Anyway, a three way three m post seems ok, if muflax also agree to move the discussion there. May I PM the draft to you? I don’t think there’s a way to coauthor a post.
If so, then that’s a pretty straightforward point that any FAI would respect
I’m merely suspicious of that, since no one still know what an FAI really looks like. You seem to tend to collate an FAI with agreeable, pleasant or fun super-singleton, but these are your heuristics. It’s obvious that a friendly AI will respect my wishes, but it’s not obvious that an AI possibly developed by the people in this forum will.
(Even though I tend to think it’s mistaken, or at least, awfully inefficient.)
And that’s exactly the reason why I made the post.
I’m merely suspicious of that, since no one still know what an FAI really looks like. You seem to tend to collate an FAI with agreeable, pleasant or fun super-singleton, but these are your heuristics. It’s obvious that a friendly AI will respect my wishes, but it’s not obvious that an AI possibly developed by the people in this forum will.
A FAI will respect your preferences (as far as it can). That’s the whole point of Friendliness. Of course, an UFAI might just ignore you. But then, asking it to spare you probably won’t help you, either. It would be an awful coincidence that we got an AI that will try to force people into orgasmium, but allow them to opt-out.
That was not my point. I was not appealing to some abstract “friendly AI in the sky” as a new kind of rationalist god.
My post was meant to address the people in the forum, a sort of warning that shouted: “Hey, lesswrongians! I saw you are in love with complexity and so on, but don’t forget that people might have radically different preferences. Indeed, these are mine. Don’t forget to include them in a friendly AI, if you happen to build one”.
Baseline: I think the fun theory that circulates the most in this forum, as per Yudkowsky sequence, is dangerously narrow. That could have potentially catastrophic consequences, and I wanted to prevent that.
The most commonly discussed FAI won’t have any (human) terminal values built in (except maybe for bootstrapping), but will examine humans to get a complete understanding of what humans actually value and then optimize that.
Therefore, whatever we humans now think about what might be fun or not is fairly irrelevant for the FAI simply because we don’t know what our values are. (That is also why I referenced the confusion between terminal and instrumental values in my first comment. I think that pretty much everyone talking about their “values” is only talking about instrumental values and that properly reduced terminal values are much, much simpler.)
Then I’m outright confused. Cooler existences likely contain very few of your listed pleasures, since they’re not the most intense or the most complex or the most social, and when they do, they’re not likely to be linked to the corresponding activities (sexual pleasure could be linked to conversation, taste could be linked to ideas). So you object to being cooler. Why?
I do not object that. I object that an eutopia is made solely of cooler existence, and I made formal request that I’m left the option to choose the uncooler one. That is not trivial, depending on the fun theory you use.
Would you object if (on top of making you cooler) we tied your responses to other situations? Say, looking up a simple fact would feel like scratching an itch, exchanging ideas would feel like sex, whatever environment you’re most often in would trigger your “this is home and normal and safe” response, manipulating fleets of nanobots would feel like carving, etc.
Yes, I would object to that too. To put it differently, I would object to everything that violates the integrity of the present correlation between my brain states and the external world (that includes the fact that my body is actually the mediator).
Been thinking more about your stated preferences. To untangle them more:
Do you insist on keeping simple pleasures because you fear you might lose them if cooler ones were available, i.e., increasing happiness set levels would eventually lower your total fun?
I once had a discussion with my mother in which I was optimistic about the increasing influence of machines and software in our lives and that in a near future we might spend most of our leisure uploaded in advanced MMOs. She objected that she valued gardening and similar menial tasks and that she did not like this direction. What I found out after some probing was that she objected to the aesthetic. “Machines” brought up fictional evidence of a very sterile future with heroin-addict-like wireheads being hooked up to computers.
So what I’m getting at, might that be a factor? Do you have a specific vision of what a complex future of the kind you are opposing might look like?
If I understand you right, you think that increasing fun leads to either wireheading and/or increasing complexity. (I generally agree.) I see how you can have a problem with wireheading. What’s your issue with complexity? Why would you want to not want to indulge in complex fun with the consequence of less interest in simple fun along the way?
In other words, wouldn’t you always want to have as much fun as possible? (With “fun” in the fun theory sense that includes all your terminal values, not just pleasure.) It seems to me like this should be true for any properly functioning agent, although agents might disagree on what fun is. But you seem to agree that you would actually enjoy more complex fun more.
Do you insist on keeping simple pleasures because you fear you might lose them if cooler ones were available, i.e., increasing happiness set levels would eventually lower your total fun?
No, my opinion is that a cooler existence would make them meaningless. It’s not a question of fun or happiness: in a eutopia those are cheap commodities. It is more a question of identity and futility.
So what I’m getting at, might that be a factor? Do you have a specific vision of what a complex future of the kind you are opposing might look like?
I feel it’s important to say that I’m not opposing a future like that. I like AIs and robots and think that we need more of that in our lives. What I’m opposing is that for some people, it’s unnecessary and unwanted to increase the complexity of existence per se. I simply don’t value terminally complexity, so for me an existence which is built on that is simply an existence which I don’t prefer.
In other words, wouldn’t you always want to have as much fun as possible? (With “fun” in the fun theory sense that includes all your terminal values, not just pleasure.) It seems to me like this should be true for any properly functioning agent, although agents might disagree on what fun is.
That, in essence, is the central ‘dogma’ of your theory of fun. I’m telling you however that for some people (me, for example), that is just not true. I just don’t want to have more and more fun, it strikes me as meaningless and ‘childish’ (that is not an exact description. I would need to dig deeper into the precise feeling).
I would like to add to your theory of fun that there are agents who, once a certain level of fun/happiness is reached, they just need no more and can continue happily forever in that state of mind.
I can understand “maxing out” fun. I even suspect that my ability to experience fun is bounded and that even without post-singularity tech I might maximize it. I wonder, what happens then? Once all your values are fulfilled (and sustainability is not an issue), what do you do?
(Obviously, self-modify to not get bored and enjoy the ride, says the wirehead. I’m not so sure about that anymore.)
Why would you ever choose the uncool option?! I don’t intend that to be a rude response, I just can’t wrap my head around it. Do you just feel that way? How do you know? Have you thought about that you might be confused, or might have some kind of status-quo bias?
(Not that I have a problem with your choice. I think a utopia should totally allow people to do things I find stupid, as long as it doesn’t cause any other harm.)
Have you thought about that you might be confused, or might have some kind of status-quo bias?
Even though it’s too strong to say that I would never choose the cooler existence, for what I understand of my present preferences, I’m not tempted at all about living in a simulation. By saying that I mean:
I’m aware there’s no difference between reality and a perfect simulation;
I don’t need particular complexity in my life, and most importantly, I don’t need more complexity than I have now;
I would choose simulation if it was a matter of survival, e.g. a gamma burst approaching Earth
if I were granted in a eutopia the state of happiness that I have in the present life, forever, that would be very ok for me.
(Not that I have a problem with your choice. I think a utopia should totally allow people to do things I find stupid, as long as it doesn’t cause any other harm.)
That is very cool from you: I promise that if I were to code a friendly AI, I will let it have people upload in a cooler existence, as long as they don’t cause any harm to the ‘meatballs’ ;)
Please elaborate?
You can value other things than actions themselves and the pleasure they bring. Maybe you like choosing and planning and acting. Maybe you like solving complex problems. Maybe you like variety (laughter and food and discovery and music, rather than the best out of four) and complexity. Maybe you like growing and learning. Maybe you like creating things. Maybe you like overcoming obstacles. Maybe you like exchanging ideas with other minds and bulding relationships and helping. Maybe you like intricate webs of feelings that aren’t all pleasant but are made more enjoyable because they’re bittersweet or frightening or weird.
So you’ll select activities that maximize that and your zillion other values, not just activities that maximize pleasure. But there’s no particular reason why current activities should be optimal. The pleasure of scratching an itch is unique, and somewhat complex. I wouldn’t trade it for, say, a more intense copy of the pleasure of eating a potato chip—I already have that one. But I would trade it for a different, more intense, more complex pleasure.
Which is my thesis: outside of denying change, the only two dimension under which is significant to find an improvement are intensity and complexity. However your position appears to me not to be reflectively consistent: after all, you could consider the complexity and intensity of your whole human experience and be seduced by a different, more complex, more intense type of existence.
As I parse it, yes! It’d have to be done by gradual change (I don’t want to die and be replaced by someone cooler), but yeah, that’s what I want. Where am I inconsistent?
Thank Aumann, we agree! I said it was inconsistent not to be seduced by the cooler existence :)
Hmm, don’t we disagree about connotations? You seem to think “orgasmium” covers that kind of being that has varied, complex, intense fun, and does things instead of just feeling like it does them. Do you?
No I don’t. My reply to muflax was not related from this one. I don’t have a name for this state of affair, it doesn’t even seem to have an official name. Cooler existence is ok for me
So what is it you are asking for, then? Merely that you don’t just get the fun, but the things that cause the fun as well?
If so, then that’s a pretty straightforward point that any FAI would respect. (Even though I tend to think it’s mistaken, or at least, awfully inefficient.)
I’m curious. You’re alone in the universe, will always be. After centuries of hard work, you’ve finished your Theory of Everything and Then Some. Your mind strains under the ecstasy of your discovery. You’re now in a room with two buttons. Button A resets you to your mindstate thirty seconds ago, lets you experience those thirty seconds of extreme bliss, then repeats, for five minutes. Button B entertains you for five minutes with mildly amusing videos of cats dressed as anime characters. Which button do you press?
A, obviously. Why?
(I consider both options good, but there might be a better C.)
You are not resolving my curiosity! (Well, you’re telling me I haven’t completely misunderstood your position, which is something.) I’d like to know your reasons for your reasons for choosing A (your reasons for choosing A are as you say obvious).
My intuitions say that A is wrong on the same level that replacing all kinds of fun by a big orgasm is sad. This is connected to the intuition that overcoming things is good (and you can’t cheat by tying the “Behold! I, human, have vanquished this!” emotion to eating a potato chip), and that choosing and planning and acting are fun. I’m not done unpacking them.
(Come to think of it, I’m not sure I press B, though I probably do. But if B gives me a small, pleasant, new insight (say, a new way to prove Pythagoras’s theorem) instead, I press it.)
I choose A because A is the most fun thing to do. I find it really hard to get any deeper reasons than that. I’ll try to elaborate anyway and hope it doesn’t get too messy. It would be really weird for humans to have that large a value dissonance, so there’s probably some confusion around here somewhere.
A couple of months ago, I even would’ve agreed with you and would’ve opposed straight orgasmium. But I just can’t find any argument against it. I tried tracing my enjoyment of activities, but I couldn’t find nuanced, complex fun there. It’s all composed of few (probably <5) components. I enjoy solving complex problems, but that’s because it’s one of the few ways I know to trigger the sensation of enjoying-to-learn. It’s the only reason I even bother with programming anymore. If I could trigger the same learning experience by reading a dictionary, I’d get back to studying Latin. It seems very much possible to just straight maximize all components and reach perfect orgasmium. (I strongly suspect they are all bounded, as I already have found my personal Best Movie[1] and Best Song[2] and have for years not been able to improve one bit on that enjoyment, only match it.)
If I could self-modify to eat chips like Light (Death Note spoiler), I totally would. Why is that sad or cheating? Who are you wronging? What code are you breaking? If it’s a way to maximize fun, then so be it. It doesn’t match common tropes, it doesn’t have much status and you don’t look clever for doing it, but if it works, then that’s that. Not my fault that the universe has obvious cheat codes built in. Screw the dungeon, just get the cheese already.
I strongly suspect that I don’t actually care that my values are fulfilled outside of my experience. I see no reason why anyone would. The only explanation that even begins to make sense to me is memetic hijacking, as Tim Tyler often proposes. Claiming to oppose hedonistic optimization might easily be evolutionary beneficial. It looks similar to normal altruistic trade-offs. (Of course this doesn’t discredit these more complicated values. A rational agent is free to value whatever it wants, regardless how it got these values. But it would explain why some agents have them and others don’t.)
(I’m assuming the least convenient world where there aren’t any choices around that are more enjoyable than A, where being locked in a loop of pressing A doesn’t prevent me from fulfilling some other desires and so on. As I already have a Theory of Everything and am alone, I can’t off the top of my head think of any value that would be left out anyway.)
[1] Moulin Rouge, though Wanted) and Sky Captain and the World of Tomorrow and others are equally good
[2] 嘘とワンダーランド - Asian Kung-Fu Generation (I listened to it literally ~4000 times in a row when I found it)
You like G. R. R. Martin and you don’t have complex fun?! You said you have depression, this is probably linked (boo for flat affect). I do have complex fun. Like, just yesterday I ate a peach and a pastry and compared everything about the texture and the taste and the temperature, had an uncontrollable urge to run my fingers over every surface in the house, and thought about reducing qualia. (And, uh, I already eat crisps like Light, but like him I would enjoy it more if it were part of an intricate world-conquering plot.)
Clippy would say “If it’s a way to maximize paperclips, then so be it. ” I don’t care only about fun, I care about a zillion things only a few of which fall under the usual concept of fun. I want the dungeon. You don’t go in the cave of trials for the items, you go for the challenge and the made-more-valuable-by-the-challenge items. I don’t want to stare at a “You win” screen forever.
And I don’t actually know why! All I can say is “Becoming orgasmium? That sounds pointless. I’m not going to renounce complexity and variety to make my value system more elegant, it’s just not worth it”.
I do know why I value things outside my own experience. (Will shoot you and get tortured for it rather than let you press the blow-up-the-world-when-we’re-both-dead button, for example.) There’s no reason optimization processes should optimize their input sequences only. Clippy doesn’t want to overestimate the number of paperclips in the world.
GRRM is one of the few authors who writes fiction that doesn’t insult my intelligence. It’s not that I prefer his work to, say, Naruto. It’s just that his is the only I can enjoy at all. I remember reading much more trash in my childhood and found them equally enjoyable. If I try reading them now, I just keep on rolling my eyes and raging.
I don’t think so. I don’t feel depressed right now (luckily, for months). I sometimes did have a fairly flat affect; that ain’t now. The degradation effect I described seems fairly common. Lots of experiments in happiness studies show that set levels adjust ruthlessly. Also, Daoists and Buddhists, among many others, have advocated simplicity in favor of complexity and it seems to work.
(Which is the reason I intentionally don’t learn to cook so I can still enjoy really cheap and simple food. I don’t want to fall into the same trap my obsession with fiction has gotten me into.)
Yes, and Clippy would be right. So? I don’t see how this is a counter-argument. This reminds me of AIXI agents wireheading themselves. I don’t think that’s a failure mode. It’s a perfect valid action and if anything shows that they actually are clever. Of course it sucks for the programmers who wanted to get other things done. Similarly, evolution might’ve taken care to prevent this happening in humans for the same reasons, although it failed in my case. (Pure speculation.)
Why? Honest question, I just get “404” if I ask myself that.
My perspective is that there are rewards and actions to get these rewards. I think that the rewards are the important thing. They are not instruments to get to my “real” values, they are my values. I was set up to favor specific pathways to these rewards for reasons that are not my own (but those of my genes, memes or other influences). I, too, sometimes feel bad for bypassing them, but that is a very obvious security feature and I’m much more willing to consider this aversion to be in error than the bypass.
For example, D&D has many ways to get Infinite Wish Spells. They completely break the game and prevent all the intended benefits of the game, so it’s a bad idea for the players to use them as shortcuts. It’s much better to go through long, dangerous dungeons than to just ignore everyone else and ascend to godhood and take over the whole universe. But that’s because the one who is making the decisions is the player, who wants to socialize and boast and so on. If you lived in D&D, this intuition, this aversion to do things that weren’t “intended”, would not serve you well. Now you should go for the game-breaker.
The most obvious anti-hedonism feature is boredom. I agree with you that
But I consider this boredom to be a bad feature! I wish I could just stare at “you win” instead of having to jump through all the hoops every time, in increasing difficulty, with dwindling supply, as has already happened. I’ve watched >90% of every anime even remotely interesting to me and can’t rarely get any real enjoyment out of them anymore because this stupid boredom is adjusting my happiness set level down. I’d much prefer to just get The Perfect Entertainment and have it stay that way, forever. Simplicity and elegance are good things.
I notice that this discussion makes me feel resigned and sad, so I will adjust down my confidence that this is right. Can you try to elaborate on why you value external things? Why you think a holodeck is bad, except for the emotions that come up?
(Besides, just pay me more than the 10 bucks Dave offered and I’m not pressing anything. I’m very pain-averse, so no need to go through such extremes. ;))
New fun gets old. We want variety over time as well as space. Doesn’t affect complexity or external referents.
I meant that fun-maximizers don’t have more power to move me than paperclips-maximizers.
Just that there’s no particular reason for my values to be only about my experience, like there’s no reason for them to be only about parts of the universe that are green.
Yeah, I understand that. I don’t know why I disagree. I value the reward, but I also value many things about the action.
As an example: in Judaism, you want to set obstacles for yourself and to overcome them (there even is a bunch of rules to prevent this from going overboard). There are vegan Orthodow Jews who complain that keeping kosher is too easy for them and who decide some vegetables count as milk and others as meat and they don’t mix them. Chosen fetters like that appeal to me like mad. It’s a kind of fun, but, I expect, one completely alien to you.
I have to ask where your own reasons come from, causally speaking.
Agree we should break the rules to get what we want, disagree about what we want.
So you want classical orgasmium, not complex fun with no referent (like falling in love with a chatbot). Glad that’s clear.
Yes. But not so good that I’d renounce humor and challenge and pride and aesthetics and freedom and truth and more. Same reason that I can’t decide to burn down an orphanage—I care about making a point about choosing my own values and relativity of morality, but I care about human life more.
I’ve been adjusting in the opposite direction as well. Truly we are an Aumann cage match made in heaven.
As I said earlier, why wouldn’t I? I value non-green things.
My brain sucks. It can’t represent the joys and great deeds and wisdom in a single human life (except the one it’s being). Unless other people are clever chatbots, there are more great things in a day of their lives than in my brain in its highest bliss. It sounds just odd that this should be worthless. (Also phrased as: Every morning, I weigh myself. If it’s less than Earth, I save the world.)
Also, not sure what happens to the value of suicide if you value only your subjective experience. Isn’t it undefined?
Not directly related, but values are over 4D, not 3D. (Which is why it’s not completely stupid to care about paths, not just endpoints.)
Same reason I think that if I could never have anything more pleasant than ice cream (but I’d live forever and get as much complex and varied fun as I want and nothing horribly bad would be happening), it’d be bad. It’s missing stuff.
Shooting you would be a last resort. I like you humans, I don’t wanna kill you.
Been thinking more and noticed that I’m confused about how “terminal values” actually work.
It seems like my underlying model of preferences is eliminativist. (Relevant caricature.) Because the decision making process uses (projected and real) rewards to decide between actions, it is only these rewards that actually matter, not the patterns that triggered them. As such, there aren’t complex values and wireheading is a fairly obvious optimization.
To take the position of a self-modifying AI, I might look at my source code and find the final decision making function that takes a list of possible actions and their expected utility. It then returns the action with the maximum utility. It is obvious to me that this function does not “care” about the actions, but only about the utility. I might then be tempted to modify it such that, for example, the list always contains a maximum utility dummy action (aka I wirehead myself). This is clearly what this function “wants”.
But that’s not what “I” want. At the least, I should include the function that rates the actions, too. Now I might modify it so that it simply rates every action as optimal, but that’s taking the perspective of the function that picks the action, not the one that rates it! The rating function actually cares about internal criteria (its terminal values) and circumventing this would be wrong.
The problem then becomes how to find out what those terminal values are and which of those to optimize for. (As humans are hypocritical and revealed preferences often match neither professed nor introspected preferences.) Picking the choosing function as an optimization target is much easier and always consistent.
I’m not confident that this view is right, but I can’t quite reduce preferences in any other consistent way. I checked the Neuroscience of Desire again, but I don’t see how you can extract caring about referents from that. In other words, it’s all just neurons firing. What these neurons optimize is being triggered, not some external state of the world. (Wireheading solution: let’s just trigger them directly.)
For now, I’m retracting my endorsement of wireheading until I have a better understanding of the issue. (I will also try to not blow up any world as I might still need it.)
Good point. I can’t just disown all reasons or “I” become a rock, which doesn’t appeal to me, identity-wise. I like minimalist identities the most, so I retain pleasure = good, but not reproductive success, for example. In other words, I keep the basic mechanism that evolution gave me to achieve goals, I ignore the meta-goal of reproductive success it had.
I’m not happy with this argument, but I find extended versions that care about externals just as implausible. The choice between both seems arbitrary, so I go with the simpler one for now.
Yes. Death itself has fairly close to 0 utility to me, but I don’t like dying (because of the pain and shame it causes me, mostly), so I’m normally against suicide.
Ok, fair. I can’t provide a better case for even why “pleasure” is good, but “pain” ain’t. It just feels that way to me. That’s just how the algorithm works. I’m just surprised that this difference in perceived values exists. If I further add MrMind’s stated values, either terminal value acquisition is fairly shaky and random in humans or easy to manipulate or very hard to introspect on, despite the appearance to the contrary.
A thought experiment. Imagine “reality” disappears suddenly and you wake up in Omega’s Simulation Chamber. Omega explains that all your life has been a simulation of the wallpaper kind. There weren’t any other minds, only ELIZA-style chatbots (but more sophisticated). Would this make you sad?
I don’t get a particularly bad response from that, maybe only slight disappointment because I was mistaken about the state of the world. I take that as weak evidence that I don’t care much about referents. But maybe I just have shitty relationships with people and nothing much to lose, so I’ll try improving in that regard first, to make that intuition more reliable. (That’s gotta take me some time.)
ETA:
What about sustainability? What if we run out of interesting complexity?
Been thinking more and noticed that I’m confused about how “terminal values” actually work.
It seems like my underlying model of preferences is eliminativist. (Relevant caricature.) Because the decision making process uses (projected and real) rewards to decide between actions, it is only these rewards that actually matter, not the patterns that triggered them. As such, there aren’t complex values and wireheading is a fairly obvious optimization.
To take the position of a self-modifying AI, I might look at my source code and find the final decision making function that takes a list of possible actions and their expected utility. It then returns the action with the maximum utility. It is obvious to me that this function does not “care” about the actions, but only about the utility. I might then be tempted to modify it such that, for example, the list always contains a maximum utility dummy action (aka I wirehead myself). This is clearly what this function “wants”.
But that’s not what “I” want. At the least, I should include the function that rates the actions, too. Now I might modify it so that it simply rates every action as optimal, but that’s taking the perspective of the function that picks the action, not the one that rates it! The rating function actually cares about internal criteria (its terminal values) and circumventing this would be wrong.
The problem then becomes how to find out what those terminal values are and which of those to optimize for. (As humans are hypocritical and revealed preferences often match neither professed nor introspected preferences.) Picking the choosing function as an optimization target is much easier and almost consistent.
I’m not confident that this view is right, but I can’t quite reduce preferences in any other consistent way. I checked the Neuroscience of Desire again, but I don’t see how you can extract caring about referents from that. In other words, it’s all just neurons firing. What these neurons optimize is being triggered, not some external state of the world. (Wireheading solution: let’s just trigger them directly.)
For now, I’m retracting my endorsement of wireheading until I have a better understanding of the issue. (I will also try to not blow up any world as I might still need it.)
I think we are approaching the core of the problem raised by my post. Maybe this thread warrants an entrirely new discussion post (I’ve still 14 karma point to put in the line ;)), if you and Muflax agree to start afresh.
Yeah, I think we’ve got each others’ positions here (MrMind wants to stay in meatspace, muflax wants to move to a holodeck, MixedNuts wants to move to an expensive simulation and also to know with so many handles start with ‘m’). That’s a better starting point for debate than bluntly stating one of your conclusions. Is there a way to co-write posts? (Also, what does muflax think? What about the rest of LW?)
Sure, no problem. Both of you can feel free to contact me via mail or Jabber (IRC, too.) if that helps, or just keep the discussion going.
I get the impression both of you are arguing essentially the Humans’ side in Three Worlds Collide, which is just outright insane to me. You are defending pain and boredom and challenge! Me, I’m SuperHappy all the way. I find the Baby Eaters easier to sympathize with than the Humans!
Nit-picking: I’m not fully endorsing a holodeck. In a laboratory metaphor, I want the cheese. Whether the best way to get there is through the labyrinth (your way) or around it (the holodeck) doesn’t matter to me. I suspect a bypass is easier, but I’m not sure. I’m qualifying my position so I don’t get forced into a more fanatic position through argument than the evidence warrants. :)
I’m defending boredom and challenge, undecided on pain so far. MrMind wants to retain a configuration that has those abilities but didn’t say these abilities mattered in themselves, though that’s a likely implication.
Well, obviously you’d rather fall in love with a real person than with wallpaper if the wallpaper is more expensive. That ain’t likely; a system to write arbitrary input to your brain is easier than the galaxy-spanning civilizations I want.
Nitpicking: I was not stating bluntly my conclusion, but my preferences.
Anyway, a three way three m post seems ok, if muflax also agree to move the discussion there. May I PM the draft to you? I don’t think there’s a way to coauthor a post.
Okay. Having a real-time chat and posting the transcript would work but I can’t now.
I’m merely suspicious of that, since no one still know what an FAI really looks like. You seem to tend to collate an FAI with agreeable, pleasant or fun super-singleton, but these are your heuristics. It’s obvious that a friendly AI will respect my wishes, but it’s not obvious that an AI possibly developed by the people in this forum will.
And that’s exactly the reason why I made the post.
A FAI will respect your preferences (as far as it can). That’s the whole point of Friendliness. Of course, an UFAI might just ignore you. But then, asking it to spare you probably won’t help you, either. It would be an awful coincidence that we got an AI that will try to force people into orgasmium, but allow them to opt-out.
That was not my point. I was not appealing to some abstract “friendly AI in the sky” as a new kind of rationalist god.
My post was meant to address the people in the forum, a sort of warning that shouted: “Hey, lesswrongians! I saw you are in love with complexity and so on, but don’t forget that people might have radically different preferences. Indeed, these are mine. Don’t forget to include them in a friendly AI, if you happen to build one”.
Baseline: I think the fun theory that circulates the most in this forum, as per Yudkowsky sequence, is dangerously narrow. That could have potentially catastrophic consequences, and I wanted to prevent that.
The most commonly discussed FAI won’t have any (human) terminal values built in (except maybe for bootstrapping), but will examine humans to get a complete understanding of what humans actually value and then optimize that.
Therefore, whatever we humans now think about what might be fun or not is fairly irrelevant for the FAI simply because we don’t know what our values are. (That is also why I referenced the confusion between terminal and instrumental values in my first comment. I think that pretty much everyone talking about their “values” is only talking about instrumental values and that properly reduced terminal values are much, much simpler.)
Then I’m outright confused. Cooler existences likely contain very few of your listed pleasures, since they’re not the most intense or the most complex or the most social, and when they do, they’re not likely to be linked to the corresponding activities (sexual pleasure could be linked to conversation, taste could be linked to ideas). So you object to being cooler. Why?
I do not object that. I object that an eutopia is made solely of cooler existence, and I made formal request that I’m left the option to choose the uncooler one. That is not trivial, depending on the fun theory you use.
Would you object if (on top of making you cooler) we tied your responses to other situations? Say, looking up a simple fact would feel like scratching an itch, exchanging ideas would feel like sex, whatever environment you’re most often in would trigger your “this is home and normal and safe” response, manipulating fleets of nanobots would feel like carving, etc.
Yes, I would object to that too. To put it differently, I would object to everything that violates the integrity of the present correlation between my brain states and the external world (that includes the fact that my body is actually the mediator).
Been thinking more about your stated preferences. To untangle them more:
Do you insist on keeping simple pleasures because you fear you might lose them if cooler ones were available, i.e., increasing happiness set levels would eventually lower your total fun?
I once had a discussion with my mother in which I was optimistic about the increasing influence of machines and software in our lives and that in a near future we might spend most of our leisure uploaded in advanced MMOs. She objected that she valued gardening and similar menial tasks and that she did not like this direction. What I found out after some probing was that she objected to the aesthetic. “Machines” brought up fictional evidence of a very sterile future with heroin-addict-like wireheads being hooked up to computers.
So what I’m getting at, might that be a factor? Do you have a specific vision of what a complex future of the kind you are opposing might look like?
If I understand you right, you think that increasing fun leads to either wireheading and/or increasing complexity. (I generally agree.) I see how you can have a problem with wireheading. What’s your issue with complexity? Why would you want to not want to indulge in complex fun with the consequence of less interest in simple fun along the way?
In other words, wouldn’t you always want to have as much fun as possible? (With “fun” in the fun theory sense that includes all your terminal values, not just pleasure.) It seems to me like this should be true for any properly functioning agent, although agents might disagree on what fun is. But you seem to agree that you would actually enjoy more complex fun more.
No, my opinion is that a cooler existence would make them meaningless. It’s not a question of fun or happiness: in a eutopia those are cheap commodities. It is more a question of identity and futility.
I feel it’s important to say that I’m not opposing a future like that. I like AIs and robots and think that we need more of that in our lives. What I’m opposing is that for some people, it’s unnecessary and unwanted to increase the complexity of existence per se. I simply don’t value terminally complexity, so for me an existence which is built on that is simply an existence which I don’t prefer.
That, in essence, is the central ‘dogma’ of your theory of fun. I’m telling you however that for some people (me, for example), that is just not true. I just don’t want to have more and more fun, it strikes me as meaningless and ‘childish’ (that is not an exact description. I would need to dig deeper into the precise feeling). I would like to add to your theory of fun that there are agents who, once a certain level of fun/happiness is reached, they just need no more and can continue happily forever in that state of mind.
Thanks, now I understand you much better.
I can understand “maxing out” fun. I even suspect that my ability to experience fun is bounded and that even without post-singularity tech I might maximize it. I wonder, what happens then? Once all your values are fulfilled (and sustainability is not an issue), what do you do?
(Obviously, self-modify to not get bored and enjoy the ride, says the wirehead. I’m not so sure about that anymore.)
Why would you ever choose the uncool option?! I don’t intend that to be a rude response, I just can’t wrap my head around it. Do you just feel that way? How do you know? Have you thought about that you might be confused, or might have some kind of status-quo bias?
(Not that I have a problem with your choice. I think a utopia should totally allow people to do things I find stupid, as long as it doesn’t cause any other harm.)
Even though it’s too strong to say that I would never choose the cooler existence, for what I understand of my present preferences, I’m not tempted at all about living in a simulation. By saying that I mean:
I’m aware there’s no difference between reality and a perfect simulation;
I don’t need particular complexity in my life, and most importantly, I don’t need more complexity than I have now;
I would choose simulation if it was a matter of survival, e.g. a gamma burst approaching Earth
if I were granted in a eutopia the state of happiness that I have in the present life, forever, that would be very ok for me.
That is very cool from you: I promise that if I were to code a friendly AI, I will let it have people upload in a cooler existence, as long as they don’t cause any harm to the ‘meatballs’ ;)