Why No Wireheading?
I’ve been thinking about wireheading and the nature of my values. Many people here have defended the importance of external referents or complex desires. My problem is, I can’t understand these claims at all.
To clarify, I mean wireheading in the strict “collapsing into orgasmium” sense. A successful implementation would identify all the reward circuitry and directly stimulate it, or do something equivalent. It would essentially be a vastly improved heroin. A good argument for either keeping complex values (e.g. by requiring at least a personal matrix) or external referents (e.g. by showing that a simulation can never suffice) would work for me.
Also, I use “reward” as short-hand for any enjoyable feeling, as “pleasure” tends to be used for a specific one of them, among bliss, excitement and so on, and “it’s not about feeling X, but X and Y” is still wireheading after all.
I tried collecting all related arguments I could find. (Roughly sorted from weak to very weak, as I understand them, plus link to example instances. I also searched any literature/other sites I could think of, but didn’t find other (not blatantly incoherent) arguments.)
People do not always optimize their actions based on achieving rewards. (People also are horrible at making predictions and great at rationalizing their failures afterwards.)
It is possible to enjoy doing something while wanting to stop or vice versa, do something without enjoying it while wanting to continue. (Seriously? I can’t remember ever doing either. What makes you think that the action is thus valid, and you aren’t just making mistaken predictions about rewards or are being exploited? Also, Mind Projection Fallacy.)
A wireheaded “me” wouldn’t be “me” anymore. (What’s this “self” you’re talking about? Why does it matter that it’s preserved?)
“I don’t want it and that’s that.” (Why? What’s this “wanting” you do? How do you know what you “want”? (see end of post))
People, if given a hypothetical offer of being wireheaded, tend to refuse. (The exact result depends heavily on the exact question being asked. There are many biases at work here and we normally know better than to trust the majority intuition, so why should we trust it here?)
Far-mode predictions tend to favor complex, external actions, while near-mode predictions are simpler, more hedonistic. Our true self is the far one, not the near one. (Why? The opposite is equally plausible. Or the falsehood of the near/far model in general.)
If we imagine a wireheaded future, it feels like something is missing or like we won’t really be happy. (Intuition pump.)
It is not socially acceptable to embrace wireheading. (So what? Also, depends on the phrasing and society in question.)
(There have also been technical arguments against specific implementations of wireheading. I’m not concerned with those, as long as they don’t show impossibility.)
Overall, none of this sounds remotely plausible to me. Most of it is outright question-begging or relies on intuition pumps that don’t even work for me.
It confuses me that others might be convinced by arguments of this sort, so it seems likely that I have a fundamental misunderstanding or there are implicit assumptions I don’t see. I fear that I have a large inferential gap here, so please be explicit and assume I’m a Martian. I genuinely feel like Gamma in A Much Better Life.
To me, all this talk about “valueing something” sounds like someone talking about “feeling the presence of the Holy Ghost”. I don’t mean this in a derogatory way, but the pattern “sense something funny, therefore some very specific and otherwise unsupported claim” matches. How do you know it’s not just, you know, indigestion?
What is this “valuing”? How do you know that something is a “value”, terminal or not? How do you know what it’s about? How would you know if you were mistaken? What about unconscious hypocrisy or confabulation? Where do these “values” come from (i.e. what process creates them)? Overall, it sounds to me like people are confusing their feelings about (predicted) states of the world with caring about states directly.
To me, it seems like it’s all about anticipating and achieving rewards (and avoiding punishments, but for the sake of the wireheading argument, it’s equivalent). I make predicitions about what actions will trigger rewards (or instrumentally help me pursue those actions) and then engage in them. If my prediction was wrong, I drop the activity and try something else. If I “wanted” something, but getting it didn’t trigger a rewarding feeling, I wouldn’t take that as evidence that I “value” the activity for its own sake. I’d assume I suck at predicting or was ripped off.
Can someone give a reason why wireheading would be bad?
- Just Look At The Thing! – How The Science of Consciousness Informs Ethics by (EA Forum; 23 Sep 2022 6:20 UTC; 75 points)
- A definition of wireheading by (27 Nov 2012 19:31 UTC; 52 points)
- Much-Better-Life Simulator™ - Sales Conversation by (19 Jun 2011 12:44 UTC; 10 points)
- 's comment on Stupid Questions September 2015 by (3 Sep 2015 0:27 UTC; 0 points)
But aren’t you just setting up a system that values states of the world based on the feelings they contain? How does that make any more sense?
You’re arguing as though neurological reward maximization is the obvious goal to fall back to if other goals aren’t specified coherently. But people have filled in that blank with all sorts of things. “Nothing matters, so let’s do X” goes in all sorts of zany directions.
I’m not. My thought process isn’t “there aren’t any real values, so let’s go with rewards”; it’s not intended as a hack to fix value nihilism.
Rewards already do matter. It describes people’s behavior well (see PCT) and makes introspective sense. I can actually feel projected and real rewards come up and how decisions arise based on that. I don’t know how “I value that there are many sentients” or any other external referent could come up. It would still be judged on the emotional reaction it causes (but not always in a fully conscious manner).
I think I can imagine agents that actually care about external referents and that wouldn’t wirehead. I just don’t think humans are such agents and I don’t see evidence to the contrary. For example, many humans have no problem with “fake” experiences, like “railroaded, specifically crafted puzzles to stimulate learning” (e.g. Portal 2), “insights that feel profound, but don’t mean anything” (e.g. entheogens) and so on. Pretty much the whole entertainment industry could be called wireheading lite.
Acting based on the feelings one will experience is something that already happens, so optimizing for it is sensible. (Not-wireheaded utopias would also optimize them after all, just not only them.)
A major problem I see with acting based on propositions about the world outside one’s mind is that it would assign different value to states that one can’t experimentally distinguish (successful mindless wallpaper vs. actual sentients, any decision after being memory-wiped, etc.). I can always tell if I’m wireheaded, however. I’d invoke Occam’s Razor here and ignore any proposal that generates no anticipated experiences.
I can’t really pick apart your logic here, because there isn’t any. This is like saying “buying cheese is something that already happens, so optimizing for it is sensible”
Not really. Let me try to clarify what I meant.
We already know that rewards and punishments influence our actions. Any utopia would try to satisfy them. Even in a complex optimized universe full of un-wireheaded sentients caring about external referents, people would want to avoid pain, … and experience lots of excitement, … . Wireheading just says, that’s all humans care about, so there’s no need for all these constraints, let’s pick the obvious shortcut.
In support of this view, I gave the example of the entertainment industry that optimizes said experiences, but is completely fake (and trying to become more fake) and how many humans react positively to that. They don’t complain that there’s something missing, but rather enjoy those improved experiences more than the existent externally referenced alternatives.
Also, take the reversed experience machine, in which the majority of students asked would stay plugged in. If they had complex preferences as typically cited against wireheading, wouldn’t they have immediately rejected it? An expected paperclip maximizer would have left the machine right away. It can’t build any paperclips there, so the machine has no value to it. But the reversed experience machine seems to have plenty of value for humans.
This is essentially an outside view argument against complex preferences. What’s the evidence that they actually exist? That people care about reality, about referents, all that? When presented with options that don’t fulfill any of this, lots of people still seem to choose them.
So, when people pick chocolate, it illustrates that that’s what they truly desire, and when they pick vanilla, it just means that they’re confused and really they like chocolate but they don’t know it.
Absolutely. Sudoku has been described as “a denial of service attack on human intellect”, and see also the seventh quote here.
PCT is not good to cite in this connection. PCT does not speak of rewards. According to PCT, behaviour is performed in order to control perceptions, i.e. to maintain those perceptions at their reference levels.
While it is possible for a control system to be organised around maximising something labelled a reward (or minimising something labelled a penalty), that is just one particular class of possible ways of making a control system. Unless one has specifically observed that organisation, there are no grounds for concluding that reward is involved just because something is made of control systems.
Good point, I oversimplified here. I will consider this in more detail, but it naively, isn’t this irrelevant in terms of wireheading? Maintaining perceptions is maybe a bit trickier to do, but there would still be obvious shortcuts. Maybe if these perceptions couldn’t be simplified in any relevant way, then we’d need at least a full-on matrix and that would disqualify wireheading.
Is this just a case of the utility function not being up for grabs? muflax can’t explain to me why wireheading counts as a win, and I can’t explain to muflax why wireheading doesn’t count as a win for me. At least, not using the language of rationality.
It might be interesting to get a neurological or evo-psych explanation for why non-wireheaders exist. But I don’t think this is what’s being asked here.
Well, ultimately it might be, but it really weirds me out. We’re all running on essentially the same hardware. I don’t think either those that find wireheading intuitive or those that don’t are that non-neurotypical. I would expect that wireheading is right for either all or no humans and any other result needs a really good explanation.
I’m not explicitly asking it, but I would be very interested in why it seems like there are two different kinds of minds, yes.
This is just my opinion, not particularly evidence-based: I don’t think that there are two different kinds of mind, or if there are it’s not this issue that separates them. The wireheading scenario is one which is very alien to our ancestral environment so we may not have an “instinctive” preference for or against it. Rather, we have to extrapolate that preference from other things.
Two heuristics which might be relevant:
where “wanting” and “liking” conflict, it feels like “wanting” is broken (i.e. we’re making ourselves do things we don’t enjoy). So given the opportunity we might want to update what we “want”. This is pro-wireheading.
where we feel we are being manipulated, we want to fight that manipulation in case it’s against our own interests. Thinking about brain probes is a sort of manipulation-superstimulus, so this heuristic would be anti-wireheading.
I can very well believe that wireheading correlates with personality type, which is a weak form of your “two different minds” hypothesis.
Sorry for the ultra-speculative nature of this post.
Makes sense in terms of explaining the different intuition, yes, and is essentially how I think about it.
The second heuristic about manipulation, then, seems useful in practice (more agents will try to exploit us than satisfy us), but isn’t it much weaker, considering the actual wireheading scenario? The first heuristic actually addresses the conflict (although maybe the wrong way), but the second just ignores it.
I agree; the second heuristic doesn’t apply particularly well to this scenario. Some terminal values seem to come from a part of the brain which isn’t open to introspection, so I’d expect them to arise as a result of evolutionary kludges and random cultural influences rather than necessarily making any logical sense.
The thing is, once we have a value system that’s reasonably stable (i.e. what we want is the same as what we want to want) then we don’t want to change our preferences even if we can’t explain where they arise from.
As you know, we’ve already seen this statement, with “wireheading”, with “increased complexity”, etc.
Until we get a definition of meta-value and their general axiological treatment, people will always be baffled that others have different meta-values then theirs.
Think about a paper-clip maximiser (people tend get silly about morality, and a lot less silly about paper-clips so its a useful thought experiment for meta-ethics in general). Its a simple design, it lists all the courses of action it could take, computes the expected_paper-clips given each one using its model of the world, and then takes the one that gives the largest result. It isn’t interested in the question of why paper-clips are valuable, it just produces them.
So, does it value paper-clips, or does it just value expected paper-clips?
Consider how it reacts to the option “update your current model of the world to set Expected paper-clips = BB(1000)”. This will appear on its list of possible actions, so what is its value?
(expected paperclips | “update your current model of the world to set Expected paper-clips = BB(1000)”)
The answer is a lot less than BB(1000). Its current model of the world states that updating its model does not actually change reality (except insofar as the model is part of reality). Thus it does not predict that this action will result in the creation of any new paper-clips, so its expected paper-clips is roughly equal to the number of paper-clips that get produced anyway.
Expected expected paper-clips given this action is very large, but the paper-clipper doesn’t give a rat’s arse about that.
Hopefully, I have convinced you that that there is a difference between caring about some aspect of the world and using your internal model to predict that aspect, versus caring about your internal model. Furthermore, in the space of all possible minds the vast majority are in the first category, since an agent’s own mind is generally only a tiny portion of the world, so if humans value both then it is the internal part that makes us unusual.
I can’t make you value something any more than I can make a rock value it, the best I can do is convince you that you are allowed to value non-wireheading, and if you don’t feel like you want it then it is privileging the hypothesis to even consider the possibility that you do.
The question is if humans, unlike paperclip maximizer’s, are actually more concerned with maximizing their reward number irregardless of how it is being increased.
If there is a way for humans to assign utility non-arbitrarily, then we are able to apply rational choice to our values, i.e. look for values that are better at yielding utility. If humans measure utility in a unit of bodily sensations, then we can ask what would most effectively yield the greatest amount of bodily sensations. Here wireheading seems to be more efficient than any other way to maximize bodily sensations, i.e. utility.
There even is some evidence for this, e.g. humans enjoy fiction. Humans treat their model of reality as part of reality. If you can change the model, you can change reality.
I don’t agree with all that though, because I think that humans either are not utility maximizer’s or assign utility arbitrarily.
It seems to me that I value both my internal world and the external world. I enjoy fiction, but the prospect of spending the rest of my life with nothing else fails to thrill me.
A lot of people express scepticism of this claim, usually acting as if there is a great burden of proof required to show the external part is even possible. My point is that the external part is both possible and unsurprising.
So my argument against wire heading goes; I don’t feel like I want to be a wirehead, the vast majority of minds in general don’t want to become wireheads, low prior + no evidence = “why has this even been promoted to my attention?”
That depends on the exact implementation. The paperclipper might be purely feedback-driven, essentially a paperclip-thermostat. In that case, it will simulate setting its internal variables to BB(1000), that will create huge positive feedback and it happily wireheads itself. Or it might simulate the state of the world, count the paperclips and then rate it, in which case it won’t wirehead itself.
The second option is much more complex and expensive. What makes you think humans are like that?
I agree with you that there are non-wireheading agents in principle. I just don’t see any evidence to conclude humans are like that.
The former is incredibly stupid, an agent that consistently gets its imagination confused with reality and cannot, even in principle, separate them would be utterly incapable of abstract thought.
‘Expected Paper-clips’ is completely different to paper-clips. If an agent can’t tell the difference between them it may as well not be able to tell houses from dogs. The fact that I can even understand the difference suggests that I am not that stupid.
Really? You can’t see any Bayesian evidence at all!
How about the fact that I claim not to want to wire head? My beliefs about my desires are surely correlated with my desires. How about all the other people who agree with me, including a lot of commenters on this site and most of humanity in general? Are our beliefs so astonishingly inaccurate that we are not even a tiny bit more likely to be right than wrong?
What about the many cases of people strongly wanting things that did not make them happy and acting on those desires, or vice versa?
You are privileging the hypothesis. Your view has a low prior (most of the matter in the universe is not part of my mind, so given that I might care about anything it is not very likely that I will care about one specific lump of meat?). You don’t present any evidence of your own, and yet you demand that I present mine.
Welcome to evolution. Have you looked at humanity lately?
(Ok, enough snide remarks. I do agree that this is fairly stupid design, but it would still work in many cases. The fact that it can’t handle advanced neuroscience is unfortunate, but it worked really well in the Savannah.)
(I strongly disagree that “most of humanity” is against wireheading. The only evidence for that are very flawed intuition pumps that can easily be reversed.)
However, I do take your disagreement (and that of others here) seriously. It is a major reason why I don’t just go endorse wireheading and why I wrote the post in the first place. Believe me, I’m listening. I’m sorry if I made the impression that I just discard your opinion as confused.
It would have a low prior if human minds were pulled out of mind space at random. They aren’t. We do know that they are reinforcement-based and we have good evolutionary pathways how complex minds based on that would be created. Reinforcement-based minds, however, are exactly like the first kind of mind I described and, it seems to me, should always wirehead if they can.
As such, assuming no more, we should have no problem with wireheading. The fact that we do needs to be explained. Assuming there’s an additional complex utility calculation would answer the question, but that’s a fairly expensive hypothesis, which is why I asked for evidence. On the other hand, assuming (unconscious) signaling, mistaken introspection and so on relies only on mechanisms we already know exist and equally works, but favors wireheading.
Economic models that do assume complex calculations like that, if I understand it correctly, work badly, while simpler models (PCT, behavioral economics in general) work much better.
You are correct that I have not presented any evidence in favor of wireheading. I’m not endorsing wireheading and even though I think there are good arguments for it, I deliberately left them out. I’m not interested in “my pet theory about values is better than your pet theory and I’m gonna convince you of that”. Looking at models of human behavior and inferred values, however, wireheading seems like a fairly obvious choice. The fact that you (and others) disagree makes me think I’m missing something.
What do you mean it can’t handle advanced neuroscience? Who do you think invented neuroscience!
One of the points I was trying to make was that humans can, in principle, separate out the two concepts, if they couldn’t then we wouldn’t even be having this conversation.
Since we can separate these concepts, it seems like our final reflective equilibrium, whatever that looks like is perfectly capable of treating them differently. I think that wire-heading is a mistake that arose from the earlier mistake of failing to preserve use-mention distinction. Defending one mistake once we have already overcome its source is like trying to defend the content of Leviticus after admitting that God doesn’t exist.
I didn’t actually think you were ignoring my opinion, I was just using a little bit of hyperbole, because people saying “I see no evidence” when there clearly is some evidence is a pet peeve of mine.
This point interests me. Lets look a little deeper into this signalling hypothesis. Am I correct that you are claiming that while my concious mind utters sentences like “I don’t want to be a wire-head” subconsciously I actually do want to be a wire-head?
If this is the case, then the situation we have is two separate mental agents with conflicting preferences, you appear to be siding with Subconscious!Ben rather than Conscious!Ben on the grounds that he is the ‘real Ben’.
But in what sense is he more real, both of them exist as shown by their causal effect on the world? I may be biased on this issue but I would suggest you side with Conscious!Ben, he is the one with Qualia after all.
Do you, in all honesty, want to be wire-headed? For the moment I’m not asking what you think you should want, what you want to want or what you think you would want in reflective equilibrium, just what you actually want. Does the prospect of being reduced to orgasmium, if you were offered it right now, seem more desirable than the prospect of a complicated universe filled with diverse being pursuing interesting goals and having fun?
Not that I wanna beat a dead horse here, but it took us ages. We can’t even do basic arithmetic right without tons of tools. I’m always astonished to read history books and see how many really fundamental things weren’t discovered for hundreds, if not thousands of years. So I’m fairly underwhelmed by the intellectual capacities of humans. But I see your point.
Capable, sure. That seems like an overly general argument. The ability to distinguish things doesn’t mean the distinction appears in the supposed utility function. I can tell apart hundreds of monospace fonts (don’t ask), I don’t expect monospace fonts to appear in my actual utility function as terminal values. I’m not sure how this helps either way.
Not exactly like this. I don’t think the unconscious part of the brain is conspiring against the conscious one.
I don’t think it’s useful to clearly separate “conscious” and “unconscious” into two distinct agents. They are the same agent, only with conscious awareness shifting around, metaphorically like handing around a microphone in a crowd such that only one part can make itself heard for a while and then has to resort to affecting only its direct neighbors or screaming really loud.
I don’t think there’s a direct conflict between agents here. Rather, the (current) conscious part encounters intentions and reactions it doesn’t understand, doesn’t know the origin or history of, and then tries to make sense of them, so it often starts confabulating. This is most easily seen in split-brain patients.
I can clearly observe this by watching my own intentions and my reactions to them moment-to-moment. Intentions come out of nowhere, then directly afterwards (if I investigate) a reason is made up why I wanted this all along. Sometimes, this reason might be correct, but it’s clearly a later interpolation. That’s why I generally tend to ignore any verbal reasons for actions.
So maybe hypocrisy is a bit of an misleading term here. I’d say that there are many agents that don’t always have privileged access (and aren’t always conscious), so that they get somewhat ignored, which screws up complex decision making, which causes akrasia. Like, “I’m not getting my needs fulfilled and can’t change that myself right now, so I’m going to veto everything!”. On the other hand, the conscious part is now stuck with actions that don’t make sense, so it makes up a story. It signals “oh, I would’ve studied all day, but I somehow couldn’t get myself to stop watching cat videos, even though I hated it”. Really, it just avoided pain of boredom when studying and needed instant gratification. But “akrasia” is a much nicer cover story.
I’m not saying this is perfectly correct or the whole picture, but I think assuming models like this fits my own experiences closer than assuming actual conflicting agents. Also, those unconscious parts, I suspect, are too simple to actually understand wireheading. They want rewards. If they were smart enough, they might see that wireheading is a good solution.
On a somewhat related note, Susan Blackmore often makes the point when talking about free will that she doesn’t have any and doesn’t even have the illusion of free will anymore, but it doesn’t interfere with her actual behavior. Example quote from Conversations On Consciousness (she talks more about this in several radio shows I can’t find right now):
Susan Greenfield: “[Searle] said that when he goes into a restaurant and orders a hamburger, he doesn’t say, ‘Well, I’m a determinist, I wonder what my genes are going to order.’” Susan Blackmore: “I do. You’re right that Searle doesn’t do that, but when I go in a restaurant, I think, ‘Ooh, how interesting, here’s a menu, I wonder what she’ll choose’; so it is possible to do that.”
I’m totally like Blackmore here. I have no idea what I’ll choose tomorrow or even in ten minutes, only that it will be according to rewards, aversion and so on. Not even considering counterfactuals in my decision making (and not making up verbal reasons anymore) hasn’t crippled me in any way, as far as I can tell.
That makes me skeptical that there’s really all that complex a machinery behind all this, and it makes insistence on “but I really value this complex, external thing!” so puzzling.
Also, I don’t think that qualia are a useful concept ever. Let’s not drag any dualism into this by accident. Besides, what makes you think that “what you call qualia” is something your unconscious processes don’t have, right now? What makes you think you have exactly one conscious mind in your skull?
I don’t have an opinion on that, deliberately. I find wireheading very attractive and it seems about equally nice as the complicated universe, but much easier and more of an elegant solution. The halo effect is way too powerful here and I don’t wanna screw myself over just because I didn’t see a fundamental flaw over how pretty the solution was.
(Of course, as per the nature of wireheading, even if I thought it were a good idea, I would spend no effort on convincing anyone of it. What for, because I value them? Then what am I wireheading myself for?)
Let’s use the example of the Much Better Life Simulator from the post of a similar name, which is less repellent than a case of pure orgasmium. My objections to it are these:
1: Involves memory loss. (Trivially fixable without changing the basic thought experiment; it was originally introduced to avoid marring the pleasure but I think I’m wired strangely with regard to information’s effect on my mood.)
2: Machine does not allow interaction with other real people. (Less-trivially fixable, but still very fixable. Networked MBLSes would do the trick, and/or ones with input devices to let outsiders communicate with folks who were in them.
If these objections were repaired and there were no “gotcha” side effects I haven’t thought of, I would enter an MBLS with only negligible misgivings, which are not endorsed and would be well within my ability to dismiss.
Let’s consider another case: suppose my neurochemistry were altered so I just had a really high happiness set point, and under ordinary circumstances was generally pleased as punch (but had comparable emotional range to what I have now, and reacted in isomorphic ways to events, so I could dip low when unpleasant things happened and soar high when pleasant things happened). I have no objections to this whatsoever, assuming as is customary for thought experiments that the neurochemistry alteration has no other effects.
Let’s consider a third: orgasmium. “I”, in some sense of that word, am turned into an optimally efficient enjoying-thing. I will assume for the sake of charitableness that the enjoying-thing can experience a full range of sensations that I find enjoyable. I’ll try to restrain myself from the “it’s not me” argument aside from some scare quotes, because I don’t think I can express that line of thought in a way comprehensible to someone who diverges so significantly in intuition.
I don’t object to creating orgasmium (ceteris paribus). I think if you’re going to create orgasmium or, say, a rock, go with the former, because orgasmium is enjoying itself and the rock definitely is doing less than that. But I would object to being replaced with orgasmium myself.
I have both of the objections I mentioned regarding the MBLS to this scenario, and several more.
1: It does not seem like a transmuted orgasmium version of “me” would remember much (except maybe how nice everything has always been for all time). Remembering things is not universally enjoyable, and anyway it’s rarely the most enjoyable thing I could be doing; this faculty would be replaced. This objection weakens, but does not evaporate, if all my memories are stored somewhere in the orgasmium, and simply never happen to be bothered with.
2: Also, orgasmium would not interact with real people (it would just directly have the pleasing sensations associated with doing so). Networking orgasmium would make it a less efficient enjoying-thing (the other nodes might say less than maximally enjoyable things), and would seem to violate the thought experiment.
3: Orgasmium would not react to changes in the world. The MLBS involves a complete, complex simulation that I could react to, and also other people, same. The neurochemistry scenario I introduced stipulates that I don’t lose emotional range, I just add a positive number to all the values. Orgasmium would be a less effective enjoying-thing if it allowed this sort of fluctuation; it just turns it all up to eleven and tapes the button down. I do not approve of losing this ability.
4: Orgasmium would not have an interest in accomplishing many of my goals, and would probably not have the cognitive complexity to do it anyway. Most of these goals boil down to interacting with people in some way (writing for an audience), so that folds into the above.
I think in general this boils down to: I don’t want to lose capacities that I currently have. (Capacity to access information, capacity to interact with humans, capacity to experience emotional range.) There are some capacities that I don’t happen to care about (capacity to affect physical objects instead of just indistinguishable simulations thereof), and I would trade those in for a relatively modest increase in enjoyment if the offer were on the table.
WoW already qualifies as that sort of MBLS for some subset of the world.
I tried WoW—weekend free trial. Didn’t see what the fuss was about.
that’s because your life is better than WoW.
I’m rarely attacked by horrifying monsters, that’s one thing. I also have less of a tendency to die than my character demonstrated.
How could you tell the difference? Let’s say I claim to have build a MBLS that doesn’t contain any sentients whatsoever and invite you to test it for an hour. (I guarantee you it won’t rewire any preferences or memories; no cheating here.) Do you expect to not be happy? I have taken great care that emotions like loneliness or guilt won’t arise and that you will have plenty of fun. What would be missing?
Like in my response to Yasuo, I find it really weird to distinguish states that have no different experiences, that feel exactly the same.
Why would you want that? To me, that sounds like deliberately crippling a good solution. What good does it do to be in a low mood when something bad happens? I’d assume that this isn’t an easy question to answer and I’m not calling you out on it, but “I want to be able to feel something bad” sounds positively deranged.
(I can see uses with regards to honest signaling, but then a constant high set-point and a better ability to lie would be preferable.)
Yes, I would imagine orgasmium to essentially have no memory or only insofar as it’s necessary for survival and normal operations. Why does that matter? You already have a very unreliable and sparse memory. You wouldn’t lose anything great in orgasmium; it would always be present. I can only think of the intuition “the only way to access some of the good things that happened to me, right now, is through my memory, so if I lost it, those good things would be gone”. Orgasmium is always amazing.
But then, that can’t be exactly right, as you say you’d be more at ease to have memory you simply never use. I can’t understand this. If you don’t use it, how can it possibly affect your well-being, at any point? How can you value something that doesn’t have a causal connection to you?
How do you know that? I’m not trying to play the postmodernism card “How do we know anything?”, I’m genuinely curious how you arrived at this conclusion. If I try to answer the question “Do I care about losing capacities?”, I go through thought experiments and try to imagine scenarios that are only distinguished by the amount of capacities I have and then see what emotional reaction comes up. But then I’m still answering the question based on my (anticipated and real) rewards, so I’m really deciding what state I would enjoy more and pick the more enjoyable one (or less painful one). Wireheading, however, is always maximally enjoyable, so it seems I should always choose it.
(For completeness, I would normally agree with you that losing capacities is bad, but only because losing optimization power makes it harder to arrive at my goals. If I saw no need for more power, e.g. because I’m already maximally happy and there’s a system to ensure sustainability, I’d happily give up everything.)
(Finally, I really appreciate your detailed and charitable answer.)
I’d probably test such a thing for an hour, actually, and for all I know it would be so overwhelmingly awesome that I would choose to stay, but I expect that assuming my preferences and memories remained intact, I would rather be out among real people. My desire to be among real people is related to but not dependent on my tendency towards loneliness, and guilt hadn’t even occurred to me (I suppose I’d think I was being a bit of a jerk if I abandoned everybody without saying goodbye, but presumably I could explain what I was doing first?) I want to interact with, say, my sister, not just with an algorithm that pretends to be her and elicits similar feelings without actually having my sister on the other end.
In a sense, emotions can be accurate sort of like beliefs can. I would react similarly badly to the idea of having pleasant, inaccurate beliefs. It would be mistaken (given my preferences about the world) to feel equally happy when someone I care about has died (or something else bad) as when someone I care about gets married (or something else good).
Lying is wrong.
I know. It is one of the many terrible things about reality. I hate it.
Memories are a way to access reality-tracking information. As I said, remembering stuff is not consistently pleasant, but that’s not what it’s about.
Counterfactually.
Well, I wrote everything above that in my comment, and then noticed that there was this pattern, and didn’t immediately come up with a counterexample to it.
I think it’s fine if you want to wirehead. I do not advocate interfering with your interest in doing so. But I still don’t want it.
Apologies for coming to the discussion very, very late, but I just ran across this.
How could you possibly get into this epistemic state? That is, how could you possibly be so sure of the sustainability of your maximally happy state, without any intervention from you, that you would be willing to give up all your optimization power?
(This isn’t the only reason why I personally would not choose wireheading, but other reasons have already been well discussed in this thread and I haven’t seen anyone else zero in on this particular point.)
Why does my intuition reject wireheading? Well, I think it has something to do with the promotion of instrumental values to terminal values.
Some pleasures I value for themselves (terminal) - the taste of good food, for example. As it happens, I agree with you that there is no true justification for rejecting wireheading for these kinds of pleasures. The semblance of pleasure is pleasure.
But some things are valued because they provide me with the capability, ability, or power (instrumental) to do what I want to do, including experiencing those terminal pleasures. Examples are money, knowledge, physical beauty, athletic abilities, and interpersonal skills.
Evolution has programmed me to derive pleasure from the acquisition and maintenance of these instruments of power. So, a really thorough wireheading installation would make me delight in my knowledge, beauty, charisma, and athleticism—even if I don’t actually possess those attributes. And therein lies the problem. The semblance of power is not power.
What I am saying is that my intuitive rejection of wireheading arises because at least some of the pleasures that it delivers are a lie, a delusion. And I’m pretty sure that I don’t want to be deluded.
But do I really need power if all of my more basic terminal values are guaranteed? That doesn’t really matter. The way I am currently wired, I want the actual power—not just the terminal pleasure that the power can someday deliver, and certainly not just the satisfied feeling that believing that I am powerful can generate.
Hope that helps.
But you just said you value those things instrumentally, so you can get pleasurable sensations. Raw power itself doesn’t do anything for you, just sitting there.
I can see how, when considering being wireheaded, you would come to reject it based on that. Essentially, you’d see (e.g.) wireheaded power as not actually instrumentally useful, so you reject the offer. It sounds like snake-oil.
But isn’t that a false conclusion? It might feel like it, but you won’t actually feel any worse off when you’re completely wireheaded. Fake capabilities are a problem when interacting with agents who might exploit you, so the heuristic is certainly useful, but it fails in the case of wireheading that actually delivers on its promises.
You won’t need knowledge and power and so on when you’re in wirehead heaven, so wireheading can simply ignore or fake them.
(Disclaimer: muflax does not advocate giving up your autonomy to Omegas claiming to provide wirehead heaven. Said Omegas might, in fact, be lying. Caution is advised.)
It does.
Because writing big numbers on the speedometer with a sharpie doesn’t get me to the destination sooner.
I think the question is: why do you really need to get there?
Exactly that.
Are you looking for a definition? Specifically coming up with a dictionary definition for the word “value” doesn’t seem like it would be very instrumental to this discussion. But really, I think just about everyone has a pretty good sense for what we’re talking about when we post the symbols “v”, “a”, “l”, “u”, and “e” on less wrong for us to simply discuss the concept of value without trying to come up with a “definition”.
Completely understanding what it is we value is a cognitive psychology or AI question. For our purposes, we can get some pretty decent bayesian evidence on what our values are by simply asking “which future scenario do I want to steer the world towards?” Is that going to give us perfect information on exactly what we value? No. But is it a pretty good start? Yes.
Evolution. Next question.
That’s what EY talks about when he uses the phrase “how _ feels from inside”. I do feel like I care about states directly. That means, of course, that what is actually being fed into that algorithm is my own mental model of a future world. Now, when we talk about wireheading, the mental model of that particular future world causes the algorithm to return the response “I’d prefer a different future” from my own decision making algorithm. I don’t want my future self to be wireheaded. If someone else does, I will be disappointed, because I will not get a chance to interact or socialize with them any more. We will become so different that we will never get a chance to have some awesome experience together. I won’t necessarily stop them, but I would try to talk them out of it.
No, I’m trying to understand the process others use to make their claims about what they value (besides direct experiences). I can’t reproduce it, so it feels like they are confabulating, but I don’t assume that’s the most likely answer here.
That seems horribly broken. There are tons of biases that make asking such questions essentially meaningless. Looking at anticipated and real rewards and punishments can easily be done and fits into simple models that actually predict people’s behaviors. Asking complex question leads to stuff like the Trolley problem which is notoriously unreliable and useless with regards to figuring out why we prefer some options to others.
It seems to me that assuming complex values requires cognitive algorithms that are much more expensive than anything evolution might build and don’t easily fit actually revealed preferences. Their only strength seems to be that they would match some thoughts that come up while contemplating decisions (and not even non-contradictory ones). Isn’t that privileging a very complex hypothesis?
Because wireheading is death.
Beyond the definitions, a person walks into a room, something happens, they never walk out again, nor is the outside world impacted, nor is anything changed by them. They might as well walk into the wireheading room and have their brains dashed upon the floor. Their body may be breathing, but they are dead just the same.
If the wireheading were un-doable, then it would be nothing more than suspended animation. Pleasurable, but it’s still a machine you plug into then do nothing until you unplug. Frankly, I haven’t the years to waste in an orgasmium.
No matter what you say happens, when you take away the labels and blur your eyes a bit, you’re asking us if we want to die. Granted, we’re dieing and being sent immediately and verifiably to an earthly heaven, but it’s still death. Adding “you experience Xtreme pleasure” before you kill us doesn’t make it any more appetizing.
Death is the eternal enemy of mankind. I will not join its side just because it’s slightly more pleasurable.
Question reversal: suppose Omega reveals to you that your life has been a simulation. Your actions inside the simulation don’t affect the outside, ‘real’ world—nobody is watching you.
However, Omega offers to remove you from the simulation and instantiate you in the real world outside. Unfortunately, Omega predicts that your future life on the outside won’t be nearly as fun as the one you’ve had in the simulation up until now. The difference in satisfaction—including satisfying your preferences that apply to “affecting the ‘real’ world”—may be as great as the possible improvement due to wireheading...
Would you accept the offer and risk a life of extreme misery to improve your chance of affecting the “real” world? Would you consider yourself “dead” if you knew you were being simulated?
(Apologies for replying late.)
I would accept Omega’s offer to ‘pop’ me up a level. I would accept even if it meant misery and pain. I would always accept this offer. Actually, bar that. I would accept the offer conditional on the fact that I’d be able to impact the ‘real’ world more outside the simulation than inside. I’d be comfortable staying in my current level if it was providing some useful effect in the higher levels of reality that I couldn’t provide if I were ‘popped’ out.
Upon learning I was in a simulation, I would make it my life’s sole purpose to escape. I think this would be a common reaction. It is my understanding that Buddhism believes this world is a simulation and the goal of each Buddhist is to ‘pop’ themselves unto a higher plane of reality. Many branches of Christianity also put strong emphasis on proving one’s worth on Earth solely to be in as good a position as possible once we die and ‘pop’ into the ‘real’ world in the Kingdom of Heaven.
Exploring your question more, I realize that there are at least two situations this wouldn’t work in. The first situation would be if reality consisted of a circularly linked list of ‘real’ worlds, and ‘popping’ up or ‘pushing’ down enough times would bring you back to the same world you started at. The second situation would be if there were infinitely many layers to ‘pop’ up through. I’m actually not sure what I would do if reality were in such an impossible configuration.
Why do you think infinitely many layers would be an impossible configuration? If anyone, anywhere has an actual real turing machine (as opposed to a finite approximation of a turing machine), creating such a configuration is basically child’s play.
Have you read The Finale of the Ultimate Meta Mega Crossover which explores just this possibility ?
Wireheads are still experiencing the pleasure. They are not in suspended animation, stuff is still happening in their brains. They don’t affect the outside world anymore (beyond ensuring their survival), but so what? The fact that it is superficially similar to death does not bother me at all. If no more optimization is needed, why bother with optimizing?
You’re essentially just restating the basic intuition against wireheading, just more emphatically. I find it just as incomprehensible.
(For completeness, I don’t share your aversion to death at all. I’m totally indifferent to it. I essentially agree with teageegeepea here. Maybe this influences the intuition.)
I do not mean that Wireheading is metaphorical death. It is not just an emotionally charged statement that means I am really against Wireheading. I mean that Wireheading is literally death.
The cluster of death-space consists of more than just stopping breathing. I am arguing that the important boundary in the definition-space of death is not ‘stopped breathing’ but ‘inability to affect the outside world’. Imagine the following Omega enabled events, rest assured that none of them are reversible once Omega stops toying with you and finishes this experiment. Ask yourself if you consider the following states death:
1 -Omega transforms your body into a corpse! You cannot move or do anything a corpse cannot do.
2-Omega transforms your body into a corpse, but lets you keep moving and taking actions. You return back to work on monday, and thankfully there’s no extra smell.
3-Omega teleports you to a dimension of nothingness, and you’re stuck there for all eternity.
4-Omega teleports you to a dimension full of nothingness, then brings you back out a year later.
5-Omega turns you into a tree. You’re not able to do anything a tree cannot do, like think, move, or anything of the sort.
6-Omega turns you into a tree, but gives you the power to move and think and talk in rhymes.
7-Omega keeps your body the same, but severs your ability to do anything including moving your eyes or blinking. Luckily your autonomic system keeps you breathing and someone puts you a nutrient drip before that ‘not eating’ thing catches up to you.
8-Omega keeps your body the same, but separates your ability to do anything into a separate non-corporeal facility. IE, you can move things with your mind.
9-Omega replaces your body with a corpse doll and shifts you into a parallel plane where you can view the world but not interact.
10-Omega replaces your body with a corpse doll and shifts you into a parallel plane where you can both view and interact with the world.
All the odd numbers seem straight up death to me. 1 is regular death, 3 is getting sucked into a black hole, 5 is well dieing and having a tree planted on you, 7 is brain death, and 9 is christian death. All the even numbers, even though they’re identical except your how much you can effect the world, feel like you’re gaining superpowers. Well, except that solitary confinement for a year one. The meaningful divide of death is not if we’ve stopped breathing or even stopped existing, it’s whether or not we can effect the outside world.
Being strapped into the pleasure machine lets us still breath like brain death, but takes away our ability to do anything, just like brain death. Wireheading that takes away our ability to effect the outside world kills us.
(Thanks for the clarification, that makes your comment much clearer.)
How would 2) work? What do you mean, my body becomes a corpse, but goes to work? As a corpse, I won’t have blood circulation for example, so how could I walk? Unless Omega magically turns me into an actual zombie, but what’s the use of thinking about magic?
Similarly, 6) ain’t a tree, but at best a brain stuck in a tree.
Does 3) include myself as separate from the nothingness? So I’m essentially “floating” in nothingness, kinda like a Boltzmann brain?
8) isn’t possible in principle. There are no separate mental events, unless Omega can change metaphysics, but that’s uninteresting.
I’d consider 3), 4), 7), 9) and 10) totally alive, assuming mental processing is still happening, stuff is still getting experienced, it’s just that any outgoing signals to influence the world are getting ignored. If this isn’t happening (e.g. I’m in a deep coma), then I’m straight-up dead. As long as I have subjective experiences, I’m alive.
Overall though, arguing about the definition of “death” isn’t gonna be useful.
(Omega was supplied so that magical scenarios would be possible for the thought experiment.)
My definition vs your definition of death is very enlightening in light of our differences on wireheading.
You view being alive as being able to think, to receive input and experience. I view being alive as being able to act, to change and shape the world. This division cuts through the experience of wireheading; it is the state of thinking without the ability to act. Life to you; death to me. I would venture a guess that anyone who is pro-wireheading would hold your view of life/death while anyone who is anti-wireheading would hold my view of life/death.
You wanted to know why all those other arguments sounded good to everybody, but not to you. We have incompatible priors. There is no sufficiently convincing argument that can cross the gulf between life and death. I do not have sufficient rationalist superpowers to try and change your priors (or even make you want to change them, as I wouldn’t want to change mine). But if you wish to understand what other people are thinking as they reject Wireheading, simply close your eyes and try and imagine the choice you would make if you instead believed your time of death were the instant you never acted upon the world again.
They are not being convinced by insufficient arguments. They are merely starting from a different metaphysical position than you.
That doesn’t dissolve the problem completely for me, it just moves the confusion from “Why do humans disagree on wireheading?” to “Why do humans have different views on what constitutes death?”. Is it just something you memetically pick up and that then dominates your values?
I’d rather assume that the (hypothetical) value difference comes first and we then use this to classify what counts as “dead”. “yup, can still get pleasure there, I must be alive” vs. “nope, can’t affect the external world, I must be dead”.
That is a very interesting question. I’m sure I feel quite as puzzled looking at you from this side as you do looking at me from that side. I would also assume that there is some other first factor.
Sadly, it would be a bit outside of the depth of my understanding of metaphysics (and the scope of this page) to try and discover what it is. Still, I am intrigued about it and will keep thinking on the subject.
unpack “the world” and you’ll maybe sympathize with wireheaders more.
This perspective does explain why I would be much less worried about wireheading if I was older than I am right now and had already reproduced. If I had kids who were off having their own kids, I could think “Ah good, my DNA is off replicating itself and at this point, and whether or not I die is unlikely to change that. In fact, the best way to help them out would probably be to make sure I don’t spend too much of the money they might theoretically inherit, so if wireheading was cheaper than a world yacht tour, my kids and grandkids might even benefit from me deciding to wirehead.
That being said, I say this as someone who hasn’t even experienced a world yacht tour. I mean, now that I’m a working adult, I can barely manage to acquire much more then about 10 consecutive days of not working, which gives one just barely enough time to scratch through the surface of your current hedonism and encounter boredom with choices (The last time I was bored and had a choice of activity, it felt refreshing because of how RARELY I’m bored and have choices, as opposed to being bored because you are stuck in your current activity with no control) Before deciding to wirehead, it seems like it might be well worth while to at the very least take some time to experience being retired to make sure I have a good feel for what it is that I’m giving up.
But I also realize at this point I also feel like it’s a bit presumptuous of me to say what I would want at 70 or so, at 27. I’ve experienced too many changes in philosophy in the last 10 years to feel assured that my current set of desires are stable enough to suggest something that far in the future. I mean, it doesn’t feel likely they’ll change, but it didn’t feel likely they would change before either, and yet they did.
So do you think these reasons for maybe wanting to wirehead at 70 would be good enough reasons to kill yourself? Because if you are accepting Xachariah’s response then it seems like that is the standard you’d have to meet.
Yes, there are definitely a set of circumstances where I could see myself willing to essentially suicide when I’m significantly older. I mean, when you’re old, cheap wireheading seems to be equivalent to being given a choice between:
1: Die pleasantly and painlessly after a grand farewell party, allowing your family to have a good inheritance and ascend to the technological equivalent of heaven. 2: Die in a hospital bed after horrible mind crushing suffering where you are incoherent, draining away money and resources for your family, and then nothing.
If you’re going to die anyway (and I am assuming Immortal life is not on the table. If it is, then the entire scenario is substantially different), option 1 sure sounds a lot better.
And yes, there are also a large number of circumstances where I can see myself not wireheading as well. Maybe my Grandfatherly advice will prove absolutely crucial to my grandchildren, who think that my great grandchildren just won’t be the same without getting to meet me in person. It’s entirely possible that everyone around me will still need me even when I’m 70, or still when I’m 80, or even when I’m 90. (With medical technology improving, maybe 90 will be the new 70?)
That’s why I mentioned I’d want to get a feel for retired life before deciding to wirehead. I don’t really know what it’s going to be like being a retired person for me.
For that matter, the entire concept of retirement may not even be around by the time I’m 70. It’s not just my own philosophy that can change in 43 years. Our entire economic system might be different. And I also had the implicit assumption of cheap wireheading, but it may turn out that wireheading would be horribly expensive. That’s an entirely different set of calculations.
The scenario stipulates your wireheading experience will be the best one possible. If you really enjoy yacht tours, you’ll experience simulated yacht tours. You’re not giving anything up in terms of experience.
That’s a good point, and It made me think about this again, but my understanding is that I think I must be giving up SOME possible experience. Wouldn’t it break the laws of physics for a finitely sized wireheading world to contain more possible states to experience than the universe which contains the wireheading world and also contains other things?
Now, for yacht tours, I don’t think this matters. Yacht tours don’t require that kind of complexity. Actually, I’m not even sure how this kind of complexity would be expressed or if it’s something I could notice even if I was a theoretical physicist with trillions of dollars of equipment.
But after rethinking this, I think this complexity represents some type of experience and I don’t want to rush into trading it away before I understand it unless I feel like I have to, so I still feel like I may want to wait on wireheading.
I suppose an alternate way of looking at it might be that I have a box of mystery, which might contain the empty vastness of space or some other concept beyond my understanding, and if I trade it, I will never be able to access it again, but in exchange I get offered the best possible experience of everything that ISN’T in the box, many of which I already know.
There is a distinct possiblity I’m just being irrationally afraid of rushing into making permanent irreversible decisions. I’ve had that type of fear for decisions which are much more minor than wireheading, and it might be coming up again.
That being said, being unsure of this point represents a contradiction to something that I had thought earlier. So I’m definitely being inconsistent about something and I appreciate you pointing it out. I’ll try to break it down and see if I can determine which point I need to discard.
How familiar are you with expected utility maximizers? Do you know about the difference between motivation and reward (or “wanting” and “liking”) in the brain?
We can model “wanting” as a motivational thing—that is, if there was an agent that knew itself perfectly (unlike humans), it could predict in advance what it would do, and this prediction would be what it wanted to do. If we model humans as similar to this self-knowing agent, then “wanting” is basically “what we would do in a hypothetical situation.” For example, I want to eat a candy bar, so if I had a candy bar I would eat it. And some people want to wirehead, which is the same as saying that if they could they would.
Although “wanting” and “liking” aren’t the same, they are correlated, so you could make an argument for wireheading that went something like this: “wireheading would make my pleasure centers light up, therefore it’s much more likely to be what I want than something that doesn’t make my pleasure centers light up, like many parts of ordinary life.” But the trouble with this argument is that it doesn’t take into account other sorts of evidence, with the most notable being the output of our self-modeling processes. If I could, I wouldn’t.
I think I’m familiar with that and understand the difference. I don’t see it’s relevance. Assuming “wanting” is basically the dopamine version of “liking” seems more plausible and strictly simpler than assuming there’s a really complex hypothetical calculation based on states of the world being performed.
Also, I suspect you are understanding wireheading as too narrow here. It’s not just the pleasure center (or even just some part of it, like in “inducing permanent orgasms”), but it would take care of all desirable sensations, including the sensation of having one’s wants fulfilled. The intuition “I get wireheaded and still feel like I want something else” is false, which is why I used “rewards” instead of “pleasure”. (And it doesn’t require rewiring one’s preferences.)
Confabulation and really bad introspective access seem much more plausible to me. If you modify details in thought experiments that shouldn’t affect wireheading results (like reversing Nozick’s experience machine), people do actually change their answers, even though they previously claimed to have based their decisions on criteria that clearly can’t have mattered.
I’d much rather side with revealed preferences, which show that plenty of people are interested in crude wireheading (heroin, WoW and FarmVille come to mind) and the better those options get, the more people choose them.
Why assume? It’s there in the brain. It’s okay to model reality with simpler stuff sometimes, but to look at reality and say “not simple enough” is bad. The model that says “it would be rewarding, therefore I must want it” is too simple.
Except the brain is a computer that processes data from sensory organs and outputs commands—it’s not like we’re assuming this from nothing, it’s an experimental result. I’m including all sorts of thing in “the world” here (maybe more than you intended), but that’s as it should be. And ever since mastering the art of peek-a-boo, I’ve had this concept of a real world, and I (i.e. me, my brain) use it in computation all the time.
This is part of why I referenced expected utility maximizers. Expected utility maximizers don’t choose what just makes them feel like they’ve done something. They evaluate the possibilities with their current utility function. The goal (for an agent who does this) truly isn’t to make the utility meter read a big number, but to do things that would make their current utility function read a big number. An expected utility maximizer leading a worthwhile life will always turn down the offer to be overwritten with orgasmium (as long as one of their goals isn’t something internal like “get overwritten with ogasmium”).
And plenty of people aren’t, or will play tetris but won’t do heroin. And of course there are people who will lay down their lives for another—to call wireheading a revealed preference of humans is flat wrong.
Correct and I don’t disagree with this. An actual expected utility maximizer (or an approximation of one) would have no interest in wireheading. Why do you think humans are best understood as such utility maximizers? If we were, shouldn’t everyone have an aversion, or rather, indifference to wireheading? After all, if you offered an expected paperclip maximizer the option of wireheading, it would simply reject it as if you had offered to build a bunch of staples. It would have no strong reaction either way. That isn’t what’s happening with humans.
I’m trying to think of a realistic complex utility function that would predict such behavior, but can’t think of anything.
True, there isn’t anything like a universally compelling wirehead option available. Each option is, so far, preferred only by minorities, although in total, they are still fairly widespread and their market share is rising. I did express this to sloppily.
Yeah, true. For humans, pleasure is at least a consideration. I guess I see it as part of our brain structure used in learning, a part that has acquired its own purpose because we’re adaptation-executers, not fitness maximizers. But then, so is liking science, so it’s not like I’m dismissing it. If I had a utility function, pleasure would definitely be in there.
So how do you like something without having it be all-consuming? First, care about other things too—I have terms in my hypothetical utility function that refer to external reality. Second, have there be a maximum possible effect—either because there is a maximum amount of reward we can feel, or because what registers in the brain as “reward” quickly decreases in value as you get more of it. Third, have the other stuff you care about outweigh just pursuing the one term to its maximum.
I actually wrote a comment about this recently, which is an interesting coincidence :D I’ve become more and more convinced that a bounded utility function is most human-like. The question is then whether the maximum possible utility from internal reward outweighs everyday values of everything else or not.
I agree with you on the bounded utility function.
I still need to think more about whether expected utility maximizers are a good human model. My main problem is that I can’t see realistic implementations in the brain (and pathways for evolution to get them there). I’ll focus my study more on that; I think I dismissed them too easily.
Apparently, most of us here are not interested in wireheading. The short version of mulfax’s question is: Are we wrong?
My answer is simple: No, I am not wrong, thanks for asking. But let me try to rephrase the question in a way that makes it more relevant for me:
Would we change our mind about wireheading after we fully integrated all the relevant information about neuroscience, psychology, morality, and the possible courses of action for humanity? Or to paraphrase Eliezer, would we choose wireheading if we knew more, thought faster, were more the people we wished we were?
Despite Eliezer’s emotive language, the answer to this question is not immediately obvious to me. And this question is something CEV proponents must tackle somehow, because sceptics will not accept the obvious answer. (I mean, the answer that if CEV chooses wireheading, then it must be good after all.)
To clarify, I’m not interested in convincing you, I’m interested in understanding you.
Hey, humans are reward-based. Isn’t wireheading a cool optimization?
Nope.
That’s it?
That’s it.
But reinforcement. It’s neat and elegant! And some people are already doing crude versions of it. And survival doesn’t have to be an issue. Or exploitation.
Still nope.
Do you have any idea what causes your rejection? How the intuition comes about? Do you have a plausible alternative model?
No.
O… kay?
I know that “let me give you a coredump of my complete decision algorithm so you can look through it and figure it out” isn’t an option, but “nope” doesn’t really help me.
Good point about CEV, though.
You aren’t getting a “nope” muflax.
This is where you’re wrong. Reward is just part of the story. Humans have complex values, which you seem to be willfully ignoring, but that is what everyone keeps telling you.
To clarify, are you claiming that wireheading is actually a good thing for everyone, and we’re just confused? Or do you merely think wireheading feels like a fine idea for you but others may have different values? At times, your post feels like the former, but that seems too bizarre to be true.
My own view is that people probably have different values on this issue. Just as suicide seems like a good idea to some people, while most people are horrified by the idea of committing suicide, we can genuinely disagree and have different values and make different decisions based on our life circumstances.
As I said here, it really weirds me out if it weren’t a universally good or bad idea. As such, it should be good for everyone or no-one. Wireheading doesn’t seem like something agents as similar as humans should be able to agree to disagree on.
Suicide is even more basic than wireheading, yet humans disagree about whether or not to commit suicide. There are even some philosophers who have thought about it and concluded suicide is the “rational” decision. If humans cannot, in fact agree about whether to exist or not, how can you think wireheading has a “right” answer?
Humans do also still disagree on p-zombies or, more basic, evolution. That doesn’t mean there isn’t a correct answer.
But you’re right that pretty much any value claim is disputed and when taking into account past societies, there aren’t even obvious majority views on anything. Still, I’m not comfortable just giving up. “People just are that different” is a last resort, not the default position to take in value disputes.
The distinction is that evolution and zombies are factual disputes. Factual views can be objectively wrong, preferences are purely subjective. There is no particular reason any one mind in the space of possible minds should prefer wireheading.
To clarify, the claim is not “all agents should prefer wireheading” or “humans should have wireheading-compatible values”, but “if an agent has this set of values and this decision algorithm, then it should wirehead”, with humans being such an agent. The wireheading argument does not propose that humans change their values, but that wireheading actually is a good fulfillment of their existent values (despite seeming objections). That’s as much a factual claim as evolution.
The reason I don’t easily expect rational disagreement is that I expect a) all humans to have the same decision algorithm and b) terminal values are simple and essentially hard-coded.
b) might be false, but then I don’t see a realistic mechanism how they got there in the first place. What’s the evolutionary advantage of an agent that has highly volatile terminal values and can easily be hijacked, or relies on fairly advanced circuitry to even do value calculations?
Humans seem to act as general meme hosts. It seems fairly easy for a human to be hijacked by a meme in a way that decreases their genetic inclusive fitness. Presumably this kind of design at least had an evolutionary advantage, in our EEA, or we wouldn’t be this way.
If you can host arbitrary memes, then “external referent consequentialism” doesn’t really need any extra circuitry. You just have to be convinced that it’s something you ought to do.
By the way, by a universally good idea you mean a) an idea about which any person can be persuaded by a rational argument, or b) objectively morally good idea, or c) something else? Because if a), it is very unlikely to be so. There are people who don’t accept logic.
I mean an idea that, if properly understood, every human would agree with, so a). Well, there are some humans that, for various reasons, might not be able to actually follow the reasoning or who are so broken to reject correct arguments and whatnot. So “every” is certainly exaggerated, but you get the idea. I would not expect rational disagreement.
Well, we don’t want our machines to wirehead themselves. If they do, they are less likely to be interested in doing what we tell them to—which would mean that they are then less use to us.
Sure, but what about us?
As designers, we have good reasons to find a way around wireheading (and somewhat less seriously and metaphorically, Azathoth has good reasons to prevent us from wireheading). So making wireheading-proof agents is important, I agree, but that doesn’t apply to ourselves.
The connection with us could be that (to the extent we can) we choose what we want as though we were machines at our disposal.
… There is a component that wants doughnuts for breakfast, but actually “I” want eggs for breakfast since I’d rather be healthy … and the machine that is me obstensibly makes eggs. The hedeonistic component of our brain that wants wire-heading is probably/apparently repressed when it comes down to conflicts with real external goals.
You should try nicotine-addiction to understand this. That’s possible, because “reward” and “pleasure” are different circuits in the brain.
John Wesley said, “earn all you can; save all you can; give all you can.” He was serious.
What does that have to do with wireheading? As far as I can tell, that quote resonates with me on a deep level, though I replace “give” with something like “optimize” or “control.” And so when presented a choice between pleasure and control, I choose control. (If actually presented the choice, I calculate the tradeoff and decide if the control is worth it.) So, even though orgasmium!Vaniver would be more satisfied with itself, current!Vaniver is more satisfied with future!Vaniver, because the second is still shaping reality in its image, and that matters more to me than sensed pleasure.
I know people who specifically said that if orgasmium were available, they’d take it in an instant. I also know people that would not. Wireheading doesn’t have to be univerally, objectively “good” or “bad.” If wireheading would satisfy all your values, and it becomes available to you, well, go for it.
I know that if I was given access to orgasmium, I’d probably be content living on it for the rest of my life. That doesn’t change the fact that BEFORE having access to orgasmium, I simply prefer not to accept it, and instead create art and discover things. That’s my personal preference.
I’ve actually considered that, even ignoring the orgasmium element, I could probably self modify into a person who doesn’t care about accomplishing stressful art projects, who just does things that make him immediately happy. But my eventual response to myself: I’m the sort of person who prefers not to self modify into that kind of person. No, I don’t need to be able to justify that to anyone else, I just need to feel that way.
You seem to classify each argument against wireheading as a bias: since the argument doesn’t persuade you, the ones who are persuaded must make some error in judgement. But those arguments aren’t (all) meant to make people value “reality” more than pleasure. Most of them aim at people who already do prefer real achievements over pleasure (whatever it means) but are confused about the possibility of wireheading. In particular,
Isn’t an argument against wireheading per se, but against some sorts of wireheading which stimulate the reward mechanisms while ignoring the more complex desires.
Yes, it is possible to enjoy something while not wanting to do it. For example, I enjoy writing this response while I wish I had a motivation to do something more practical and productive, say cook a lunch. I know that after one hour, I will regret that I haven’t prepared my lunch, since I would have to do that anyway, and I will be unnecessarily hungry one more hour. That’s what akrasia is like. If you have never experienced it, you are a lucky person. Now there are some cynics (very often found among economists) who say that this is a confusion—that people always want what they really do, by definition, and the perceived wanting is a self-serving illusion useful for signalling purposes. Well, I don’t agree.
If an agent wants to achieve a task which takes a non-negligible time to complete, it better also has to want to preserve its wanting to achieve it. If the values themselves are of no value to you, you could be argued out of them easily, change values often, and achieve nothing. That’s perhaps the evolutionary reason for the presence of value preservation value in humans (with all caveats common for amateur ev-psych speculations). Anyway, most people don’t want to become grossly different from their present state, period. It’s possible that’s not your case, but that doesn’t make it universally false.
If you want to dissolve meaning of “want”, you’d better make some survey and analyse people’s opinions, rather than starting from the assumption “it’s all about anticipating and achieving rewards” which seems plausible to you, and then rationalise all desires that don’t fit as biased or confused. “I don’t want it” may not be the most persuasive argument, it has even some negative social signalling cost in this respect (it’s basically like saying “I don’t want to argue about it anymore”), but’s that doesn’t mean that it expresses an invalid position. “Why?” It’s somehow built in the brain. How do you know what you “want”? In the same way as we know what we feel. Many components of wanting are directly accessible for the consciousness in a form similar to sensory inputs, and are not a result of complex deliberation which can be meaningfully analysed by questions “why” and “how do you know”. Also notice that you can attack your own position in the same way: Why do you care about reward? How do you know that you are really rewarded by the wireheading system? This doesn’t seem to be a sensible direction of inquiry.
Any popular opinion is likely burdened by biases and if there is a better way to know, surveying the majority intuition shall be discarded and replaced by the better method. Now, when it comes to assessing real human desires, what is the better method?
The post you link sounds much more plausible than its opposite, but it’s perhaps difficult to explain to somebody who never experienced akrasia. (Akrasia really isn’t a signalling game.) Distinction between near and far may not be the best way to approach the issue, but anyway, this argument is only a combination of #2 and #3.
Intuition pumps are legitimate sort of argument. This one only reiterates the point of #1, putting in vivid imagery in place of abstract arguments about “values”. What is wrong here?
That’s a meta-argument and doesn’t address wireheading directly. I doubt a person who wants to argue against wireheading would use it. It is, indeed, not socially acceptable to support wireheading, but it is also not socially acceptable to use social acceptability of an opinion to support it, at least not on a pretty non-conformist rationalist forum.
I did not intend this. I simply find them all very unconvincing and (briefly) gave my reasons why. I assume that at least some of them rely on hidden assumptions I don’t see and only look like an error to me. I don’t have an opinion on wireheading either way (I’m deliberately suspending any judgment), but I can only see good arguments for it, but none against it. If that were really the case, I would expect many more experienced rationalists to be convinced of it (and I highly respect the opinions of pretty much everyone I linked to), so I’m operating on the assumption of an inferential gap.
I don’t think that’s cynical and I do find it very plausible. Explaining akrasia (which I do have) in terms of being mistaken what I like and having a (often unconscious) conflict between different parts of the brain works just fine for me. The moment I realize I’m not actually enjoying what I do, I either stop immediately or find that I’m fulfilling some other emotional demand, typically avoiding guilt or embarrassment.
No, just no, especially if they give different results based on minor modifications, like with Nozick’s experience machine. (Or look at the reactions to Eliezer’s Three Worlds Collide and various failed utopias.) I’d rather have no opinion than base it on a complex intuition pump.
Your comments on 1) and 8) I agree with. The other points I addressed in other comments here, I think.
I don’t think there is an inferential gap of the usual type (i.e. implicit hidden knowledge of facts or arguments). It’s more probably a value disagreement, made harder by your objection to well-definedness of “value”.
Agreed about the unconscious conflict, but not about the conclusion. A real akrasic wants to do two incompatible things X and Y, chooses X and later regrets the choice. He knows that he will regret the choice in advance, is full aware of the problem, and still chooses X. An akrasic “enjoys” X (at the moment), but is genuinely unhappy about it later—and if he realises the problem, the unhappiness emerges already during X so that X is no longer enjoyable, but still it is hard to switch to Y. It is a real and serious problem.
The cynical (no moral judgement intended) explanation of akrasia basically tells that the agent really “prefers” X over Y, but for some reason (which usually involves hypocrisy) is mistaken about his preference. But, if it is true, tell me: why do akrasics try to fight akrasia, often privately? Why they insist that they want Y, not X, even if there are no negative consequences for admitting the desire for X? Why they are happy after doing Y and unhappy after doing X, and often remember being more happy doing Y than doing X?
Of course, you can redefine the words “want” and “prefer” to mean “what you actually do”, for the price of people being mistaken about significant part of what they want. But then, these words become useless, and we lose words denoting the stuff people people report to “want” (in the conventional meaning).
Failed utopias regularly fail mainly because people can’t envisage all consequences of a drastic change of social order, which is caused by complexity of human societies. Being mistaken about what we want is a part of it, but not the most important one. Early communists weren’t surprised that they didn’t like party purges and mass executions that much. They were surprised that these things happened.
Different reactions to some fictional scenarios may well represent different preferences. Why is this explanation worse than that people are mistaken about their preferences or confused by an intuition pump? (I agree that pumps don’t make reliable arguments, but sometimes they are the only accessible arguments. If you ever decide to convince me to support wireheading, you would probably need a powerful intuition pump to do it.)
You make a good point about private akrasia conflicts. I’ll have to think more about this. It doesn’t make sense either way right now.
The reason I object to major preference differences among humans is that this breaks with the psychological unity of humanity. It’s not just that there are some minor variations or memetic hijackings in the utility function, but it seems like some are maximizing rewards, while others maximize expected utility. That’s a really big difference, so it makes more sense to find an explanation that assumes only one mechanism and explains the respective “unusual” behavior in terms of it.
If we’re expected utility maximizers, why are some attracted to wireheading? In terms of reinforcement and things like operant conditioning, raking up “superstitions” and tons of instrumental goals makes sense. Highly splintered and hugely divergent terminal value, however, seems weird to me. Even weird for Azathoth’s standards.
About failed utopias, you misunderstood me. I meant Eliezer’s scenarios of failed utopias, like this one.
Fair enough.
Utility is so general a term that it can encompass rewards. It can be said that all people are maximising utility whenever their decisions don’t exhibit cyclic preferences or some other blatant (but nevertheless common) error, but this would also be a bit misleading—recalling the von Neumann-Morgenstern theorem usually begs for the cynic interpretation of utility that does care more about what people do rather than what they really want.
It’s probably better to say that there are at least two distinct decision processes or systems working together in the brain, and, depending on circumstances, one of them prevails. The unconscious process steers the decision towards safe immediate psychological rewards; the conscious one plans further in advance and tries to accomplish more complex aims related to the external world. (Generalisation to the case of more than two processes working on several different time scales should be straightforward.)
Sometimes—in stress, during akrasic behaviour, presumably also under wireheading, the unconscious system overrides the conscious one and executes its commands. In other situations the conscious system can take priority. The conscious system wants to remain in control, but knows that it can be overriden. Therefore it tries to avoid situations where that can happen.
Now into the more speculative realm. I would guess that retaining at least some control should be strongly prioritised over any amount of pleasure on the level of the conscious system, and that this may even be a human universal. But the conscious mind can be fooled into thinking that the control will not be lost in spite of a real danger. For example, the drug addicts overwhelmingly report that they can always stop—when they finally realise that it is not the case, the relevant part of their behaviour is already firmly controlled by the unconscious mind.
The rejection of wireheading may be the manifestation of the desire of the conscious mind to remain in control. Wireheading was traditionally described as total dictatorship of the unconscious mind, and is therefore rejected whenever the conscious mind is under control. But there is a way to overcome that: present wireheading in a different way, more akin to computer game worlds than to heroin. Computer games are basically safe for most users—there were cases of people dying during play, but these are rare. The conscious mind may think that the wireheading will simply be analogous to moving to a different country and that the control will not be lost. So that may be the reason for differing opinions—different intuitions about wireheading. We don’t know how would wireheading feel and so it’s hardly surprising that the intuitions differ.
But even if it were not true, well, some people move abroad leaving their families and friends and jobs behind, others can’t imagine that. Does it break the psychological unity of humanity? There were people who didn’t leave their country even if it was the only real chance to save their lives. Why do you expect that we will all agree on a hypothetical whose role in our evolution is non-existent and which belongs to the class of things which we consistently can’t reason well about, when we differ in more mundane (and therefore evolutionary salient) decisions?
(I’m not fully convinced of the conscious/unconscious split you outline, but let’s go with it for the sake of the argument. It’s certainly a reasonable hypothesis.)
Why would you side with the conscious mind? Do you have a specific reason for this, besides “because it’s the one that holds the power” (which is perfectly acceptable, just not what I’d do in this case)?
As a data point, I personally reject it. Regardless of whether wireheading is actually a good idea, I don’t care about staying in control. I also don’t see my conscious mind as being particularly involved in decision making or value considerations (except as a guiding force on an instrumental level) and I see no reason to change that.
I’m generalizing fairly sloppily now, but I’d expect this to be a fairly widespread Buddhist attitude, for example (and that’s also my background, though I wouldn’t identify with it anymore).
My most obvious objection to wireheading was, “it might be awesome, but I might miss something and end up in a local maximum instead of a global one”, not “it’s gonna enslave me”. I’m perfectly aware that, if wireheaded, I’d have little conscious control left, if any. That does not bother me in the least. Caring that much about control is a perspective I did not anticipate and it does help explain the problem.
Point taken. I thought wireheading was a simple, easy-to-understand and realistic scenario. That doesn’t seem to be the case at all. Taken as a more complicated thought experiment, the rejection and varying intuitions do make sense.
This gets even clearer when I look at this framing:
That’s pretty much the opposite way of how I’d describe it, even though it’s factually totally fine. The metaphor that I was thinking of the first time I saw wireheading described was liberation and freedom from suffering, not dictatorship!
Also, when evaluating it, I was falling back on “wireheady” experiences I already had, like states of high absorption or equanimity in meditation, use of particular drugs (very limited, for health reasons, but never regretted, nor was addiction ever an issue), very intense (semi-)lucid dreams and so on. So I classified wireheading always as “totally doable and somewhat familiar”, not “who knows what will happen?”. I assumed that anyone thinking seriously about it would have comparable experiences to rely on; maybe that’s not so.
Maybe this very different perspective explains the intuitions, but I’m not sure it helps me form an opinion on actual wireheading.
I am not siding with it, I am it. When it holds the power, there is nothing besides it to communicate with you in this dialog.
Good point. The choice of words unconscious/conscious was probably not the best one. Not all parts of the latter process feel conscious, and the former can be involved in conscious activities, e.g. use of language. I should have rather said short term or long term, or have stuck with the standard near/far, although I am not sure whether the meanings precisely overlap.
Buddhism, experiences with drugs, meditations: That may be the core reason for disagreement. Not only experiences can change preferences—inferential gap of sorts, but not one likely to be overcome by rational argument—but reactions to specific experiences differ. Some people hate certain drugs after the first use, others love them.
Buddhism, as far as I know, is certainly a powerful philosophy whose values and practices (meditation, introspection, nirvana) are more compatible with wireheading than most of the western tradition. It is also very alien to me.
My objection is precisely the opposite of what some have said here.
You are affecting the outside world, but only negatively—using resources that could be put to more important things than one person blissing out, without creating anything for anyone else. I therefore see wireheading just you as an unethical choice on your part.
I am not sure if I have an objection to “wirehead everyone sustainably forever”, if that were ever practical.
Edited to clarify very slighty:
I do have some revulsion at the thought but I have no idea what it would be grounded in, if anything.
I believe that humans have natural psychological defenses against the lure of wireheading, because the appeal is something we navigate on a daily basis in our every day lives. In my case, I know I would really enjoy entertaining myself all the time (watching movies, eating good food, reading books) but eventually I would run out of money or feel guilty I’m not accomplishing anything.
Even if you tell people there will be no long-term consequences to wire-heading, they don’t believe you. It’s a matter of good character, actually, to be resistant to wanting to wirehead. For example, when people signal that they wouldn’t wirehead because they prefer real interaction with external reality, there is reward (social and self-generated). (When I decide that I would ‘never wirehead’, I feel a sense of security and well-being.) I don’t know why the intuitions don’t work on you, perhaps you have a different set of background experiences so that you bypass the ‘if-I-don’t-stay-aware-now-I’ll-lose-later’ associations.
It seems to me that those who insist that they wouldn’t wirehead, besides reaping social rewards for signaling a willingness to be altruistic, haven’t fully taken to heart that values are not externally validated. If you have complex values x and y, they might as well be simple ones s and t. I think there are real (and healthy) physical/biological barriers to realizing this, so that even intellectuals won’t become psychopaths.
But I agree with you it is how you phrase the question, and there are intuition pumps that pump the other way:
Futurists imagine a utopian future. But even if we achieved such, it wouldn’t change the past. Why should the future time be so elevated in importance? No, any perfect future would be marred by the fact that for thousands of years in the past there was human suffering.
If the future can’t be perfect due to the past, perhaps instead we could create a perfect sub-universe, where everything is perfect from beginning to end. Even if the outer universe can’t be utopian, it can simulate utopias.
Then you might realize that by wireheading, you are simulating a utopian universe and thereby doing your part (one consciousness at a time) to create utopias. Then the only moral reason not to wirehead is if you think you can have enough influence to create more subjective happiness by not wireheadong than with your consciousness wireheaded alone.
Sometimes, my thoughts bend solopsistic (or at least simulation-based) and I wonder if the universe I’m in is already optimised. I’m skeptical because I hear there is suffering, but perhaps that is some necessary negativity to optimize my existence (part and parcel with having ‘purpose’). I think I must actually believe this because stories of real suffering cause me intense disillusionment, as I don’t expect it to be real and when I try to imagine it being real there is this strong resistance. I observe that many people seem to feel this way, if they’re not in outright denial all the time about suffering being ‘real’.
I was thinking earlier today—I’ll just throw this in—that my intuition about values tends to be a little different than often described here because I feel that 1 consciousness with no suffering would be better than 10 consciousness with a mixed bag of experience. The only reason 10 consciousnesses might be better than 1 is because of their enjoyment of one another. So I guess I think quality is better than quantity, and also, I don’t distinguish among consciousnesses. 10 moments of consciousness can be distributed over 10 people or 1 person, it doesn’t matter. So if there really was a wire-heading machine, I might be convinced that not stepping in is equivalent to causing suffering to another person, equal to all the relative suffering I would encounter by not wireheading. However, I’m glad such choices are not available because—for some reason, as I said I think it is biological—wireheading just feels like a terrible negative choice, like entering a coffin. It feels isolated and not real.
If it’s healthy to not be a psychopath, on what values do you base that? I think you’re sneaking in a value judgment here that, if valid, would rule out wireheading.
(It might be evolutionary successful to not be a (full) psychopath, but that’s a very different matter.)
I do find your overall thought process in your first few paragraphs plausible, but “anyone who disagrees with me is just not admitting that I’m right” sounds way too much like the kind of toxic reasoning I’m trying to avoid, so I’m fairly skeptical of it.
Just in case, I don’t argue that people who say they don’t want to wirehead are wrong about that. I think it’s ultimately inconsistent with a full appreciation that values are not externally validated. I think this full appreciation is prevent by biological stop-guards.
Equivalence of Wire-Heading and Modifying Values As Giving Up On External Satisfaction of our Current Values
Something I think about in relation to wireheading, so close together in my brain that when talking about one I find myself conflating with the other, is that it should follow that if values aren’t externally validated, it should be equivalent to ‘make the world better’ by (a) changing the world to fit our values or by (b) changing our values to fit the world. We have a strong preference for the former, but we could modify this preference so (b) would seem just as good a solution. So by modifying their value about solutions (a) and (b), a person in theory could then self-modify to be perfectly happy with the universe as it is. This is equivalent to wireheading, because in both cases you have a perfectly happy person without altering the universe outside their brain.
What I think people don’t admit.
I think what ‘anyone who disagrees with you is not admitting’ is that the universe in which your values are altered to match reality (or in which a person chooses to wirehead) is just as good as any other universe.
Well, maybe they do admit it, but then their arational preference for their current values is not unlike a preference for wireheading.
The goodness of the universe is subjective, and for any subjective observer, the universe is good if it satisfies their preferences. Thus, a universe in which our values are modified to match the universe is just as good as our values modified to match the universe. I think that is clear.
However, people who don’t want to wirehead compare the universe (b) (one in which their values are modified but the universe is not) with the universe they currently prefer—I guess as they are supposed to—and decide that universe (b) is not as good—relative of course to their current set of values.
But I don’t understand their preference for their original set of preferences if they know these preferences aren’t actually, really, externally better. This is the contradiction I find: they insist that external reality is what matters to them, rather than happiness through wireheading. But preferring to want to prefer a set of values that have no external significance is preferring to live in a wire-headed universe, in the sense that the values of these preferences are just in their head after all.
One difference I suppose is that with respect to our preferences we’re wired by biology, which for now is hard-wired, whereas choosing to wirehead for happiness would be a choice. If we want to minimize the extent that we’re wired, we’d stick to a minimum set. In which case, as soon as we have the choice to shake off the yoke of biological preferences, we should self-modify ourselves into blissfully happy rocks (an inert object entirely satisfied with the way the universe currently is).
Yup, full agreement.
That’s exactly how it appears to me, though I’m not confident this is correct. It seems like others should’ve thought of the same thing, but then they shouldn’t disagree, which they do. So either this is far less convincing than I think (maybe these safeguards don’t work in my case) or it’s wrong. Dunno right now.
By ‘healthy’, I did mean evolutionarily successful. However, I wouldn’t go to great lengths to defend the statement, so I think you did catch me saying something I didn’t entirely mean.
Someone can be intellectual and emotionally detached at times, and this can help someone make more rational choices. However, if someone is too emotionally detached they don’t empathize with other people (or even themselves) and don’t care about their goals. So I meant something more general like apathy than lack of empathy. So my claim is that biological stop-guards prevent us from being too apathetic about external reality. (For example, when I imagine wireheading, I start empathizing with all the people I’m abandoning. In general, a person should feel the tug of all their unmet values and goals.)
Ok, then I misunderstood you and we do in fact agree.
What ever makes you happy! - Wait a minute. . .
No but seriously as been pointed out, one problem with wireheading is that it tend to resemble death in the sense that you stop being a person, since you stop caring about everything you used to care about (as well as act upon) such as finding out why other people don’t like the whole idea of head wiring, you are just in a kind of bliss-stasis. I don’t see much difference between me being head-wired and me being shot in the head, then someone/something building something that is put into bliss-stasis, since I will not have any psychological continuity I’m as dead in both cases. Of course this all depends on what your attitude towards death is, I tend to be strongly against it, cherishing “my” center of narrative gravity.
Maybe you have a different take on “the self” that resembles that of Blackmore’s? That in turn makes you more willing to get rid of it and become a wirehead.
I’m basically in complete agreement with Blackmore, yes. I also don’t consider wireheading death, at least not any more than any other event.
I’ve been in states of raw concentration that had no thought processes, no memory and no spatial or temporal perception going on, but I still perceived a specific emotion. I don’t think these states killed me, any more than blinking kills me. If it’s meaningful to say I’ve been the same person throughout meditation, then I’m the same person when wireheaded. (However, I would rather agree with Blackmore that no continuity exists, ever, though I suspect that would mostly be a disagreement about semantics.)
I don’t see how caring-about-the-narrative-center is essential to having-a-narrative-center. I can still tell fiction about a wirehead, even a static one. The wirehead themselves might not, but that doesn’t change the truth of the fiction. It seems to me that you can either reject this particular fiction (in which case I’d be interested in your reasons, not so much as justification, but more to understand how we came to differ), or you care about perceiving-the-fiction, independent of truth, in which case Omega will take care to include “this is making narrative sense” into your customized bliss.
(Disclaimer: I’m not endorsing muflax_june2011′s views anymore, though I’m still sympathetic to some forms of wireheading. muflax_june2011 would’ve just answered “screw being a person!” and rejected the whole psychological unity thing. muflax_october2011 is not so sure about that anymore.)
Well being in a state of raw concentration I consider somewhat as having your car in the garage, right now I’m not driving it, but it has the capabilities needed to be driven, capabilities that are regularly exercised, a person is not one brain/mental state but a repertoire of brain/mental states with diffrent temporal distribution. But I guess that you could ask Omega to configure the wiring machinery to have a multitude of states that retained to some degree what “I” would call “me”. My beef is with the whole idea of bliss-ing out of existence.
While there’s no particular reason not to go orgasmium, there’s also no particular reason to go there.
We already know that wanting and desiring are expressed by two different, although overlapping, neural circuitries, so it is natural that we may value things that doesn’t directly feedback pleasure. Let’s say that we find meaningful to be an active part of the world, that we assign a general utility to our actions as they effect the environment: descending into an orgasmium state would kill this effectiveness.
The general argument is that orgasmium can be considered a life of very much low complexity than our present, maybe unboundedly so, and we can, while we’re not forced to, value either the complexity of our present life or complexity per se. This is not akin to say that we value something outside our own state of the mind or saying that, once orgasmium, we would be tempted to opt-out. It means however that the process of turning “myself” into orgasmium, that is depleting complexity for the sake of pleasure, has very, very, very low utility attached to it.
It is possible of course to ask “why do you value complexity?”, “Has this preference any objectively grounded reason?”. I suspect that, even if a future eutopia would cancel all the value we have just because they are instrumental to survival, we would still hold some of those value because that’s how we are wired. There’s no particular reason other than the fact that those values are essentially what makes us human (not in a general sens, my values are what makes me human, your values are what makes you human, etc). If we do not accept this point of view, then orgasmium becomes just as viable as pleasure asymbolia, dronization and suicide (that is, descent to complexity 0).
In the choice between “some control over reality” and “maximum pleasure,” it seems to me that “maximum pleasure” comes highly recommended.
Ceteribus paribus yes, of course, but not if “achieve maximum pleasure” violates some moral code or drastically diminishes something else you might value, as could be in the case of orgasmium collapse.
I commented that because your first sentence seemed odd- there may be no one reason to not go orgasmium, but there’s only one reason to go orgasmium.
There are many situations over which I would prefer a wirehead state. For example I would prefer to be orgasmium than paperclips. But it isn’t exactly inspiring. I like novelty.
This is arbitrary.
Because any decision to self-modify will more likley than not be determined more by my future self.
It can’t be. Only by the current self.
If I sign up to be uploaded into a computer six months from now, which part of me made the decision? My current self is biased towards keeping previous commitments.
By the time technology and society change enough for self-modification to be a current self affair, current self will already be heavily modified.
Non the less. The current makes decisions in expecting something from the decisions made. There is no time reversal causality here.
Perhaps this is clearer formulation:
Because any decision to self-modify will more likley than not be determined more by my future self[’s values than my current self’s values].
I see wireheading as another problem that is the result of utility maximization. The question is, can utility be objectively grounded for an agent? If that is possible, wireheading might be objectively rational for a human utility maximizer.
Consider what it would mean if utility was ultimately measured in some unit of bodily sensations. We do what we do for what it does with us (our body (brain)). We do what we do because it makes us feel good, and bad if we don’t do it. It would be rational to fake the realization of our goals, to receive the good feelings with less effort and risk, if we were equally happy doing so and the utility that is the result of wireheading would outweigh the expected utility of actually realizing our goals.
But that doesn’t seem to be the case. Either humans are not utility maximizer or utility is not objectively grounded, i.e. humans assign arbitrary amounts of utility to arbitrary decisions and world states.
Why do people sacrifice their own life’s to save their loved ones? From the point of view of a utility maximizer with complex values this is hard to justify. After all it is unlikely that the utility you would be able to receive from the limited time of saving your loved one’s can outweigh the utility of the rest of your life without your loved one’s. Even if you think you could not live without someone, there are therapies. Such decisions only seem to make sense if humans are either not utility maximizer’s or are able to assign an infinite amount of utility to whatever they want to do based on naive introspection.
For example, I want to learn and understand as much as possible about the nature of reality. A way to objectively ground this desire, and thereby measure its expected utility, is by the amount of positive bodily sensations I receive from pursuing that goal. But such a payoff could probably be realized artificially (simulated) much more efficiently and with less risks (don’t have to create that high-energy particle accelerator for real). So if what I really want is to be happy, to feel good, then I should probably choose the simulated revelation over the real truth. But I would never do that, even if I knew that I would have to suffer quite often as a result of my desire to learn about the universe and how it works. I would even accept its destruction over the choice of being wireheaded to believe that I figured it all out.
I agree that if humans made decisions based on utility calculations that aren’t grounded in direct sensations, then that’d be a good argument against wireheading.
I see, however, no reason to believe that humans actually do such things, except that it would make utilitarianism look really neat and practical. (The fact that currently no-one actually manages to act based on utilitarianism of any kind seems like evidence against it.) It doesn’t look realistic to me. People rarely sacrifice themselves for causes and it always requires tons of social pressure. (Just look at suicide bombers.) Their actual motivations are much more nicely explained in terms of the sensations (anticipated and real) they get out of it. Assuming faulty reasoning, conflicting emotional demands and just plain confabulation for the messier cases seems like the simpler hypothesis, as we already know all those things exist and are the kinds of things evolution would produce.
Whenever I encounter a thought of the sort “I value X, objectively”, I always manage to dig into it and find the underlying sensations that give it that value. If it put them on hold (or realize that they are mistakenly attached, as X wouldn’t actually cause those sensations I expect), then that value disappears. I can see my values grounded in sensations, I can’t manage to find any others. Models based on that assumption seem to work just fine (like PCT), so I’m not sure I’m actually missing something.
So you want to wirehead. Do you think you’ll have access to that technology in your lifetime?
To be explicit about this, I don’t have an opinion on whether I’d choose it, but I do find it attractive. Just repeating this because everyone seems to think I’m advocating it and so I probably didn’t make this clear enough.
But your actual question:
Basically, I think it’s long here. The Tibetans in particular have developed some neat techniques that are still somewhat time-intensive to learn, but work reasonably well. The route (several specific forms of) meditation + basic hedonism seems like a decent implementation, especially because I already know most of the underlying techniques.
Also, MDMA and related drugs and basic implants already exist, though they’re still fairly crude and hard to sustain. I’d expect the technology for “good enough” wireheading through direct stimulation to be available in at most 20 years, though probably not commercially.
Chronic MDMA use causes a decrease in concentration of serotonin transporters.
Lottery winners end up no where near as happy, long-term, as they imagined they would be when they bought the ticket (Brickman, Coates, Janoff-Bulman 1978).
This is weak evidence, but it suggests that wire-heading in practice isn’t going to look like it does in the thought experiment—I imagine neural down-regulation would play a part.
You are right, muflax. There just isn’t any explanation for badness of the “wireheading”.
The case is the following:
Did you read this?
Neurologically, wanting and liking are two separate albeit very related things. When you measure liking and wanting, you find that you can manipulate the two separately.
If your dopamine receptors are blocked, you don’t want things as badly, even though your enjoy them equally well. If you increase dopamine, you (or experimental rats) work harder for something, even though you don’t enjoy it more when you get it.
I have the subjective impression that when I’m happy for no particularly external reason, I still want to do things that I previously wanted to do. So directly stimulated pleasure centers would be pleasant, but I’d still want external things.
See the link for specific experiments.
EDIT: Oops, I didn’t read carefully enough and didn’t consider wireheading which stimulated non-want parts of your brain. Please ignore this comment.
That was the first thing his list of arguments linked to.
I don’t think that addresses the substance of the argument. Wireheading doesn’t have to be about increasing dopamine; what if you were wireheaded to really, really like being wireheaded? And, in case it mattered, not to like anything else, so you don’t have any regrets about the wireheading.
The “Much Better Life” scenario is even more different; here, you presumably continue wanting and liking much the same things that you used to, unless you choose to self-modify later, you just get rid of the frustrating parts of life at the expense of no longer living in the real world.
I’ve been thinking about torture and the nature of my values. Many people here have defended the importance of external referents or complex desires. My problem is, I can’t understand these claims at all.
To clarify, I mean torture in the strict “collapsing into painium” sense. A successful implementation would identify all the punishment circuitry and directly stimulate it, or do something equivalent. It would essentially be a vastly improved box jellyfish. A good argument for either keeping complex values (e.g. by requiring at least a personal matrix) or external referents (e.g. by showing that a simulation can never suffice) would work for me.
Also, I use “punishment” as short-hand for any unpleasant feeling, as “pain” tends to be used for a specific one of them, among despair, fear and so on, and “it’s not about feeling X, but X and Y” is still wireheading after all.
I tried collecting all related arguments I could find. (Roughly sorted from weak to very weak, as I understand them, plus link to example instances. I also searched any literature/other sites I could think of, but didn’t find other (not blatantly incoherent) arguments.)
People do not always optimize their actions based on avoiding punishments. (People also are horrible at making predictions and great at rationalizing their failures afterwards.)
It is possible to hate doing something while wanting to continue or vice versa, do something despite hating it while wanting to continue. (Seriously? I can’t remember ever doing either. What makes you think that the action is thus valid, and you aren’t just making mistaken predictions about rewards or are being exploited? Also, Mind Projection Fallacy.)
A wireheaded “me” wouldn’t be “me” anymore. (What’s this “self” you’re talking about? Why does it matter that it’s preserved?)
“I don’t want it and that’s that.” (Why? What’s this “wanting” you do? How do you know what you “want”? (see end of post))
People, if given a hypothetical offer of being wireheaded, tend to refuse. (The exact result depends heavily on the exact question being asked. There are many biases at work here and we normally know better than to trust the majority intuition, so why should we trust it here?)
Far-mode predictions tend to favor complex, external actions, while near-mode predictions are simpler, more hedonistic. Our true self is the far one, not the near one. (Why? The opposite is equally plausible. Or the falsehood of the near/far model in general.)
If we imagine a wireheaded future, it feels like something is missing or like we won’t really be sad. (Intuition pump.)
It is not socially acceptable to embrace wireheading. (So what? Also, depends on the phrasing and society in question.)
(There have also been technical arguments against specific implementations of wireheading. I’m not concerned with those, as long as they don’t show impossibility.)
Overall, none of this sounds remotely plausible to me. Most of it is outright question-begging or relies on intuition pumps that don’t even work for me.
It confuses me that others might be convinced by arguments of this sort, so it seems likely that I have a fundamental misunderstanding or there are implicit assumptions I don’t see. I fear that I have a large inferential gap here, so please be explicit and assume I’m a Martian. I genuinely feel like Gamma in A Much Better Life.
To me, all this talk about “valueing something” sounds like someone talking about “feeling the presence of the Holy Ghost”. I don’t mean this in a derogatory way, but the pattern “sense something funny, therefore some very specific and otherwise unsupported claim” matches. How do you know it’s not just, you know, indigestion?
What is this “valuing”? How do you know that something is a “value”, terminal or not? How do you know what it’s about? How would you know if you were mistaken? What about unconscious hypocrisy or confabulation? Where do these “values” come from (i.e. what process creates them)? Overall, it sounds to me like people are confusing their feelings about (predicted) states of the world with caring about states directly.
To me, it seems like it’s all about anticipating and achieving rewards (and avoiding punishments, but for the sake of the wireheading argument, it’s equivalent). I make predicitions about what actions will trigger rewards (or instrumentally help me pursue those actions) and then engage in them. If my prediction was wrong, I drop the activity and try something else. If I “wanted” something, but getting it didn’t trigger a rewarding feeling, I wouldn’t take that as evidence that I “value” the activity for its own sake. I’d assume I suck at predicting or was ripped off.
Can someone give a reason why wireheading would be bad?