Taboo “simulation”. Whatever is, is real. Also we’re probably already in a simu… I mean, in a subset of the world systematically implementing surface laws different from deeper laws.
Is the problem that the Much-Better Life Simulator only simulates feelings and not their referents? Then Customer should say so. Is it that it involves chatbots rather than other complete people inside? Then Customer should say so. Is it that it loses complexity to cut simulating costs? Then Customer should say so. Is it that trees are allowed to be made of wood, which is allowed to be made of organic molecules, which are allowed to be made out of carbon, which is allowed to be made out of nuclei, which are allowed to be made out of quarks, which are not allowed to be made out of transistors? Then Customer hasn’t thought it through.
If you’re allowed to assign utility to events which you cannot perceive but can understand and anticipate, then you can assign a big negative utility to the “going up a simulation level” event.
I think that by “going up a simulation-level” you mean “increase your simulation-depth by one”, but intuitively to me “up” sounds like a pop rather than a push.
Sure you can, but I can’t see why you would. Reality is allowed to be made out of atoms, but not out of transistors? Why? (Well, control over the world outside the simulation matters too, but that’s mostly solved. Though we do get Lamed Vav deciding to stay on Earth to help people afford the Simulator instead of ascending to Buddhahood themselves! …hang on, I think I got my religions mixed up.)
Well, control over the world outside the simulation matters too, but that’s mostly solved
Really? If everyone hooked up to the sim at the same time then you might be right, but our current world seems pretty chaotic.
As to why I’d care about increasing my simulation-depth—essentially my mind runs on hardware which was designed to care about things in the outside world (specifically my reproductive fitness) more than it cares about its own state. I’m free to re-purpose that hardware how I like, but this kind of preference (for me at least) seems to be a kind of hangover from that.
Taboo “simulation”. Whatever is, is real. Also we’re probably already in a simu… I mean, in a subset of the world systematically implementing surface laws different from deeper laws.
We’re already in a subset of the world systematically implementing surface laws different from deeper laws. All the surface laws we see around us are implemented by atoms. (And subatomic particles, and fields, but “it’s all made of atoms” is Feynman’s way of summing up the idea.)
Where this differs from “simulations” is that there are no sentient beings at the atomic level, telling the atoms how to move to simulate us. This I think, is the issue with “simulations”—at least, it’s my issue. There is another world outside. If there is another world, I’d rather be out (however many levels of “out” it takes) than in. “In” is an ersatz, a fake experience under the absolute control of other people.
Nah, our surface rules ain’t systematic. I made a laser.
Agree direct puppet-style control is icky. Disagree that is what makes simulations simulatey, or that our own universe is a puppet-theatre-style simulation. If the Matrix masters were constantly deciding “let’s add obsidian to this character’s inventory”, we would be “under the absolute control of other people”, but instead they described physical laws and initial conditions and let the simulation unfold without intervention. I’m not particularly icked by that—the universe has to come from somewhere.
Would you still prefer to be “out” if you expected your life outside to be much, much worse than your life inside? Would you accept a lifetime of suffering to gain an iota of control over the “real” outside world?
(This is the reversal of the MBLS. Also, apologies for coming late to the discussion.)
Would you still prefer to be “out” if you expected your life outside to be much, much worse than your life inside?
I would prefer to act to make my life outside better.
Scaling the imaginary situation back to everyday matters, your question is like responding to the statement “I’m taking a holiday in Venice” with “but suppose you hate it when you get there?” Or responding to “I’m starting a new business” with “but suppose it fails?” Or responding to “I’m going out with someone new” with “but suppose she’s a serial killer?”
All you have done is imagine the scenario ending in failure. Why?
All you have done is imagine the scenario ending in failure. Why?
Because I’m building it to parallel the original question of whether you’d want to go into an MBLS. In both cases, your potential future life in the simulated or “inside” world is assumed to be much better than the one you might have in the simulating “outside” world. If you give different answers (inside vs. outside) in the two cases, why?
You said:
There is another world outside. If there is another world, I’d rather be out
As a reason for not entering the MBLS. Would that reason also make you want to escape from our current world to a much more dismal life in the simulating one? To me that would be a repugnant conclusion and is why I’d prefer a much better life in a simulated world, in both cases.
I would prefer to act to make my life outside better.
An individual’s control over their life, in our current world, is far below what I consider acceptable. People are stuck with sick bodies and suffering minds and bad relationships and die in unexpected or painful ways or, ultimately, of unavoidable old age. I would happily trade this for the MBLS experience which would surely offer much greater control.
Do you attach intrinsic value to affecting (even if weakly) the true ultimate level of reality, or do you disagree with my preference for a different reason? If the former, how would you deal with not knowing if we’re simulated, or infinite recursions of simulation, or scenarios where infinite numbers of worlds are simulated and simulate others? Would it mean you give high priority to discovering if we’re in a simulation and, if so, breaking out—at the expense of efforts to optimize our life in this world?
There is another world outside. If there is another world, I’d rather be out
As a reason for not entering the MBLS. Would that reason also make you want to escape from our current world to a much more dismal life in the simulating one? To me that would be a repugnant conclusion and is why I’d prefer a much better life in a simulated world, in both cases.
Both scenarios involve the scenario-setter putting their hand on one side of the scales and pushing hard enough to sway my preferences. You might as well ask if I would torture babies for a sufficiently high incentive. These questions are without significance. Ask me again when we actually have uploads and simulations. Meanwhile, strongly rigged scenarios can always beat strong hypothetical preferences, and vice versa. It just becomes a contest over who can name the biggest number.
how would you deal with not knowing if we’re simulated, or infinite recursions of simulation, or scenarios where infinite numbers of worlds are simulated and simulate others?
I don’t take such speculations seriously. I’ve read the arguments for why we’re probably living in a simulation and am unimpressed; I am certainly not going to be mugged à la Pascal into spending any substantial effort considering the matter.
Utility maximization destroys complex values by choosing the value that yields the most utility, i.e. the best cost-value ratio. One unit of utility is not discriminable from another unit of utility. All a utility maximizer can do is to maximize expected utility. If it turns out that one of its complex values can be effectively realized and optimized, it might turn out to outweigh all other values. This can only be countered by changing one’s utility function and reassign utility in such a way as to outweigh that effect, which will lead to inconsistency, or by discounting the value that threatens to outweigh all others, which will again lead to inconsistency.
Can’t your utility function look like “number of paperclips times number of funny jokes” rather than a linear combination? Then situations where you accept very little humor in exchange for loads of paperclips are much rarer.
Relevant intuition: this trade-off makes me feel sad, so it can’t be what I really want. And I hear it’s proven that wanting can only work if it involves maximizing a function over the state of the universe.
For this reason, I almost wish LW would stop talking about utility functions entirely.
That it is theoretically possible for functions to be arbitrarily complex does not seem to be a good reason to reject using a specific kind of function. Most information representation formats can be arbitrary complex. That’s what they do.
(This is to say that while I respect your preference for not talking about utility functions your actual reasons are probably better than because utility functions can be arbitrarily complex.)
Right, sorry. The reason I meant was something like “utlity functions can be arbitrarily complex and in practice are extremely complex, but this is frequently ignored”, what with talk about “what utility do you assign to a firm handshake” or the like.
Edit: And while they have useful mathematical features in the abstract, they seem to become prohibitively complex when modeling the preferences of things like humans.
...what with talk about “what utility do you assign to a firm handshake” or the like.
World states are not uniform entities, but compounds of different items, different features, each adding a certain amount of utility, weight to the overall value of the world state. If you only consider utility preferences between world states that are not made up of all the items of your utility-function, then isn’t this a dramatic oversimplification? I don’t see what is wrong in asking how you weigh firm handshakes. A world state that features firm handshakes must be different from one that doesn’t feature firm handshakes, even if the difference is tiny. So if I ask how much utility you assign to firm handshakes I ask how you weigh firm handshakes, how the absence of firm handshakes would affect the value of a world state. I ask about your utility preferences between possible world states that feature firm handshakes and those that don’t.
World states are not uniform entities, but compounds of different items, different features, each adding a certain amount of utility, weight to the overall value of the world state. If you only consider utility preferences between world states that are not made up of all the items of your utility-function, then isn’t this a dramatic oversimplification?
So far as I can tell, you have it backwards—those sorts of functions form a subset of the set of utility functions.
The problem is that utility functions that are easy to think about are ridiculously simple, and produce behavior like the above “maximize one value” or “tile the universe with ‘like’ buttons”. They’re characterized by “Handshake = (5*firmness_quotient) UTILS” or “Slice of Cheesecake = 32 UTILS” or what have you.
I’m sure it’s possible to discuss utility functions without falling into these traps, but I don’t think we do that, except in the vaguest cases.
There are very few instances in which I would ask “what utility do you assign?” regarding a concrete, non-contrived good. I tend to consider utility preferences between possible world states that could arise depending on a specific decision or event and then only consider actual numbers if actually necessary for the purpose of multiplying.
I would certainly prefer to limit use of the term to those who actually understand what it means!
There are very few instances in which I would ask “what utility do you assign?” regarding a concrete, non-contrived good.
Exactly. Perhaps if we used a different model (or an explicitly spelled-out simplified subset of the utility functions) we could talk about such things.
I would certainly prefer to limit use of the term to those who actually understand what it means!
But if you do not “assign utility” and only consider world states, how do you deal with novel discoveries? How does a hunter gatherer integrate category theory into its utility function? I mean, you have to somehow weigh new items?
I just go ahead and assign value directly to “novelty” and “variety”.
Isn’t that too unspecific? Every sequence of digits of the variety of transcendental numbers can be transcribed into musical scores. Or you could use cellular automata to create endless amounts of novel music. But that is not what you mean. If I asked you for a concrete example you could only tell me something that you already expect but are not sure of, which isn’t really novel, or say that you will be able to point out novelty in retrospect. But even with the latter answer there is a fundamental problem, because novelty can’t be crowned in retrospect if you are able to recognize it. In other words, it is predictable what will excite you and make you label something n-o-v-e-l. In this respect what you call “novelty” is just like the creation of music by the computation of the sequences of transcendental numbers, uncertain but ultimately computable. My point, to assign value to “novelty” and “variety” can not replace the assignment of utility to discrete sequences that make interesting music. You have to weigh discrete items, because those that are sufficiently described by “novelty” and “variety” are just random noise.
You have to weigh discrete items, because those that are sufficiently described by “novelty” and “variety” are just random noise.
Continuous random noise is quite monotonous to experience—the opposite of varied. I didn’t say that variety and novelty were my only values, just that I assign value to them. I value good music, too, as well as food and other pleasant stimuli. The theory of diminishing returns comes into play, often caused by the facility of the human mind to attain boredom. I view this as a value continuum rather than a set value.
In my mind, I’m picturing one of those bar graphs that show up when music is playing, except instead of music, it’s my mind and body moving throughout the day, and each bar represents my value of particular things in the world, with new bars added and old ones dying off, and… well, it’s way more complex than, “assign value K to music notes XYZ and call it done.” And several times I’ve been rebuked for using the phrase “assign value to something”, as opposed to “discover value as already-implemented by my brain”.
(Note that I myself do not subscribe to wireheading, I am merely trying to fathom the possible thought processes of those that subscribe to it.)
You are right. But the basic point is that if you are human and subscribe to rational, consistent, unbounded utility maximization, then you assign at least non-negligible utility to unconditional bodily sensations. If you further accept uploading and that emulations can experience more in a shorter period of time compared to fleshly humans, then it is a serious possibility that you can outweigh the extra utility you assign to the referents of rewards in the form of bodily sensations and other differences like chatbots instead of real agents (a fact that you can choose to forget).
I believe the gist of the matter to be that wireheading appears to its proponents to be the rational choice for an utility maximizing agent which is the effect of biological evolution within our universe. For what it’s worth, this could be an explanation for the Fermi paradox.
Taboo “simulation”. Whatever is, is real. Also we’re probably already in a simu… I mean, in a subset of the world systematically implementing surface laws different from deeper laws.
Is the problem that the Much-Better Life Simulator only simulates feelings and not their referents? Then Customer should say so. Is it that it involves chatbots rather than other complete people inside? Then Customer should say so. Is it that it loses complexity to cut simulating costs? Then Customer should say so. Is it that trees are allowed to be made of wood, which is allowed to be made of organic molecules, which are allowed to be made out of carbon, which is allowed to be made out of nuclei, which are allowed to be made out of quarks, which are not allowed to be made out of transistors? Then Customer hasn’t thought it through.
If you’re allowed to assign utility to events which you cannot perceive but can understand and anticipate, then you can assign a big negative utility to the “going up a simulation level” event.
EDIT: What Pavitra said. I guess I was thinking of turtles all the way down
I think that by “going up a simulation-level” you mean “increase your simulation-depth by one”, but intuitively to me “up” sounds like a pop rather than a push.
Sure you can, but I can’t see why you would. Reality is allowed to be made out of atoms, but not out of transistors? Why? (Well, control over the world outside the simulation matters too, but that’s mostly solved. Though we do get Lamed Vav deciding to stay on Earth to help people afford the Simulator instead of ascending to Buddhahood themselves! …hang on, I think I got my religions mixed up.)
Really? If everyone hooked up to the sim at the same time then you might be right, but our current world seems pretty chaotic.
As to why I’d care about increasing my simulation-depth—essentially my mind runs on hardware which was designed to care about things in the outside world (specifically my reproductive fitness) more than it cares about its own state. I’m free to re-purpose that hardware how I like, but this kind of preference (for me at least) seems to be a kind of hangover from that.
We’re already in a subset of the world systematically implementing surface laws different from deeper laws. All the surface laws we see around us are implemented by atoms. (And subatomic particles, and fields, but “it’s all made of atoms” is Feynman’s way of summing up the idea.)
Where this differs from “simulations” is that there are no sentient beings at the atomic level, telling the atoms how to move to simulate us. This I think, is the issue with “simulations”—at least, it’s my issue. There is another world outside. If there is another world, I’d rather be out (however many levels of “out” it takes) than in. “In” is an ersatz, a fake experience under the absolute control of other people.
Nah, our surface rules ain’t systematic. I made a laser.
Agree direct puppet-style control is icky. Disagree that is what makes simulations simulatey, or that our own universe is a puppet-theatre-style simulation. If the Matrix masters were constantly deciding “let’s add obsidian to this character’s inventory”, we would be “under the absolute control of other people”, but instead they described physical laws and initial conditions and let the simulation unfold without intervention. I’m not particularly icked by that—the universe has to come from somewhere.
Many people now are happy to see the Hand of Whatever involved in our daily lives. I am not sure that is a problem with hypothetical simulations.
Would you still prefer to be “out” if you expected your life outside to be much, much worse than your life inside? Would you accept a lifetime of suffering to gain an iota of control over the “real” outside world?
(This is the reversal of the MBLS. Also, apologies for coming late to the discussion.)
I would prefer to act to make my life outside better.
Scaling the imaginary situation back to everyday matters, your question is like responding to the statement “I’m taking a holiday in Venice” with “but suppose you hate it when you get there?” Or responding to “I’m starting a new business” with “but suppose it fails?” Or responding to “I’m going out with someone new” with “but suppose she’s a serial killer?”
All you have done is imagine the scenario ending in failure. Why?
Because I’m building it to parallel the original question of whether you’d want to go into an MBLS. In both cases, your potential future life in the simulated or “inside” world is assumed to be much better than the one you might have in the simulating “outside” world. If you give different answers (inside vs. outside) in the two cases, why?
You said:
As a reason for not entering the MBLS. Would that reason also make you want to escape from our current world to a much more dismal life in the simulating one? To me that would be a repugnant conclusion and is why I’d prefer a much better life in a simulated world, in both cases.
An individual’s control over their life, in our current world, is far below what I consider acceptable. People are stuck with sick bodies and suffering minds and bad relationships and die in unexpected or painful ways or, ultimately, of unavoidable old age. I would happily trade this for the MBLS experience which would surely offer much greater control.
Do you attach intrinsic value to affecting (even if weakly) the true ultimate level of reality, or do you disagree with my preference for a different reason? If the former, how would you deal with not knowing if we’re simulated, or infinite recursions of simulation, or scenarios where infinite numbers of worlds are simulated and simulate others? Would it mean you give high priority to discovering if we’re in a simulation and, if so, breaking out—at the expense of efforts to optimize our life in this world?
Both scenarios involve the scenario-setter putting their hand on one side of the scales and pushing hard enough to sway my preferences. You might as well ask if I would torture babies for a sufficiently high incentive. These questions are without significance. Ask me again when we actually have uploads and simulations. Meanwhile, strongly rigged scenarios can always beat strong hypothetical preferences, and vice versa. It just becomes a contest over who can name the biggest number.
I don’t take such speculations seriously. I’ve read the arguments for why we’re probably living in a simulation and am unimpressed; I am certainly not going to be mugged à la Pascal into spending any substantial effort considering the matter.
I want to rephrase my last comment:
Utility maximization destroys complex values by choosing the value that yields the most utility, i.e. the best cost-value ratio. One unit of utility is not discriminable from another unit of utility. All a utility maximizer can do is to maximize expected utility. If it turns out that one of its complex values can be effectively realized and optimized, it might turn out to outweigh all other values. This can only be countered by changing one’s utility function and reassign utility in such a way as to outweigh that effect, which will lead to inconsistency, or by discounting the value that threatens to outweigh all others, which will again lead to inconsistency.
Can’t your utility function look like “number of paperclips times number of funny jokes” rather than a linear combination? Then situations where you accept very little humor in exchange for loads of paperclips are much rarer.
Relevant intuition: this trade-off makes me feel sad, so it can’t be what I really want. And I hear it’s proven that wanting can only work if it involves maximizing a function over the state of the universe.
No, it doesn’t. A utility function can be as complex as you want it to be. In fact it can be more complex than is possible to represent in universe.
For this reason, I almost wish LW would stop talking about utility functions entirely.
That it is theoretically possible for functions to be arbitrarily complex does not seem to be a good reason to reject using a specific kind of function. Most information representation formats can be arbitrary complex. That’s what they do.
(This is to say that while I respect your preference for not talking about utility functions your actual reasons are probably better than because utility functions can be arbitrarily complex.)
Right, sorry. The reason I meant was something like “utlity functions can be arbitrarily complex and in practice are extremely complex, but this is frequently ignored”, what with talk about “what utility do you assign to a firm handshake” or the like.
Edit: And while they have useful mathematical features in the abstract, they seem to become prohibitively complex when modeling the preferences of things like humans.
World states are not uniform entities, but compounds of different items, different features, each adding a certain amount of utility, weight to the overall value of the world state. If you only consider utility preferences between world states that are not made up of all the items of your utility-function, then isn’t this a dramatic oversimplification? I don’t see what is wrong in asking how you weigh firm handshakes. A world state that features firm handshakes must be different from one that doesn’t feature firm handshakes, even if the difference is tiny. So if I ask how much utility you assign to firm handshakes I ask how you weigh firm handshakes, how the absence of firm handshakes would affect the value of a world state. I ask about your utility preferences between possible world states that feature firm handshakes and those that don’t.
So far as I can tell, you have it backwards—those sorts of functions form a subset of the set of utility functions.
The problem is that utility functions that are easy to think about are ridiculously simple, and produce behavior like the above “maximize one value” or “tile the universe with ‘like’ buttons”. They’re characterized by “Handshake = (5*firmness_quotient) UTILS” or “Slice of Cheesecake = 32 UTILS” or what have you.
I’m sure it’s possible to discuss utility functions without falling into these traps, but I don’t think we do that, except in the vaguest cases.
Ick. Yes. That question makes (almost) no sense.
There are very few instances in which I would ask “what utility do you assign?” regarding a concrete, non-contrived good. I tend to consider utility preferences between possible world states that could arise depending on a specific decision or event and then only consider actual numbers if actually necessary for the purpose of multiplying.
I would certainly prefer to limit use of the term to those who actually understand what it means!
Exactly. Perhaps if we used a different model (or an explicitly spelled-out simplified subset of the utility functions) we could talk about such things.
Inconceivable!
But if you do not “assign utility” and only consider world states, how do you deal with novel discoveries? How does a hunter gatherer integrate category theory into its utility function? I mean, you have to somehow weigh new items?
I just go ahead and assign value directly to “novelty” and “variety”.
Isn’t that too unspecific? Every sequence of digits of the variety of transcendental numbers can be transcribed into musical scores. Or you could use cellular automata to create endless amounts of novel music. But that is not what you mean. If I asked you for a concrete example you could only tell me something that you already expect but are not sure of, which isn’t really novel, or say that you will be able to point out novelty in retrospect. But even with the latter answer there is a fundamental problem, because novelty can’t be crowned in retrospect if you are able to recognize it. In other words, it is predictable what will excite you and make you label something n-o-v-e-l. In this respect what you call “novelty” is just like the creation of music by the computation of the sequences of transcendental numbers, uncertain but ultimately computable. My point, to assign value to “novelty” and “variety” can not replace the assignment of utility to discrete sequences that make interesting music. You have to weigh discrete items, because those that are sufficiently described by “novelty” and “variety” are just random noise.
Continuous random noise is quite monotonous to experience—the opposite of varied. I didn’t say that variety and novelty were my only values, just that I assign value to them. I value good music, too, as well as food and other pleasant stimuli. The theory of diminishing returns comes into play, often caused by the facility of the human mind to attain boredom. I view this as a value continuum rather than a set value.
In my mind, I’m picturing one of those bar graphs that show up when music is playing, except instead of music, it’s my mind and body moving throughout the day, and each bar represents my value of particular things in the world, with new bars added and old ones dying off, and… well, it’s way more complex than, “assign value K to music notes XYZ and call it done.” And several times I’ve been rebuked for using the phrase “assign value to something”, as opposed to “discover value as already-implemented by my brain”.
Does not follow necessary. A larger plethora of values can be the greatest utility.
I don’t say that it must always be so. But it can be constructed that way.
(Note that I myself do not subscribe to wireheading, I am merely trying to fathom the possible thought processes of those that subscribe to it.)
You are right. But the basic point is that if you are human and subscribe to rational, consistent, unbounded utility maximization, then you assign at least non-negligible utility to unconditional bodily sensations. If you further accept uploading and that emulations can experience more in a shorter period of time compared to fleshly humans, then it is a serious possibility that you can outweigh the extra utility you assign to the referents of rewards in the form of bodily sensations and other differences like chatbots instead of real agents (a fact that you can choose to forget).
I believe the gist of the matter to be that wireheading appears to its proponents to be the rational choice for an utility maximizing agent which is the effect of biological evolution within our universe. For what it’s worth, this could be an explanation for the Fermi paradox.