Reality is whatever you should consider relevant. Even exact simulations of your behavior can still be irrelevant (as considerations that should influence your thoughts and decisions, and consequently the thoughts and decisions of those simulations), similarly to someone you won’t possibly interact with thinking about your behavior, or writing down a large natural number that encodes your mind, as it stands now or in an hour in response to some thought experiment.
So it’s misleading to say that you exist primarily as the majority of your instances (somewhere in the bowels of an algorithmic prior), because you plausibly shouldn’t care about what’s happening with the majority of your instances (which is to say, those instances shouldn’t care about what’s happening to them), and so a more useful notion of where you exist won’t be about them. We can still consider these other instances, but I’m objecting to framing their locations as the proper meaning of “our reality”. My reality is the physical world, the base reality, because this is what seems to be the thing I should care about for now (at least until I can imagine other areas of concern more clearly, something that likely needs more than a human mind, and certainly needs a better understanding of agent foundations).
I think your position can be oversimplified as follows: ‘Being in a simulation’ makes sense only if it has practical, observable differences. But as most simulations closely match the base world, there are no observable differences. So the claim has no meaning.
However, in our case, this isn’t true. The fact that we know we are in a simulation ‘destroys’ the simulation, and thus its owners may turn it off or delete those who come too close to discovering they are in a simulation. If I care about the sudden non-existence of my instance, this can be a problem.
Moreover, if the alien simulation idea is valid, they are simulating possible or even hypothetical worlds, so there are no copies of me in base reality, as there is no relevant base reality (excluding infinite multiverse scenarios here).
Also, being in an AI-testing simulation has observable consequences for me: I am more likely to observe strange variations of world history or play a role in the success or failure of AI alignment efforts.
If I know that I am simulated for some purpose, the only thing that matters is what conclusions I prefer the simulation owners will make. But it is not clear to me now, in the case of an alien simulation, what I should want.
One more consideration is what I call meta-simulation: a simulation in which the owners are testing the ability of simulated minds to guess that they are in a simulation and hack it from inside.
TLDR: If I know that I am in simulation, simulation+owners is my base reality that matters.
I don’t think it’s clear that knowing we’re in a simulation “destroys” the simulation. This assumes that belief by the occupants of the simulation that they are being simulated creates an invalidating difference from the desired reference class of plausible pre-singularity civilizations, but I don’t think that’s true:
Actual, unsimulated, pre-singularity civilizations are in similar epistemic positions to us and thus many of their influential occupants may wrongly but rationally believe they are simulated, which may affect the trajectory of the development of their ASI. So knowing the effects of simulation beliefs is important for modeling actual ASIs.
This is true only if we assume that a base reality for our civilization exists at all. But knowing that we are in a simulation shifts the main utility of our existence, which Nesov wrote about above.
For example, if in some simulation we can break out, this would be a more important event than what is happening in the base reality where we likely go extinct anyway.
And as the proportion of simulations is very large, even a small chance to break away from inside a simulation, perhaps via negotiation with its owners, has more utility than focusing on base reality.
I agree with you, I think, but I don’t think your primary argument is relevant to this post? It’s arguing that your “physical” current reality is a simulation run for specific reasons. That is quite possibly highly relevant by your criteria, because it could have very large implications for how you should behave tomorrow. The simulation argument doesn’t mean it’s an atom by atom simulation identical to the world if it were “real” and physical. Just the possible halting criteria might change your behavior if you found it plausible, for instance, and there’s no telling what else you might conclude is likely enough to change your behavior.
By your theory, if you believe that we are near to the singularity how should we update on the likelihood that we exist at such an incredibly important time?
We can directly observe the current situation that’s already trained into our minds, that’s clearly where we are (since there is no legible preference to tell us otherwise, that we should primarily or at least significantly care about other things instead, which in principle there could be, and so on superintelligent reflection we might develop such claims). Updatelessly we can ask which situations are more likely a priori, to formulate more global commitments (to listen to particular computations) that coordinate across many situations, where the current situation is only one of the possibilities. But the situations are possible worlds, not possible locations/instances of your mind. The same world can have multiple instances of your mind (in practice most importantly because other minds are reasoning about you, but also it’s easy to set up concretely for digital minds), and that world shouldn’t be double-counted for the purposes of deciding what to do, because all these instances within one world will be acting jointly to shape this same world, they won’t be acting to shape multiple worlds, one for each instance.
And so the probabilities of situations are probabilities of the possible worlds that contain your mind, not probabilities of your mind being in a particular place within those worlds. I think the notion of the probability of your mind being in a particular place doesn’t make sense (it’s not straightforwardly a decision relevant thing formulating part of preference data, the way probability of a possible world is), it conflates the uncertainty about a possible world and uncertainty about location within a possible world.
Possibly this originates from the imagery of a possible world being a location in some wider multiverse that contains many possible worlds, similarly to how instances of a mind are located in some wider possible world. But even in a multiverse, multiple instances of a mind (existing across multiple possible worlds) shouldn’t double-count the possible worlds, and so they shouldn’t ask about the probability of being in a particular possible world of the multiverse, instead they should be asking about the probability of that possible world itself (which can be used synonymously, but conceptually there is a subtle difference, and this conflation might be contributing to the temptation to ask about probability of being in a particular situation instead of asking about probability of the possible worlds with that particular situation, even though there doesn’t seem to be a principled reason to consider such a thing).
Trying to break out of simulation is a different game than preventing x-risks in base world, and may have even higher utility if we expect almost inevitable extinction.
Reality is whatever you should consider relevant. Even exact simulations of your behavior can still be irrelevant (as considerations that should influence your thoughts and decisions, and consequently the thoughts and decisions of those simulations), similarly to someone you won’t possibly interact with thinking about your behavior, or writing down a large natural number that encodes your mind, as it stands now or in an hour in response to some thought experiment.
So it’s misleading to say that you exist primarily as the majority of your instances (somewhere in the bowels of an algorithmic prior), because you plausibly shouldn’t care about what’s happening with the majority of your instances (which is to say, those instances shouldn’t care about what’s happening to them), and so a more useful notion of where you exist won’t be about them. We can still consider these other instances, but I’m objecting to framing their locations as the proper meaning of “our reality”. My reality is the physical world, the base reality, because this is what seems to be the thing I should care about for now (at least until I can imagine other areas of concern more clearly, something that likely needs more than a human mind, and certainly needs a better understanding of agent foundations).
I think your position can be oversimplified as follows: ‘Being in a simulation’ makes sense only if it has practical, observable differences. But as most simulations closely match the base world, there are no observable differences. So the claim has no meaning.
However, in our case, this isn’t true. The fact that we know we are in a simulation ‘destroys’ the simulation, and thus its owners may turn it off or delete those who come too close to discovering they are in a simulation. If I care about the sudden non-existence of my instance, this can be a problem.
Moreover, if the alien simulation idea is valid, they are simulating possible or even hypothetical worlds, so there are no copies of me in base reality, as there is no relevant base reality (excluding infinite multiverse scenarios here).
Also, being in an AI-testing simulation has observable consequences for me: I am more likely to observe strange variations of world history or play a role in the success or failure of AI alignment efforts.
If I know that I am simulated for some purpose, the only thing that matters is what conclusions I prefer the simulation owners will make. But it is not clear to me now, in the case of an alien simulation, what I should want.
One more consideration is what I call meta-simulation: a simulation in which the owners are testing the ability of simulated minds to guess that they are in a simulation and hack it from inside.
TLDR: If I know that I am in simulation, simulation+owners is my base reality that matters.
I don’t think it’s clear that knowing we’re in a simulation “destroys” the simulation. This assumes that belief by the occupants of the simulation that they are being simulated creates an invalidating difference from the desired reference class of plausible pre-singularity civilizations, but I don’t think that’s true:
Actual, unsimulated, pre-singularity civilizations are in similar epistemic positions to us and thus many of their influential occupants may wrongly but rationally believe they are simulated, which may affect the trajectory of the development of their ASI. So knowing the effects of simulation beliefs is important for modeling actual ASIs.
This is true only if we assume that a base reality for our civilization exists at all. But knowing that we are in a simulation shifts the main utility of our existence, which Nesov wrote about above.
For example, if in some simulation we can break out, this would be a more important event than what is happening in the base reality where we likely go extinct anyway.
And as the proportion of simulations is very large, even a small chance to break away from inside a simulation, perhaps via negotiation with its owners, has more utility than focusing on base reality.
This post by EY is about breaking out of a simulationhttps://www.lesswrong.com/posts/5wMcKNAwB6X4mp9og/that-alien-message
I agree with you, I think, but I don’t think your primary argument is relevant to this post? It’s arguing that your “physical” current reality is a simulation run for specific reasons. That is quite possibly highly relevant by your criteria, because it could have very large implications for how you should behave tomorrow. The simulation argument doesn’t mean it’s an atom by atom simulation identical to the world if it were “real” and physical. Just the possible halting criteria might change your behavior if you found it plausible, for instance, and there’s no telling what else you might conclude is likely enough to change your behavior.
By your theory, if you believe that we are near to the singularity how should we update on the likelihood that we exist at such an incredibly important time?
We can directly observe the current situation that’s already trained into our minds, that’s clearly where we are (since there is no legible preference to tell us otherwise, that we should primarily or at least significantly care about other things instead, which in principle there could be, and so on superintelligent reflection we might develop such claims). Updatelessly we can ask which situations are more likely a priori, to formulate more global commitments (to listen to particular computations) that coordinate across many situations, where the current situation is only one of the possibilities. But the situations are possible worlds, not possible locations/instances of your mind. The same world can have multiple instances of your mind (in practice most importantly because other minds are reasoning about you, but also it’s easy to set up concretely for digital minds), and that world shouldn’t be double-counted for the purposes of deciding what to do, because all these instances within one world will be acting jointly to shape this same world, they won’t be acting to shape multiple worlds, one for each instance.
And so the probabilities of situations are probabilities of the possible worlds that contain your mind, not probabilities of your mind being in a particular place within those worlds. I think the notion of the probability of your mind being in a particular place doesn’t make sense (it’s not straightforwardly a decision relevant thing formulating part of preference data, the way probability of a possible world is), it conflates the uncertainty about a possible world and uncertainty about location within a possible world.
Possibly this originates from the imagery of a possible world being a location in some wider multiverse that contains many possible worlds, similarly to how instances of a mind are located in some wider possible world. But even in a multiverse, multiple instances of a mind (existing across multiple possible worlds) shouldn’t double-count the possible worlds, and so they shouldn’t ask about the probability of being in a particular possible world of the multiverse, instead they should be asking about the probability of that possible world itself (which can be used synonymously, but conceptually there is a subtle difference, and this conflation might be contributing to the temptation to ask about probability of being in a particular situation instead of asking about probability of the possible worlds with that particular situation, even though there doesn’t seem to be a principled reason to consider such a thing).
Trying to break out of simulation is a different game than preventing x-risks in base world, and may have even higher utility if we expect almost inevitable extinction.