Right. Most poeple disagree with the premise “Being in a simulation is/can be made to be indistinguishable from reality from the point of view of the simulee.”
Edit: And by most people, I mean my analysis of why I intuitively reject the conclusion, not any discussion with other, let alone 3.6+ billion people or statistically representative sampling, etc.
Most poeple disagree with the premise “Being in a simulation is/can be made to be indistinguishable from reality from the point of view of the simulee.”
I am surprised to hear this. What is your basis for claiming that this is the premise most people object to?
Also, if you are aware of or familiar with this objection—would you mind explaining the following questions I have regarding it?
What reason is there to suspect that a simulated me would have a different/distinguishable experience from real me?
What reason is there to suspect that if there were differences between simulated and real life, that a simulated life would be aware of those differences? That is, even if it is distinguishable—I have only experienced one kind of life and can’t say if my totally distinguishable experience of life is that of a simulated life or a real one.
A magic super computer from the future will be able to simulate one atom with arbitrary accuracy—right? A super-enough computer will be able to simulate many atoms interacting with arbitrary accuracy. If this super computer is precisely simulating all the atoms of an empty room containing a single human being (brain included). If this simulation is happening—how could the simulated being possibly have a different experience than its real counterpart in an empty room? Atomically speaking everything is identical.
Maybe questions 1 and 3 are similar—but I’d appreciate if you (or someone else) could enlighten me regarding these issues.
What reason is there to suspect that a simulated me would have a different/distinguishable experience from real me?
As someone who has written lots of simulations, there are a few reasons.
1) The simulation deliberately simplifies or changes some things from reality. At minimum, when “noise” is required an algorithm is used to generate numbers which have many of the properties of random numbers but
a) are not in fact random,
b) are usually much more accurately described by a particular mathematical distribution than would any measurements of the actual noise in the system be.
2) The simulation accidentally simplifies/changes LOTS of things from reality. A brain simulation at the neuron level is likely to simulate observed variations using a noise generator, when these variations arise from a) a ream of detailed motions of individual ions and b) quantum interactions. The claim is generally made that one can simulate at a more and more detailed level AND GET TO THE ENDPOINT where the simulation is “perfect.” The getting to the endpoint claim is not only unproven, but highly suspect. At every level of physics we have investigated so far, we have always found a deeper level. Further, the equations of motions at these deepest layers are not known in complete detail. So even if we can get to an endpoint, we have no reason to believe we have gotten to the endpoint in any given simulation. At some point, we are no longer compute bound, we are knowledge bound.
3) There is a great insight in software that “if it isn’t tested, its broken.” How do you even test a supremely deep simulation of yourself? If there are features of yourself you are still learning, you can’t test for them. Until you comprehensievly comprehend yourself, you can never know that a simulation was comprehensively similar.
Even something as simple as a coin toss simulation is likely to be “wrong” in detail. Perhaps you know the coin toss you are actually simulating has .500 or even .500000000 probability of giving heads (where number of zeros represents accuracy to which you know it.) But what is your confidence that the true expectation is 0.5 with a googleplex zeros following (or 3^^3 zeros to pretend to try to fit in here) is the experimental fact? Even 64 zeros would be a bitch to prove. And what are the chances that your simulation gets a ’true expectation” of 0.5 with even 64 zeros after it? With the coin toss, the variance might SEEM trivial, but consider the same uncertainty in the human. You need to predict my next post keystroke for keystroke, which necessarily includes a prediction of whether I will eat an egg for breakfast or a bowel of cereal because the posts I read while eating depend on that. And so on and so on.
My claim is that the existence of an endpoint in finally getting the simulation complete is at best vastly beyond our knowledge (and not in a compute bound way) and at worst simply unknowable for a ream of good reasons. My estimate of the probability that a simulation will ever be reliably known is < 0.01%.
Now we may get to a much easier place: good enough to convince others. That someone can write a simulation of me that cannot be distinguished from me by people who know me is a much lower bar than that the simulatino feels the same as me to itself. To convince others, the simulation may not even have to be conscious, for example. But you are going to have to build your simulation in to a fat human body good enough to fool my wife, and give it a variety of nervous and personality disorders that cause it to come up with digs that are deeply disturbing to her to do even that.
At some point, the comprehensive difficulty of a problem has to open the question: is it reasonable to sweep this under the rug by appealing to an unknown future of much greater capability than we have now, or is doing that a human bias we may need to avoid?
I think enough people are non-reductionist/materialist to have doubt about whether a simulation can be said to have experiences. And we don’t exactly have demonstration of this at this time, do we? I mean, in the example cited, Cvilization PC games, there aren’t individuals there to have experiences (unless one counts the ai which is running the entire faction), rather there are some blips in databases incrementing the number of units here or there, or raising the population from an abstract 6 to 7. I don’t think people will be able to take simulation theory seriously until they have personal interaction with a convincing ai.
That’s probably as much an answer as I can give for any of the questsions, other than that I don’t see why we can assume that magic super computers are plausible. Related, I don’t know if I trust my intuition or reasoning as to whether an infinite simulation will resemble realty in every way (assuming the supercomputer is running equations and databases, etc, rather than actually reconstucting a universe atom by atom or something).
It feels like you’re asking me to believe that a map is the same as the territory if it is a good enough map. I know that’s just an analogy, but I have a hard time comprehending the sentence that “reality is the solution to/ equations and nothing more” (as opposed to even “reality is predictable by equations”).
This is probably not the LW approved answer, but then, I did say most people and not most LW-ers.
I don’t understand subjective experience very well, so I don’t know if a simulation would have it. I know that an adult human brain does, and I’m pretty sure a rock doesn’t, but there are other cases I’m much less certain about. Mice, for example.
Yes, and this isn’t necessarily due to naive ‘no mere simulation could feel so real’-type thinking either. Once can make a decent argument that the only known method of making a simulation that can fool modern physics labs would be to simulate the entire planet and much of surrounding space at the level of quantum mechanics, which is computationally intractable even with mature nanotech. Well, that or have an SI constantly tinkering with everyone’s perceptions to make them think everything looks correct, but then you have to suppose that such entities have nothing more interesting to do with their runtime.
I once described a 240 GHz waveguide-structure radio receiver I had built as a “comprehensive analogue simulation of Maxwell’s equations incorporating realistic assumptions about the conductivity of real materials used in waveguide manufacture.” Although this simulation was insanely accurate, it was much more difficult to a) change parameters in this simulation and b) measure/calculate the results of this simulation than with the more traditional digital simulations of Maxwell’s equations we had available.
Another possibility would be to build simulated perceivers whose perceptions are systematically distorted in such a way that they will fail to notice the gaps in the simulated environment, I suppose. Which would not require constant deliberate intervention by an intelligence.
Before you build that, just to practice your skills you can build some code that will take a blurry picture and with extremely high accuracy show what the picture would have looked like had the camera been in focus. This problem would of course be much easier than knowing that you had built a simulation with holes in it but managed to correct for the absence of information in the simulation in a way that was actually simpler than fixing the simulation in the first place.
I think you might run in to limits based on considerations of information theory that make both tasks possible, but if you start with the image reconstruction problem you will save a lot of effort.
Before you build that, just to practice your skills you can build some code that will take a blurry picture and with extremely high accuracy show what the picture would have looked like had the camera been in focus.
The problem with that program is that the information was already there. The information may have been scattered in a semi-random pattern, but it was still there to be reorganized. In this hypothetical simulation, there is a lack of information. And while you can undo randomization to recreate a blurred image, you cannot create information from nothing.
However, the human brain does have some interesting traits which might make it possible for humans to think they are seeing something without creating all the information such a thing would possess. The neocortex has multiple levels. Lower levels detect things like absence and presence of light, which higher levels turn into lines and curves, which even higher levels turn into shapes, which eventually get interpreted as a specific face (the brain has clusters of a few hundred neurons responsible for every face we have memorized). All you would have to do to make a human brain think they saw someone would be to stimulate the top few hundred neurons, the bottom ones need not be given information. Imagine a general telling his troops to move somewhere. Each troop carries out an action, tells their superior, who gives their superior a generalization, who gives their superior a generalization, until the general gets one message “Move going fine”. To fool the general (human) into thinking the move is going fine (interacting with something), you don’t need to forge the entire chain of events (simulate every quark), you just need to give them the message saying everything is going great (stimulate those few hundred neurons). And then when the matrix person looks closer, the Matrix Lords just forge the lower levels temporarily.
The problem with this is is it does not match the principle “Humans simulating old earth to get information”. It would not be giving the future humans any new information they hadn’t created, because they would have to fake that information. They wouldn’t learn anything. It is possible to fool humans in that way, but the only possible use would be for the purpose of fooling someone. And that would require some serious sadism. So there is a scenario in which humans have the computational power and algorithms to make you live in a simulation you think is real, but have no reason to do so.
The original hypothetical was to create a simulated agent that merely fails to notice a gap. New information does not need to be added for this; information from around the gap merely needs to be averaged out to create what appears to be not-a-gap (much as human sight doesn’t have a visible hole in the blind spot).
Now, if the intent was to cover the gap with something specific, then your argument would apply. If, however, the intent is to simply cover up the gap with the most easily calculated non-gap data, then it becomes possible to do so. (Note that it may still remain possible, in such circumstances, to discover the gap indirectly).
Well, given that the alternative ebrownv was considering was ongoing tinkering during runtime by a superintelligence, it’s not quite clear what my ability to build such code has to do with anything.
There’s also a big difference, even for a superintelligence, between building a systematically deluded observer, building a systematically deluded high-precision observer, and building a guaranteed systematically deluded high-precision observer. I’m not sure more than the former is needed for the scenario ebrownv had in mind.
Sure, it might notice something weird one in a million times, but one can probably count on social forces to prevent such anomalous perceptions from being taken too seriously, especially if one patches the simulation promptly on the rare occasions when it doesn’t.
Also, 2 seems to imply there’s a reason to do historical simulations without the knowledge of the simulees, and to thus effectively have billions of sentient beings lead suboptimal miserable lives without their consent.
If the project is government-funded, and the government is any conceivable direct descendent of our current forms of government, then this seems like the default result.
When was the last time you didn’t hear a religious conservative claim “But they’re not really alive, they’re just machines, they don’t have a soul!” or some similar argument (or the shorthand “Who cares? They’re just robots, not living things.”) whenever the subject of reverse machine ethics (i.e. how we treat computers, robots and AIs and whether they should be happy or something) came up?
I also don’t see the point of making historical simulations of that kind in the first place. It strikes me as unnecessarily complex and costly.
Finally, if we’ve reached that level of development so that virtual constructs mistake themselves for conscious, one would think humans would have developed their use as councelors, pets, company, and so on, and would have come to value them as “more than machines”, not because it’s true, but because they want it to be. We humans have a lot of trouble fulfilling each other’s emotional needs, and, if machines were able to do it better, we’d probably refuse to acknowledge their love, affection, esteem, trust, and so on, as lies.
On the other hand, we might also want to believe that they aren’t “real people” so that we can misuse, abuse and destroy them whenever that’s convenient, like we used to do with slaves and prostitutes. It certainly raises interesting questions, and I can’t presume to know enough about human psychology (or future AI development) to confidently make a prediction one way or another.
virtual constructs mistake themselves for conscious
My brain just folded in on itself and vanished. Or at least in simulation it did. I think you may have stated a basilisk, or at least one that works on my self-simulation.
I used to think I was conscious, but then I realized I was mistaken.
Whoever it was that said “I err, therefore I am” didn’t know what he was talking about… because he was wrong in thinking he was even conscious!
I used to wonder what consciousness could be, until you all shared its qualia with me.
You know, we could simply ask; “What would convince us that the simulated humans are not conscious?” “What would convince us that we ourselves are not conscious?” Because, otherwise, “unconscious homunculi” are basically the same as P-Zombies, and we’re making a useless distinction.
Nevertheless, it is possible for a machine to be mistaken about being conscious. Make it un-conscious (in some meaningful way), but make it unable to distinguish conscious from unconscious, and bias its judgment towards qualifying itself as conscious. Basically, the “mistake” would be in its definition of consciousness.
I used to think I was conscious, but then I realized I was mistaken.
Dennett actually believes somehting like that about phenomenal consciousness.
[Dennett:] These additions are perfectly real, but they are … not
made of figment, but made of judgment. There is nothing more to
phenomenology than that
[Otto:] But there seems to be!
[Dennett:] Exactly! There seems to be phenomenology. That’s a
fact that the heterophenomenologist enthusiastically concedes. But
it does not follow from this undeniable, universally attested fact thatth ere
really is phenomenology. This is the crux. (Dennett, 1991, p.
366)
(nods) There’s an amusing bit in a John Varley novel along these lines, where our hero asks a cutting-edge supercomputer AI whether it’s genuinely conscious, and it replies something like “I’ve been exploring that question for a long time, and I’m still not certain. My current working theory is that no, I’m not—nor are you, incidentally—but I am not yet confident of that.” Our hero thinks about that answer for a while and then takes a nap, IIRC.
I also don’t see the point of making historical simulations of that kind in the first place. It strikes me as unnecessarily complex and costly.
Calibration test: How many policies that have been enforced by governments worldwide would you have made the same claim for? How many are currently still being enforced? How many are currently in planning/discussion/voting/etc. but not yet implemented?
In my own, not seeing the point and this being unnecessarily complex and costly, combined with the fact that it’s something people discuss as opposed to any random other possible hypothesis, makes it just as valid a candidate for government signaling and status-gaming as many other types of policies.
However, to clarify my own position: I agree that the second premise implies some sort of motive for running such a simulation without caring for the lives of the minds inside it, but I just don’t think that the part about having billions of “merely simulated” miserable lives would be of much concern to most people with a motive to do it in the first place.
As evidence, I’ll point to the many naive answers to counterfactual muggings of the form “If you don’t give me 100$ I’ll run a trillion simulated copies of you and torture them immensely for billions of subjective years.” “Yeah, so what? They’re just simulations, not real people.”
It certainly raises interesting questions, and I can’t presume to know enough about human psychology (or future AI development) to confidently make a prediction one way or another.
I’d be careful here about constraining your thoughts to “Either magical tiny green buck-toothed AK47-wielding goblins yelling ‘Wazooomba’ exist, or they don’t, right? So it’s about 50-50 that they do.” I’m not quite sure if there even is any schelling point in-between.
I can’t say that any policy comes to mind at this point. In my country the norm has always been to claim for more government intervention and higher direct taxes, not less. If anything, I find government spending cuts are the ones that tend to be irrationally implemented and make a mess of perfectly serviceable services, and I always see those as a threat. The private sector is the one in charge of frivolities, luxuries and unnecessary stuff that people still want to spend money on, as long as it isn’t public dimes. [I’ve been downvoted over expressing this kind of opinion in the past; if you have different opinions that consider these to be blatantly stupid, I’ll still humbly request that you refrain from doing that. Please.].
It’s only something people discuss because, as far as I can tell from the superficial data I’ve collected so far, some intellectual hipster decided it would be amusing to pose a disturbing and unfalsifiable hypothesis, and see how people reacted to it and argued against it. As far as I’m concerned, until there’s any evidence at all that this reality is not the original, it simply doesn’t make sense to promote the hypothesis to our attention.
That counterfactual mugging would at least give me pause. Though I think my response would be more long the lines of attempting pre-emptive use of potentially lethal force than simply paying the bribe. That sort of dangerous, genocidal madman shouldn’t be allowed access to a computer.
I didn’t imply that there are only two ways this can go, but the hypothesis space and the state of current evidence is such that I can’t see any specific hypothesis being promoted to my attention, so I’m sticking with the default “this is the original reality” until evidence shows up to make me consider otherwise.
I’d be careful here about constraining your thoughts to “Either magical tiny green buck-toothed AK47-wielding goblins yelling ‘Wazooomba’ exist, or they don’t, right? So it’s about 50-50 that they do.” I’m not quite sure if there even is any schelling point in-between.
Complexity-based priors solve that problem. Magical tiny green buck-toothed AK47-wielding goblins yelling ‘Wazooomba’ are complex, so, in the absence of evidence that they do exist, you’re justified in concluding that they don’t. ;)
Given several other assumptions, some empirical, some about anthropic reasoning.
Right. Most poeple disagree with the premise “Being in a simulation is/can be made to be indistinguishable from reality from the point of view of the simulee.”
Edit: And by most people, I mean my analysis of why I intuitively reject the conclusion, not any discussion with other, let alone 3.6+ billion people or statistically representative sampling, etc.
I am surprised to hear this. What is your basis for claiming that this is the premise most people object to?
Also, if you are aware of or familiar with this objection—would you mind explaining the following questions I have regarding it?
What reason is there to suspect that a simulated me would have a different/distinguishable experience from real me?
What reason is there to suspect that if there were differences between simulated and real life, that a simulated life would be aware of those differences? That is, even if it is distinguishable—I have only experienced one kind of life and can’t say if my totally distinguishable experience of life is that of a simulated life or a real one.
A magic super computer from the future will be able to simulate one atom with arbitrary accuracy—right? A super-enough computer will be able to simulate many atoms interacting with arbitrary accuracy. If this super computer is precisely simulating all the atoms of an empty room containing a single human being (brain included). If this simulation is happening—how could the simulated being possibly have a different experience than its real counterpart in an empty room? Atomically speaking everything is identical.
Maybe questions 1 and 3 are similar—but I’d appreciate if you (or someone else) could enlighten me regarding these issues.
As someone who has written lots of simulations, there are a few reasons.
1) The simulation deliberately simplifies or changes some things from reality. At minimum, when “noise” is required an algorithm is used to generate numbers which have many of the properties of random numbers but a) are not in fact random, b) are usually much more accurately described by a particular mathematical distribution than would any measurements of the actual noise in the system be.
2) The simulation accidentally simplifies/changes LOTS of things from reality. A brain simulation at the neuron level is likely to simulate observed variations using a noise generator, when these variations arise from a) a ream of detailed motions of individual ions and b) quantum interactions. The claim is generally made that one can simulate at a more and more detailed level AND GET TO THE ENDPOINT where the simulation is “perfect.” The getting to the endpoint claim is not only unproven, but highly suspect. At every level of physics we have investigated so far, we have always found a deeper level. Further, the equations of motions at these deepest layers are not known in complete detail. So even if we can get to an endpoint, we have no reason to believe we have gotten to the endpoint in any given simulation. At some point, we are no longer compute bound, we are knowledge bound.
3) There is a great insight in software that “if it isn’t tested, its broken.” How do you even test a supremely deep simulation of yourself? If there are features of yourself you are still learning, you can’t test for them. Until you comprehensievly comprehend yourself, you can never know that a simulation was comprehensively similar.
Even something as simple as a coin toss simulation is likely to be “wrong” in detail. Perhaps you know the coin toss you are actually simulating has .500 or even .500000000 probability of giving heads (where number of zeros represents accuracy to which you know it.) But what is your confidence that the true expectation is 0.5 with a googleplex zeros following (or 3^^3 zeros to pretend to try to fit in here) is the experimental fact? Even 64 zeros would be a bitch to prove. And what are the chances that your simulation gets a ’true expectation” of 0.5 with even 64 zeros after it? With the coin toss, the variance might SEEM trivial, but consider the same uncertainty in the human. You need to predict my next post keystroke for keystroke, which necessarily includes a prediction of whether I will eat an egg for breakfast or a bowel of cereal because the posts I read while eating depend on that. And so on and so on.
My claim is that the existence of an endpoint in finally getting the simulation complete is at best vastly beyond our knowledge (and not in a compute bound way) and at worst simply unknowable for a ream of good reasons. My estimate of the probability that a simulation will ever be reliably known is < 0.01%.
Now we may get to a much easier place: good enough to convince others. That someone can write a simulation of me that cannot be distinguished from me by people who know me is a much lower bar than that the simulatino feels the same as me to itself. To convince others, the simulation may not even have to be conscious, for example. But you are going to have to build your simulation in to a fat human body good enough to fool my wife, and give it a variety of nervous and personality disorders that cause it to come up with digs that are deeply disturbing to her to do even that.
At some point, the comprehensive difficulty of a problem has to open the question: is it reasonable to sweep this under the rug by appealing to an unknown future of much greater capability than we have now, or is doing that a human bias we may need to avoid?
I think enough people are non-reductionist/materialist to have doubt about whether a simulation can be said to have experiences. And we don’t exactly have demonstration of this at this time, do we? I mean, in the example cited, Cvilization PC games, there aren’t individuals there to have experiences (unless one counts the ai which is running the entire faction), rather there are some blips in databases incrementing the number of units here or there, or raising the population from an abstract 6 to 7. I don’t think people will be able to take simulation theory seriously until they have personal interaction with a convincing ai.
That’s probably as much an answer as I can give for any of the questsions, other than that I don’t see why we can assume that magic super computers are plausible. Related, I don’t know if I trust my intuition or reasoning as to whether an infinite simulation will resemble realty in every way (assuming the supercomputer is running equations and databases, etc, rather than actually reconstucting a universe atom by atom or something).
It feels like you’re asking me to believe that a map is the same as the territory if it is a good enough map. I know that’s just an analogy, but I have a hard time comprehending the sentence that “reality is the solution to/ equations and nothing more” (as opposed to even “reality is predictable by equations”).
This is probably not the LW approved answer, but then, I did say most people and not most LW-ers.
I don’t understand subjective experience very well, so I don’t know if a simulation would have it. I know that an adult human brain does, and I’m pretty sure a rock doesn’t, but there are other cases I’m much less certain about. Mice, for example.
Yes, and this isn’t necessarily due to naive ‘no mere simulation could feel so real’-type thinking either. Once can make a decent argument that the only known method of making a simulation that can fool modern physics labs would be to simulate the entire planet and much of surrounding space at the level of quantum mechanics, which is computationally intractable even with mature nanotech. Well, that or have an SI constantly tinkering with everyone’s perceptions to make them think everything looks correct, but then you have to suppose that such entities have nothing more interesting to do with their runtime.
I once described a 240 GHz waveguide-structure radio receiver I had built as a “comprehensive analogue simulation of Maxwell’s equations incorporating realistic assumptions about the conductivity of real materials used in waveguide manufacture.” Although this simulation was insanely accurate, it was much more difficult to a) change parameters in this simulation and b) measure/calculate the results of this simulation than with the more traditional digital simulations of Maxwell’s equations we had available.
Another possibility would be to build simulated perceivers whose perceptions are systematically distorted in such a way that they will fail to notice the gaps in the simulated environment, I suppose. Which would not require constant deliberate intervention by an intelligence.
Before you build that, just to practice your skills you can build some code that will take a blurry picture and with extremely high accuracy show what the picture would have looked like had the camera been in focus. This problem would of course be much easier than knowing that you had built a simulation with holes in it but managed to correct for the absence of information in the simulation in a way that was actually simpler than fixing the simulation in the first place.
I think you might run in to limits based on considerations of information theory that make both tasks possible, but if you start with the image reconstruction problem you will save a lot of effort.
This has now been done—to a first approximation, at least.
The problem with that program is that the information was already there. The information may have been scattered in a semi-random pattern, but it was still there to be reorganized. In this hypothetical simulation, there is a lack of information. And while you can undo randomization to recreate a blurred image, you cannot create information from nothing.
However, the human brain does have some interesting traits which might make it possible for humans to think they are seeing something without creating all the information such a thing would possess. The neocortex has multiple levels. Lower levels detect things like absence and presence of light, which higher levels turn into lines and curves, which even higher levels turn into shapes, which eventually get interpreted as a specific face (the brain has clusters of a few hundred neurons responsible for every face we have memorized). All you would have to do to make a human brain think they saw someone would be to stimulate the top few hundred neurons, the bottom ones need not be given information. Imagine a general telling his troops to move somewhere. Each troop carries out an action, tells their superior, who gives their superior a generalization, who gives their superior a generalization, until the general gets one message “Move going fine”. To fool the general (human) into thinking the move is going fine (interacting with something), you don’t need to forge the entire chain of events (simulate every quark), you just need to give them the message saying everything is going great (stimulate those few hundred neurons). And then when the matrix person looks closer, the Matrix Lords just forge the lower levels temporarily.
The problem with this is is it does not match the principle “Humans simulating old earth to get information”. It would not be giving the future humans any new information they hadn’t created, because they would have to fake that information. They wouldn’t learn anything. It is possible to fool humans in that way, but the only possible use would be for the purpose of fooling someone. And that would require some serious sadism. So there is a scenario in which humans have the computational power and algorithms to make you live in a simulation you think is real, but have no reason to do so.
The original hypothetical was to create a simulated agent that merely fails to notice a gap. New information does not need to be added for this; information from around the gap merely needs to be averaged out to create what appears to be not-a-gap (much as human sight doesn’t have a visible hole in the blind spot).
Now, if the intent was to cover the gap with something specific, then your argument would apply. If, however, the intent is to simply cover up the gap with the most easily calculated non-gap data, then it becomes possible to do so. (Note that it may still remain possible, in such circumstances, to discover the gap indirectly).
Well, given that the alternative ebrownv was considering was ongoing tinkering during runtime by a superintelligence, it’s not quite clear what my ability to build such code has to do with anything.
There’s also a big difference, even for a superintelligence, between building a systematically deluded observer, building a systematically deluded high-precision observer, and building a guaranteed systematically deluded high-precision observer. I’m not sure more than the former is needed for the scenario ebrownv had in mind.
Sure, it might notice something weird one in a million times, but one can probably count on social forces to prevent such anomalous perceptions from being taken too seriously, especially if one patches the simulation promptly on the rare occasions when it doesn’t.
Also, 2 seems to imply there’s a reason to do historical simulations without the knowledge of the simulees, and to thus effectively have billions of sentient beings lead suboptimal miserable lives without their consent.
If the project is government-funded, and the government is any conceivable direct descendent of our current forms of government, then this seems like the default result.
When was the last time you didn’t hear a religious conservative claim “But they’re not really alive, they’re just machines, they don’t have a soul!” or some similar argument (or the shorthand “Who cares? They’re just robots, not living things.”) whenever the subject of reverse machine ethics (i.e. how we treat computers, robots and AIs and whether they should be happy or something) came up?
There’s already some effort being done in that direction. If sentient extraterrestrial lifeforms could at least be considered legal animals, what of our own human-made creatures?
I also don’t see the point of making historical simulations of that kind in the first place. It strikes me as unnecessarily complex and costly.
Finally, if we’ve reached that level of development so that virtual constructs mistake themselves for conscious, one would think humans would have developed their use as councelors, pets, company, and so on, and would have come to value them as “more than machines”, not because it’s true, but because they want it to be. We humans have a lot of trouble fulfilling each other’s emotional needs, and, if machines were able to do it better, we’d probably refuse to acknowledge their love, affection, esteem, trust, and so on, as lies.
On the other hand, we might also want to believe that they aren’t “real people” so that we can misuse, abuse and destroy them whenever that’s convenient, like we used to do with slaves and prostitutes. It certainly raises interesting questions, and I can’t presume to know enough about human psychology (or future AI development) to confidently make a prediction one way or another.
My brain just folded in on itself and vanished. Or at least in simulation it did. I think you may have stated a basilisk, or at least one that works on my self-simulation.
I used to think I was conscious, but then I realized I was mistaken.
Whoever it was that said “I err, therefore I am” didn’t know what he was talking about… because he was wrong in thinking he was even conscious!
You know, we could simply ask; “What would convince us that the simulated humans are not conscious?” “What would convince us that we ourselves are not conscious?” Because, otherwise, “unconscious homunculi” are basically the same as P-Zombies, and we’re making a useless distinction.
Nevertheless, it is possible for a machine to be mistaken about being conscious. Make it un-conscious (in some meaningful way), but make it unable to distinguish conscious from unconscious, and bias its judgment towards qualifying itself as conscious. Basically, the “mistake” would be in its definition of consciousness.
Dennett actually believes somehting like that about phenomenal consciousness.
[Dennett:] These additions are perfectly real, but they are … not made of figment, but made of judgment. There is nothing more to phenomenology than that [Otto:] But there seems to be! [Dennett:] Exactly! There seems to be phenomenology. That’s a fact that the heterophenomenologist enthusiastically concedes. But it does not follow from this undeniable, universally attested fact thatth ere really is phenomenology. This is the crux. (Dennett, 1991, p. 366)
Er… can I get the cliff’s notes on the jargon here?
(nods) There’s an amusing bit in a John Varley novel along these lines, where our hero asks a cutting-edge supercomputer AI whether it’s genuinely conscious, and it replies something like “I’ve been exploring that question for a long time, and I’m still not certain. My current working theory is that no, I’m not—nor are you, incidentally—but I am not yet confident of that.” Our hero thinks about that answer for a while and then takes a nap, IIRC.
Calibration test: How many policies that have been enforced by governments worldwide would you have made the same claim for? How many are currently still being enforced? How many are currently in planning/discussion/voting/etc. but not yet implemented?
In my own, not seeing the point and this being unnecessarily complex and costly, combined with the fact that it’s something people discuss as opposed to any random other possible hypothesis, makes it just as valid a candidate for government signaling and status-gaming as many other types of policies.
However, to clarify my own position: I agree that the second premise implies some sort of motive for running such a simulation without caring for the lives of the minds inside it, but I just don’t think that the part about having billions of “merely simulated” miserable lives would be of much concern to most people with a motive to do it in the first place.
As evidence, I’ll point to the many naive answers to counterfactual muggings of the form “If you don’t give me 100$ I’ll run a trillion simulated copies of you and torture them immensely for billions of subjective years.” “Yeah, so what? They’re just simulations, not real people.”
I’d be careful here about constraining your thoughts to “Either magical tiny green buck-toothed AK47-wielding goblins yelling ‘Wazooomba’ exist, or they don’t, right? So it’s about 50-50 that they do.” I’m not quite sure if there even is any schelling point in-between.
I can’t say that any policy comes to mind at this point. In my country the norm has always been to claim for more government intervention and higher direct taxes, not less. If anything, I find government spending cuts are the ones that tend to be irrationally implemented and make a mess of perfectly serviceable services, and I always see those as a threat. The private sector is the one in charge of frivolities, luxuries and unnecessary stuff that people still want to spend money on, as long as it isn’t public dimes. [I’ve been downvoted over expressing this kind of opinion in the past; if you have different opinions that consider these to be blatantly stupid, I’ll still humbly request that you refrain from doing that. Please.].
It’s only something people discuss because, as far as I can tell from the superficial data I’ve collected so far, some intellectual hipster decided it would be amusing to pose a disturbing and unfalsifiable hypothesis, and see how people reacted to it and argued against it. As far as I’m concerned, until there’s any evidence at all that this reality is not the original, it simply doesn’t make sense to promote the hypothesis to our attention.
That counterfactual mugging would at least give me pause. Though I think my response would be more long the lines of attempting pre-emptive use of potentially lethal force than simply paying the bribe. That sort of dangerous, genocidal madman shouldn’t be allowed access to a computer.
I didn’t imply that there are only two ways this can go, but the hypothesis space and the state of current evidence is such that I can’t see any specific hypothesis being promoted to my attention, so I’m sticking with the default “this is the original reality” until evidence shows up to make me consider otherwise.
Complexity-based priors solve that problem. Magical tiny green buck-toothed AK47-wielding goblins yelling ‘Wazooomba’ are complex, so, in the absence of evidence that they do exist, you’re justified in concluding that they don’t. ;)