An implicit assumption (which should have been made explicit) of the post is that the cost per simulation is tiny. This is like in WW II where the US would send a long-range bomber to take photos of Japan. I agree with your last paragraph and I think it gets to what is consciousness. Is the program’s existence enough to generate consciousness, or does the program have to run to create conscious observers?
James_Miller
Karma: 15,261
Dear Paperclip Maximizer, Please Don’t Turn Off the Simulation
Great example!
I’m assuming the cost of this simulation is tiny compared to the value of learning about potential enemies and trading partners.
Yes, I agree that you can’t give too much weight to my saying I’m in pain because I could be non-conscious from your viewpoint. Assuming all humans are conscious and pain is as it appears to be, there seems to be a lot of unnecessary pain, but yes I could be missing the value of having us experience it.
I’m in low level chronic pain including as I write this comment, so while I think the entire Andromeda galaxy might be fake, I think at least some suffering must be real, or at least I have the same confidence in my suffering as I do in my consciousness.
I’m in low level chronic pain including as I write this comment, so while I think the entire Andromeda galaxy might be fake, I think at least some suffering must be real, or at least I have the same confidence in my suffering as I do in my consciousness.
They people running the Karma test deserve to lose a lot of Karma for the suffering in this world.
If your hypothesis is true, that’s a cruel civilization by my personal standards because of all the suffering in this world.
Yes, that is the same idea. “This is a big pile of speculation that I don’t take very seriously, but I feel like if we are being simulated, that’s where most simulations of me would be instantiate” Why not take it seriously, if you accept high chance that (1) our reality is a simulation, (2) we seem on track to creating a paperclip maximizer, (3) weird that I, Robert Miles, would have the personality traits that cause me to be one of the few humans so worried about humanity creating a paperclip maximizer if I’m right about us being on track to probably create one?
In your dreams do you ever see trees you think are real? I doubt your brain is simulating the trees at a very high level of detail, yet this dream simulation can fool you.
By your theory, if you believe that we are near to the singularity how should we update on the likelihood that we exist at such an incredibly important time?
We don’t know that our reality is being simulated at the molecular level, we could just be fooled into thinking it is.
Our Reality: A Simulation Run by a Paperclip Maximizer
Quantum Immortality: A Perspective if AI Doomers are Probably Right
A historical analogy might be the assassination of Bardiya, who was the king of Persia and the son of Cyrus the Great. Darius, who led the assassination, claimed that the man he killed was an impostor who used magic powers to resemble the son of Cyrus. As Darius became the next king of Persia, everyone was brute forced into accepting his narrative of the assassination.
I meant the noise pollution example in my essay to be the Coase theorem, but I agree with you that property rights are not strong enough to solve with AI risk. I agree that AI will open up new paths for solving all kinds of problems, including giving us solutions that could end up helping with alignment.
The big thing I used it for was asking it to find sentences it thinks it can improve, and then have it give me the improved sentence. I created this GPT to help with my writing: https://chat.openai.com/g/g-gahVWDJL5-iterative-text-improver
I agree with the analogy in your last paragraph, and this gives hope for governments slowing down AI development, if they have the will.
Yes. It is running trillions upon trillions simulations and ignoring almost all of the details from the simulations. Our hope is that writing this letter slightly increases the odds that it learns about the contents of this post. Also, there are multiple acausal trade equilibria and this version of me taking about them could favorably alter which equilibria we are in. Finally, agency has value and so writing this letter by itself might slightly increase the expected value of working with us.