Scott puts a high probability on us being in a simulation, and on that being related to the current AI situation in ways that are interesting for us. According to him, the only argument against simulation is that it’s too weird. I think the case for being in a simulation—in a way that matters—is much weaker than that, because ancestor simulations don’t make a lot of sense and the Bostrom thought experiment oversimplifies. I also am not so sure that we are at the “hinge of history”, or that many of his readers being AI researchers is meaningful. And importantly, the case for being in a simulation and that mattering—that we can for example plead with the simulators or trade with them—seems incredibly weak.
1. Bostrom’s forgotten condition: that these civilizations run mostly ancestor simulations
Bostrom’s trilemma goes like this: (i) Almost all civilizations like ours die out before reaching technological maturity. (ii) Almost none of the mature ones bother running ancestor simulations. (iii) We are almost certainly in one. I think this trilemma misses something important. I think there is a hidden condition: civilizations have to run mostly ancestor simulations instead of other types of simulations of sentient, intelligent beings. Otherwise we would be more likely to be in a non-ancestor simulation.
Our world makes every appearance of not being simulated, and we should treat that as evidence that it isn’t. Consider the space of simulations a future civilization (likely an ASI) might run. Many of them carry tells: there is no history, like in dath ilan; or the simulator is literally visible, as perhaps a God. A non-simulated world on the other hand has a coherent past running back to the Big Bang—just like our world. Ancestor simulations fall in the narrow band of simulations that don’t appear simulated and have a coherent past.
The fact that this world doesn’t appear simulated should update us to think this isn’t a simulated world. For Bostrom’s view to make sense, it shouldn’t only make sense to run ancestor simulations—it should make much more sense to run ancestor simulations than any other type of simulation of intelligent, sentient beings.
2. It isn’t clear that running ancestor simulations makes sense at all
Nick Bostrom argues that future civilizations might potentially simulate their ancestors for various purposes. The strongest version I have heard of this was from Nate Soares, who argued that ASIs might try to predict each other’s preferences like this, but he conceded he still found it unlikely. My suspicion is that a superintelligence in the future simply has no strong incentive to simulate us, and I find most explanations highly lacking. I think the case is similar for human civilizations that have somehow tamed the threat of AI, and they may additionally have moral reservations.
This part of the argument reminds me of similar ideas in the vicinity of “why wouldn’t the ASI spend a little money to keep humans alive?” But there are so many things an ASI could do with its resources, and for this to be one of them seems unlikely. Why would it simulate you taking a dump when it could instead be instantiating infinite bliss states, or planning war (or cooperation) with other ASIs?
I also don’t go around expending significant effort on arbitrary things I don’t care about. I would never, for instance, collect thousands of stones and arrange them into prime-numbered piles even if I had enormous resources.
I think that it is totally possible that I am wrong here, but the case that it obviously makes sense to simulate your ancestors just doesn’t hold up for me. It is not enough for that to be somewhat useful; there also can’t be a better use of resources to achieve the same thing by the ASI. And again: I believe it is not enough for there to be a reason to run ancestor simulations, there would also have to be a reason not to run other types of simulations of sentient beings at similar scale.
3. You are not a random sample from near-the-singularity; you are a selected one
Then there’s the “you are in a special place right now” argument. Scott Alexander calls on others to move to the Bay Area. The reason you read Scott Alexander’s blog is closely related to the reason you’re near the singularity. You’re the kind of person who reads Scott Alexander, and that’s the kind of person who founds AI startups or becomes an AI safety researcher. That’s the kind of person who asks big questions about existence.
In other words: you have an interest in anthropic thought experiments and you’re near the singularity, because of an underlying variable—intelligence, curiosity. You find yourself reading Scott Alexander’s thought experiments and working in AI because of the same underlying traits.
4. I’m not sure we live at the hinge of history
I do agree that we live in a very special time, but it is less clear that we live in the “hinge of history” in the sense that our actions are particularly impactful. My impression is that if, say, Hitler or WWII had never existed, that might have had a larger effect on AI alignment than anything we do now. It does feel to me like everyone’s hands are currently tied and we’re in a mindless race to superintelligence. If you are an AI engineer who has to approve Claude Code’s suggested API optimization code to go from 99.2 to 99.3 percent uptime, how much influence do you really have?
If you’re already in the waterfall and about to hit the water, you have less causal control than you had fifteen minutes earlier, when you could still have steered away from it.
Where it actually dies for me
Up until this point I still assign significant probability to the simulation argument. However, with some inevitability, the people peddling it will at some point turn from “we might be in a simulation”—which I agree is possible—to some version of let’s try to talk to the simulators. Let’s reason or trade with them. That’s the point where I think the argument fails: where we’re supposed to reason about our superintelligent descendants in order to figure out what they’re thinking.
If they really were interested in our well-being and willing to be kind to us, I don’t think we would be here.
And it matters enormously that this step requires a much stronger version of the simulation argument. It’s not enough that we’re in a simulation. It has to be one where the simulators are in some sense listening to our remarks and are likely to be swayed by us pleading with them. So there is a huge motte and bailey being applied here.
[If I had to write it again I would probably include arguments from this post]
I don’t really understand why Scott says “I am scott alexander, far more connected to the singularity than almost anyone in history, therefore that’s evidence this is a simulation” (1) Is he implying he thinks most people are P-zombies? If so, he’s implying p-zombies are possible which would be a surprising belief (2) If p-zombies are possible then why make anyone conscious at all? What’s the point of only simulating people close to the singularity
(3) Then for me personally, I have ~0 leverage over anything. I certainly follow the news closely and am connected in that sense, but I’m a random white collar worker.
(4) Would this be reason to be more confident the singularity is happening within our lifetime?
I also used to find ancestor simulators something unlikely for future AIs to do just because they would be so beyond us with their transcendental concerns etc. but then I realized we humans would be quite interested in simulating how abiogenesis originally happened (despite being unfathomably more complex and advanced compared to the earliest forms of life) so seeing simulations of us as analogous to something like that started to look not so crazy.
[Repost from X/twitter] in response to this tweet. [Wondering if it will create more discussion here]
Arguments Against Simulation:
Scott puts a high probability on us being in a simulation, and on that being related to the current AI situation in ways that are interesting for us. According to him, the only argument against simulation is that it’s too weird. I think the case for being in a simulation—in a way that matters—is much weaker than that, because ancestor simulations don’t make a lot of sense and the Bostrom thought experiment oversimplifies. I also am not so sure that we are at the “hinge of history”, or that many of his readers being AI researchers is meaningful. And importantly, the case for being in a simulation and that mattering—that we can for example plead with the simulators or trade with them—seems incredibly weak.
1. Bostrom’s forgotten condition: that these civilizations run mostly ancestor simulations
Bostrom’s trilemma goes like this: (i) Almost all civilizations like ours die out before reaching technological maturity. (ii) Almost none of the mature ones bother running ancestor simulations. (iii) We are almost certainly in one. I think this trilemma misses something important. I think there is a hidden condition: civilizations have to run mostly ancestor simulations instead of other types of simulations of sentient, intelligent beings. Otherwise we would be more likely to be in a non-ancestor simulation.
Our world makes every appearance of not being simulated, and we should treat that as evidence that it isn’t. Consider the space of simulations a future civilization (likely an ASI) might run. Many of them carry tells: there is no history, like in dath ilan; or the simulator is literally visible, as perhaps a God. A non-simulated world on the other hand has a coherent past running back to the Big Bang—just like our world. Ancestor simulations fall in the narrow band of simulations that don’t appear simulated and have a coherent past.
The fact that this world doesn’t appear simulated should update us to think this isn’t a simulated world. For Bostrom’s view to make sense, it shouldn’t only make sense to run ancestor simulations—it should make much more sense to run ancestor simulations than any other type of simulation of intelligent, sentient beings.
[I had a lengthy exchange on this subpoint]
2. It isn’t clear that running ancestor simulations makes sense at all
Nick Bostrom argues that future civilizations might potentially simulate their ancestors for various purposes. The strongest version I have heard of this was from Nate Soares, who argued that ASIs might try to predict each other’s preferences like this, but he conceded he still found it unlikely. My suspicion is that a superintelligence in the future simply has no strong incentive to simulate us, and I find most explanations highly lacking. I think the case is similar for human civilizations that have somehow tamed the threat of AI, and they may additionally have moral reservations.
This part of the argument reminds me of similar ideas in the vicinity of “why wouldn’t the ASI spend a little money to keep humans alive?” But there are so many things an ASI could do with its resources, and for this to be one of them seems unlikely. Why would it simulate you taking a dump when it could instead be instantiating infinite bliss states, or planning war (or cooperation) with other ASIs?
I also don’t go around expending significant effort on arbitrary things I don’t care about. I would never, for instance, collect thousands of stones and arrange them into prime-numbered piles even if I had enormous resources.
I think that it is totally possible that I am wrong here, but the case that it obviously makes sense to simulate your ancestors just doesn’t hold up for me. It is not enough for that to be somewhat useful; there also can’t be a better use of resources to achieve the same thing by the ASI. And again: I believe it is not enough for there to be a reason to run ancestor simulations, there would also have to be a reason not to run other types of simulations of sentient beings at similar scale.
3. You are not a random sample from near-the-singularity; you are a selected one
Then there’s the “you are in a special place right now” argument. Scott Alexander calls on others to move to the Bay Area. The reason you read Scott Alexander’s blog is closely related to the reason you’re near the singularity. You’re the kind of person who reads Scott Alexander, and that’s the kind of person who founds AI startups or becomes an AI safety researcher. That’s the kind of person who asks big questions about existence.
In other words: you have an interest in anthropic thought experiments and you’re near the singularity, because of an underlying variable—intelligence, curiosity. You find yourself reading Scott Alexander’s thought experiments and working in AI because of the same underlying traits.
4. I’m not sure we live at the hinge of history
I do agree that we live in a very special time, but it is less clear that we live in the “hinge of history” in the sense that our actions are particularly impactful. My impression is that if, say, Hitler or WWII had never existed, that might have had a larger effect on AI alignment than anything we do now. It does feel to me like everyone’s hands are currently tied and we’re in a mindless race to superintelligence. If you are an AI engineer who has to approve Claude Code’s suggested API optimization code to go from 99.2 to 99.3 percent uptime, how much influence do you really have?
If you’re already in the waterfall and about to hit the water, you have less causal control than you had fifteen minutes earlier, when you could still have steered away from it.
Where it actually dies for me
Up until this point I still assign significant probability to the simulation argument. However, with some inevitability, the people peddling it will at some point turn from “we might be in a simulation”—which I agree is possible—to some version of let’s try to talk to the simulators. Let’s reason or trade with them. That’s the point where I think the argument fails: where we’re supposed to reason about our superintelligent descendants in order to figure out what they’re thinking.
If they really were interested in our well-being and willing to be kind to us, I don’t think we would be here.
And it matters enormously that this step requires a much stronger version of the simulation argument. It’s not enough that we’re in a simulation. It has to be one where the simulators are in some sense listening to our remarks and are likely to be swayed by us pleading with them. So there is a huge motte and bailey being applied here.
[If I had to write it again I would probably include arguments from this post]
I don’t really understand why Scott says “I am scott alexander, far more connected to the singularity than almost anyone in history, therefore that’s evidence this is a simulation”
(1) Is he implying he thinks most people are P-zombies? If so, he’s implying p-zombies are possible which would be a surprising belief
(2) If p-zombies are possible then why make anyone conscious at all? What’s the point of only simulating people close to the singularity
(3) Then for me personally, I have ~0 leverage over anything. I certainly follow the news closely and am connected in that sense, but I’m a random white collar worker.
(4) Would this be reason to be more confident the singularity is happening within our lifetime?
I also used to find ancestor simulators something unlikely for future AIs to do just because they would be so beyond us with their transcendental concerns etc. but then I realized we humans would be quite interested in simulating how abiogenesis originally happened (despite being unfathomably more complex and advanced compared to the earliest forms of life) so seeing simulations of us as analogous to something like that started to look not so crazy.