I’ve been poking at the philosophy of math recently. It really seems like there’s no way to conceive of a universe that is beyond the reach of logic except one that also can’t support life. Classic posts include unreasonable effectiveness of mathematics, what numbers could not be, a few others. So then we need epistemology.
We can make all sorts of wacky nested simulations and any interesting ones, ones that can support organisms (that is, ones that are Turing complete), can also support processes for predicting outcomes in that universe, and those processes appear to necessarily need to do reasoning about what is “simple” in some sense in order to work. So that seems to hint that algorithmic information theory isn’t crazy (unless I just hand waved over a dependency loop, which I totally might have done, it’s midnight), which means that we can use the equivalence of Turing complete structures to assume we can infer things about the universe. Maybe not solononoff induction, but some form of empirical induction. And then we’ve justified ordinary reasoning about what’s simple.
Okay, so we can reason normally about simplicity. What universes produce observers like us and arise from mathematically simple rules? Lots of them, but it seems to me the main ones produce us via base physics, and then because there was an instance in base physics, we also get produced in neighboring civilizations’ simulations of what other things base physics might have done in nearby galaxies so as to predict what kind of superintelligent aliens they might be negotiating with before they meet each other. Or, they produce us by base physics, and then we get instantiated again later to figure out what we did. Ancestor sims require very good outcomes which seem rare, so those branches are lower measure anyway, but also ancestor sims don’t get to produce super ai separate from the original causal influence.
Point is, no, what’s going on in the simulations is nearly entirely irrelevant. We’re in base physics somewhere. Get your head out of the simulation clouds and choose what you do in base physics, not based on how it affects your simulators’ opinion of the simulation’s moral valence. Leave that sort of crazy stuff to friendly ai, you can’t understand superintelligent simulators which we can’t even get evidence exist besides plausible but very galaxy brain abstract arguments.
(Oh, might be relevant that I’m a halfer when making predictions, thirder when choosing actions—see anthropic decision theory for an intuition on that.)
Thank you, I feel inclined to accept that for now.
But I’m still not sure, and I’ll have to think more about this response at some point.
Edit: I’m still on board with what you’re generally saying, but I feel skeptical of one claim:
It seems to me the main ones produce us via base physics, and then because there was an instance in base physics, we also get produced in neighboring civilizations’ simulations of what other things base physics might have done in nearby galaxies so as to predict what kind of superintelligent aliens they might be negotiating with before they meet each other.
My intuition tells me there will probably be superior methods of gathering information about superintelligent aliens. To me, it seems like the most obvious reason to create sims would be to respect the past for some bizarre ethical reason, or for some weird kind of entertainment, or even to allow future aliens to temporarily live in a more primitive body. Or perhaps for a reason we have yet to understand.
I don’t think any of these scenarios would really change the crux of your argument, but still, can you please justify your claim for my curiosity?
Sims are very cheap compared to space travel, and you need to know what you’re dealing with in quite a lot of detail before you fly because you want to have mapped the entire space of possible negotiations in an absolutely ridiculous level of detail.
Sims built for this purpose would still be a lot lower detail than reality, but of course that would be indistinguishable from inside if the sim is designed properly. Maybe most kinds of things despawn in the sim when you look away, for example. Only objects which produce an ongoing computation that has influence on the resulting civ would need modeling in detail. Which I suspect would include every human on earth, due to small world effects, the internet, sensitive dependence on initial conditions, etc. Imagine how time travel movies imply the tiniest change can amplify—one needs enough detail to have a good map of that level of thing. Compare weather simulation.
Someone poor in Ghana might die and change the mood of someone working for ai training in Ghana, which subtly affects how the unfriendly AI that goes to space and affects alien civs is produced, or something. Or perhaps there’s an uprising when they try to replace all human workers with robots. Modeling what you thought about now helps predict how good you’ll be at the danceoff in your local town which affects the posts produced as training data on the public internet. Oh, come to think of it, where are we posting, and on what topic? Perhaps they needed to model your life in enough detail to have tight estimates of your posts, because those posts affect what goes on online.
But most of the argument for continuing to model humans seems to me to be the sensitive dependence on initial conditions, because it means you need an unintuitively high level of modeling detail in order to estimate what von Neumann probe wave is produced.
Still cheap—even in base reality earth right now is only taking up a little more energy than its tiny silhouette against the sun’s energy output in all directions. A kardashev 2 civ would have no problem fuelling an optimized sim with a trillion trillion samples of possible aliens’ origin processes. Probably superintelligent kardashev 1 even finds it quite cheap, could be less then earth’s resources to do the entire sim including all parallel outcomes.
I’m pretty worried that we can’t understand the universe “properly” even if we’re in base physics! It’s not yet clearly forbidden that the foundations of philosophy contain unanswerable questions, things where there’s a true answer that affects our universe in ways that are not exposed in any way physically, and can only be referred to by theoretical reasoning; which then relies on how well our philosophy and logic foundations actually have the real universe as a possible referent. Even if they do, things could be annoying. In particular, one possible annoying hypothesis would be if the universe is in Turing machines, but is quantum—then in my opinion that’s very weird but hey at least we have a set in which the universe is realizable. Real analysis and some related stuff gives us some idea things can be reasoned about from within a computation based understanding of structure, but which are philosphically-possibly-extant structures beyond computation, and whether true reality can contain “actual infinities” is a classic debate.
So sims are small potatoes, IMO. Annoying simulators that want to actively mess up our understandings are clearly possible but seem not particularly likely by models I believe right now; seems to me they’d rather just make minds within their own universe; sims are for pretending to be another timeline or universe to a mind you want to instantiate, whatever your reason for that pretense. If we can grab onto possible worlds well enough, and they aren’t messing up our understanding on purpose, then we can reason about plausible base realities and find out we’re primarily in a sim by making universe sims ourselves and discovering the easiest way to find ourselves is if we first simulate some alien civ or other.
But if we can’t even in principle have a hypothesis space which relates meaningfully to what structures a universe could express, then phew, that’s pretty much game over for trying to guess at tegmark 4 and who might simulate us in it or what other base physics was possible or exists physically in some sense.
My giving up on incomprehensible worlds is not a reassuring move, just an unavoidable one. Similar to accepting that if you die in 3 seconds, you can’t do much about it. Hope you don’t, btw.
But yeah currently seems to me that the majority of sim juice comes from civs who want to get to know the neighbors before they meet, so they can prepare the appropriate welcome mat (tone: cynical). Let’s send an actualized preference for strong egalitarianism, yeah? (doesn’t currently look likely that we will, would be a lot of changes from here before that became likely.)
(Also, hopefully everything I said works for either structural realism or mathematical universe. Structural realism without mathematical universe would be an example of the way things could be wacky in ways permanently beyond the reach of logic, while still living in a universe where logic mostly works.)
We have to infer how reality works somehow.
I’ve been poking at the philosophy of math recently. It really seems like there’s no way to conceive of a universe that is beyond the reach of logic except one that also can’t support life. Classic posts include unreasonable effectiveness of mathematics, what numbers could not be, a few others. So then we need epistemology.
We can make all sorts of wacky nested simulations and any interesting ones, ones that can support organisms (that is, ones that are Turing complete), can also support processes for predicting outcomes in that universe, and those processes appear to necessarily need to do reasoning about what is “simple” in some sense in order to work. So that seems to hint that algorithmic information theory isn’t crazy (unless I just hand waved over a dependency loop, which I totally might have done, it’s midnight), which means that we can use the equivalence of Turing complete structures to assume we can infer things about the universe. Maybe not solononoff induction, but some form of empirical induction. And then we’ve justified ordinary reasoning about what’s simple.
Okay, so we can reason normally about simplicity. What universes produce observers like us and arise from mathematically simple rules? Lots of them, but it seems to me the main ones produce us via base physics, and then because there was an instance in base physics, we also get produced in neighboring civilizations’ simulations of what other things base physics might have done in nearby galaxies so as to predict what kind of superintelligent aliens they might be negotiating with before they meet each other. Or, they produce us by base physics, and then we get instantiated again later to figure out what we did. Ancestor sims require very good outcomes which seem rare, so those branches are lower measure anyway, but also ancestor sims don’t get to produce super ai separate from the original causal influence.
Point is, no, what’s going on in the simulations is nearly entirely irrelevant. We’re in base physics somewhere. Get your head out of the simulation clouds and choose what you do in base physics, not based on how it affects your simulators’ opinion of the simulation’s moral valence. Leave that sort of crazy stuff to friendly ai, you can’t understand superintelligent simulators which we can’t even get evidence exist besides plausible but very galaxy brain abstract arguments.
(Oh, might be relevant that I’m a halfer when making predictions, thirder when choosing actions—see anthropic decision theory for an intuition on that.)
Thank you, I feel inclined to accept that for now.
But I’m still not sure, and I’ll have to think more about this response at some point.
Edit: I’m still on board with what you’re generally saying, but I feel skeptical of one claim:
My intuition tells me there will probably be superior methods of gathering information about superintelligent aliens. To me, it seems like the most obvious reason to create sims would be to respect the past for some bizarre ethical reason, or for some weird kind of entertainment, or even to allow future aliens to temporarily live in a more primitive body. Or perhaps for a reason we have yet to understand.
I don’t think any of these scenarios would really change the crux of your argument, but still, can you please justify your claim for my curiosity?
Sims are very cheap compared to space travel, and you need to know what you’re dealing with in quite a lot of detail before you fly because you want to have mapped the entire space of possible negotiations in an absolutely ridiculous level of detail.
Sims built for this purpose would still be a lot lower detail than reality, but of course that would be indistinguishable from inside if the sim is designed properly. Maybe most kinds of things despawn in the sim when you look away, for example. Only objects which produce an ongoing computation that has influence on the resulting civ would need modeling in detail. Which I suspect would include every human on earth, due to small world effects, the internet, sensitive dependence on initial conditions, etc. Imagine how time travel movies imply the tiniest change can amplify—one needs enough detail to have a good map of that level of thing. Compare weather simulation.
Someone poor in Ghana might die and change the mood of someone working for ai training in Ghana, which subtly affects how the unfriendly AI that goes to space and affects alien civs is produced, or something. Or perhaps there’s an uprising when they try to replace all human workers with robots. Modeling what you thought about now helps predict how good you’ll be at the danceoff in your local town which affects the posts produced as training data on the public internet. Oh, come to think of it, where are we posting, and on what topic? Perhaps they needed to model your life in enough detail to have tight estimates of your posts, because those posts affect what goes on online.
But most of the argument for continuing to model humans seems to me to be the sensitive dependence on initial conditions, because it means you need an unintuitively high level of modeling detail in order to estimate what von Neumann probe wave is produced.
Still cheap—even in base reality earth right now is only taking up a little more energy than its tiny silhouette against the sun’s energy output in all directions. A kardashev 2 civ would have no problem fuelling an optimized sim with a trillion trillion samples of possible aliens’ origin processes. Probably superintelligent kardashev 1 even finds it quite cheap, could be less then earth’s resources to do the entire sim including all parallel outcomes.
I should also add:
I’m pretty worried that we can’t understand the universe “properly” even if we’re in base physics! It’s not yet clearly forbidden that the foundations of philosophy contain unanswerable questions, things where there’s a true answer that affects our universe in ways that are not exposed in any way physically, and can only be referred to by theoretical reasoning; which then relies on how well our philosophy and logic foundations actually have the real universe as a possible referent. Even if they do, things could be annoying. In particular, one possible annoying hypothesis would be if the universe is in Turing machines, but is quantum—then in my opinion that’s very weird but hey at least we have a set in which the universe is realizable. Real analysis and some related stuff gives us some idea things can be reasoned about from within a computation based understanding of structure, but which are philosphically-possibly-extant structures beyond computation, and whether true reality can contain “actual infinities” is a classic debate.
So sims are small potatoes, IMO. Annoying simulators that want to actively mess up our understandings are clearly possible but seem not particularly likely by models I believe right now; seems to me they’d rather just make minds within their own universe; sims are for pretending to be another timeline or universe to a mind you want to instantiate, whatever your reason for that pretense. If we can grab onto possible worlds well enough, and they aren’t messing up our understanding on purpose, then we can reason about plausible base realities and find out we’re primarily in a sim by making universe sims ourselves and discovering the easiest way to find ourselves is if we first simulate some alien civ or other.
But if we can’t even in principle have a hypothesis space which relates meaningfully to what structures a universe could express, then phew, that’s pretty much game over for trying to guess at tegmark 4 and who might simulate us in it or what other base physics was possible or exists physically in some sense.
My giving up on incomprehensible worlds is not a reassuring move, just an unavoidable one. Similar to accepting that if you die in 3 seconds, you can’t do much about it. Hope you don’t, btw.
But yeah currently seems to me that the majority of sim juice comes from civs who want to get to know the neighbors before they meet, so they can prepare the appropriate welcome mat (tone: cynical). Let’s send an actualized preference for strong egalitarianism, yeah? (doesn’t currently look likely that we will, would be a lot of changes from here before that became likely.)
(Also, hopefully everything I said works for either structural realism or mathematical universe. Structural realism without mathematical universe would be an example of the way things could be wacky in ways permanently beyond the reach of logic, while still living in a universe where logic mostly works.)