I don’t know about “by a paperclip maximizer”, but one thing that stands out to me:
If we’re in a simulation, we could be in a simulation where the simulator did 1e100 rollouts from the big bang forward, and then collected statistics from all those runs.
But we could also be in a simulation where the simulator is doing importance sampling—that is, doing fewer rollouts from states that tend to have very similar trajectories given mild perturbations, and doing more rollouts from states that tend to have very different trajectories given mild perturbations.
If that’s the case, we should find ourselves living in a world where events seem to be driven by coincidences and particularly by things which are downstream of chaotic dynamics and which had around a 50⁄50 chance of happening vs not. We should find more such coincidences for important things than for unimportant things.
Hey, maybe it’s not our fault we live in the clown world. Maybe the clown world is a statistical inevitability.
Soooo, muddying the waters makes you more alive? Like, IF you robustly steer the world to some predictable outcome (aligned asi or some particular type of misaligned asi) then you get instantiated less? something something chaos reigns?
a more extreme version of the “god created worlds starting from the best, and kept making more until running out of just-barely-good-enough ones”. in this one, it would be a world-creator which has no interest in seeking out making good worlds, focusing on the ones that are most difficult to understand. if that’s the case, we should expect to be in an impactful part of the underlying real world, and so should focus our actions there. we’ll tend to observe continuing to be in an impactful part of the world, but in the underlying real worlds that impact the simulators, we’ll be having impacts that affect things in ways (hopefully somewhat, if we’re skilled and lucky) closer to what we hope for.
Let’s say I want to evaluate an algorithmic Texas Hold’em player against a field of algorithmic opponents.
The simplest approach I could take would be pure monte-carlo: run the strategy for 100 million hands and see how it does. This works, but wastes compute.
Alternatively, I could use the importance sampled approach:
Start with 100,000 pre-flop scenarios (i.e., all players have received their pocket cards, button position is fixed, no community cards yet)
Do 100 rollouts from each scenario
Most rollouts won’t be “interesting” (e.g., player has 7-2o UTG, player folds every time → EV = 0BB). If the simulation hits these states, you can say with high confidence how they’ll turn out, so additional rollouts won’t significantly change your EV estimate—you’ve effectively “locked in” your EV for the “boring” parts of the pre-flop possibility space.
Pick the 10,000 highest-variance preflop scenarios. These are cases where your player doesn’t always fold, and opponents didn’t all fold to your player’s raise). e.g.
AA in position where you 3-bet and everyone folds → consistently +3BB.
KQs facing a 3-bet → sometimes win big, sometimes lose big → super high variance
Run 1,000 rollouts for each of these high-variance scenarios.
Figure out which flops generate the highest variance over those 1,000 rollouts.
If your player completely missed the flop while another player connected and bet aggressively, your player will fold every time—low variance, predictable EV.
If your player flopped two pair or your opponent is semi-bluffing a draw, those are high-variance situations where additional rollouts provide valuable information.
Etc etc through each major decision point in the game
By skipping rollouts once I know what the outcome is likely to be, I can focus a lot more compute on the remaining scenarios and come to a much more precise estimate of EV (or whatever other metric I care about).
Would this help with the simulation goal hypothesized in the OP? It’s asking how often different types of AGIs would be created. A lot of the variance is probably carried in what sort of species and civilization is making the AGI, but some of it is carried by specific twists that happen near the creation of AGI. Getting a president like Trump and having him survive the (fairly likely) assasination attempt(s) is one such impactful twist. So I guess sampling around those uncertain impactful twists would be valuable in refining the estimate of, say, how frequently a relatively wise and cautious species would create misaligned AGI due to bad twists and vice-versa.
I don’t know about “by a paperclip maximizer”, but one thing that stands out to me:
If we’re in a simulation, we could be in a simulation where the simulator did 1e100 rollouts from the big bang forward, and then collected statistics from all those runs.
But we could also be in a simulation where the simulator is doing importance sampling—that is, doing fewer rollouts from states that tend to have very similar trajectories given mild perturbations, and doing more rollouts from states that tend to have very different trajectories given mild perturbations.
If that’s the case, we should find ourselves living in a world where events seem to be driven by coincidences and particularly by things which are downstream of chaotic dynamics and which had around a 50⁄50 chance of happening vs not. We should find more such coincidences for important things than for unimportant things.
Hey, maybe it’s not our fault we live in the clown world. Maybe the clown world is a statistical inevitability.
When the bullet missed Trump by half an inch I made a lot of jokes about us living in an importance-sampled simulation.
Soooo, muddying the waters makes you more alive? Like, IF you robustly steer the world to some predictable outcome (aligned asi or some particular type of misaligned asi) then you get instantiated less? something something chaos reigns?
Exactly!
a more extreme version of the “god created worlds starting from the best, and kept making more until running out of just-barely-good-enough ones”. in this one, it would be a world-creator which has no interest in seeking out making good worlds, focusing on the ones that are most difficult to understand. if that’s the case, we should expect to be in an impactful part of the underlying real world, and so should focus our actions there. we’ll tend to observe continuing to be in an impactful part of the world, but in the underlying real worlds that impact the simulators, we’ll be having impacts that affect things in ways (hopefully somewhat, if we’re skilled and lucky) closer to what we hope for.
I’m sorry, I don’t get it. Why would it be doing more sampling around divergent points?
Let’s say I want to evaluate an algorithmic Texas Hold’em player against a field of algorithmic opponents.
The simplest approach I could take would be pure monte-carlo: run the strategy for 100 million hands and see how it does. This works, but wastes compute.
Alternatively, I could use the importance sampled approach:
Start with 100,000 pre-flop scenarios (i.e., all players have received their pocket cards, button position is fixed, no community cards yet)
Do 100 rollouts from each scenario
Most rollouts won’t be “interesting” (e.g., player has 7-2o UTG, player folds every time → EV = 0BB). If the simulation hits these states, you can say with high confidence how they’ll turn out, so additional rollouts won’t significantly change your EV estimate—you’ve effectively “locked in” your EV for the “boring” parts of the pre-flop possibility space.
Pick the 10,000 highest-variance preflop scenarios. These are cases where your player doesn’t always fold, and opponents didn’t all fold to your player’s raise). e.g.
AA in position where you 3-bet and everyone folds → consistently +3BB.
KQs facing a 3-bet → sometimes win big, sometimes lose big → super high variance
Run 1,000 rollouts for each of these high-variance scenarios.
Figure out which flops generate the highest variance over those 1,000 rollouts.
If your player completely missed the flop while another player connected and bet aggressively, your player will fold every time—low variance, predictable EV.
If your player flopped two pair or your opponent is semi-bluffing a draw, those are high-variance situations where additional rollouts provide valuable information.
Etc etc through each major decision point in the game
By skipping rollouts once I know what the outcome is likely to be, I can focus a lot more compute on the remaining scenarios and come to a much more precise estimate of EV (or whatever other metric I care about).
Thanks, I get it now.
Would this help with the simulation goal hypothesized in the OP? It’s asking how often different types of AGIs would be created. A lot of the variance is probably carried in what sort of species and civilization is making the AGI, but some of it is carried by specific twists that happen near the creation of AGI. Getting a president like Trump and having him survive the (fairly likely) assasination attempt(s) is one such impactful twist. So I guess sampling around those uncertain impactful twists would be valuable in refining the estimate of, say, how frequently a relatively wise and cautious species would create misaligned AGI due to bad twists and vice-versa.
Hm.
New EA cause area just dropped: Strategic variance reduction in timelines with high P(doom).
BRB applying for funding
Great example!
They provide more surprising information, as I understand