If glargs were some object with moral worth but not the ability to make decisions, then finding yourself able to determine the fate of 3^^^3 glargs wouldn’t be unusually asymmetric, and so wouldn’t have a probability penalty.
There’s more to the asymmetry than just the anthropic considerations though. Like, having that much influence over anything you actually care about is super improbable unless you’ve already pumped a lot of optimization into the world.
It is perhaps worth noting that this happens a lot in real life and yet people don’t seem to have any problem ignoring the asymmetry. Specifically I am thinking about people who think that God has chosen them and only them for some sacred and very important task, and similar delusions. Even ignoring trying to explain why you would find yourself as God’s chosen prodigy, which is itself pretty weird, how do you explain why any person roughly as ordinary as you would be singled out in the first place?
Even so, the kicker remains that someone like Eliezer who’s trying to take over I mean optimize the universe would still seem to need to grapple with the lingering anthropic improbabilities even if he could justify his apparently-naively asymmetrical expectation of future world optimization by pointing at the causal chain stemming from his past optimization, ’cuz he would also have to explain how he ended up as himself in such a seemingly promising strategic position in the first place. But I don’t know who actually tries to reason via anthropics qua anthropics these days, instead of daringly adopting some garbled bastardization of UDT-like reasoning, which I think lets you neatly dissolve what looks like mysterious double counting.
There’s more to the asymmetry than just the anthropic considerations though. Like, having that much influence over anything you actually care about is super improbable unless you’ve already pumped a lot of optimization into the world.
It is perhaps worth noting that this happens a lot in real life and yet people don’t seem to have any problem ignoring the asymmetry. Specifically I am thinking about people who think that God has chosen them and only them for some sacred and very important task, and similar delusions. Even ignoring trying to explain why you would find yourself as God’s chosen prodigy, which is itself pretty weird, how do you explain why any person roughly as ordinary as you would be singled out in the first place?
Even so, the kicker remains that someone like Eliezer who’s trying to take over I mean optimize the universe would still seem to need to grapple with the lingering anthropic improbabilities even if he could justify his apparently-naively asymmetrical expectation of future world optimization by pointing at the causal chain stemming from his past optimization, ’cuz he would also have to explain how he ended up as himself in such a seemingly promising strategic position in the first place. But I don’t know who actually tries to reason via anthropics qua anthropics these days, instead of daringly adopting some garbled bastardization of UDT-like reasoning, which I think lets you neatly dissolve what looks like mysterious double counting.