Anthropically Blind: the anthropic shadow is reflectively inconsistent
For the purposes of this post, the anthropic shadow is the type of inference found in How Many LHC Failures Is Too Many?.
“Anthropic principle! If the LHC had worked, it would have produced a black hole or strangelet or vacuum failure, and we wouldn’t be here!”
In other words, since we are “blind” to situations in which we don’t exist, we must adjust how we do bayesian updating. Although it has many bizarre conclusions, it is more intuitive than you think and quite useful!
There are many similar applications of anthropics, such as Nuclear close calls and Anthropic signature: strange anti-correlations.
This actually has implications for effective altruism. Since we are so early into humanity’s existence, we can infer from the anthropic shadow that humans will probably soon die out. Also see The Hero With A Thousand Chances.
More practically, the anthropic shadow should give us useful advice on how to reason about personally risky activities like driving or perhaps even aging. I have not actually seen any advice based on this principle, but theoretically there should be some conclusions you could draw.
The problem, as you probably deduced from the title, is that it is reflectively inconsistent.
Central Claim: Someone using the anthropic shadow should update their decision making to no longer use it. This can be justified with their current decision making procedure.
(This also suggests that if you used it in the past, that was probably the wrong thing to do.)
A weaker (and obvious) claim that is also sometimes called the anthropic shadow is that we do not have experience with situations in which we have died. I agree with this version, but isn’t what I will be arguing against.
Note that I am not the first to notice paradoxes with the anthropic shadow. See No Anthropic Evidence for example. I have not yet seen the result about reflective inconsistency though, hence why I am making this post.
I also introduce the concepts of “Anthropic undeath”, “Anthropic angel” (how you would explain an absurdly large number of weird coincidences having to do with death), “Fedora shadow”, and apply the central claim to a couple examples. To my knowledge, these contributions are novel.
Anthropically undead: ghosts are as good as gone
This section contains the most general form of the argument. (This could be mathematically formalized; I just haven’t gotten around to doing it.) If it seems strange to you, a generalized version of this section might also work.
First, we establish the basic framing of how we will check if something is reflectively consistent. Imagine yourself before a catastrophe potentially happens. You are an expected utility maximizer (as all good agents should be, although this assumption can probably be weakened). You are trying to come up with a policy for your future-self to follow.
Consider the following scenario: in any situation that you would die, imagine instead that you become an agent with only one choice each time it takes an action: “do nothing”. This state is still given the same utility as before (including the utility change from physically dying (a reinforcement learner would stop getting rewards, for example)), but as an agent you never stop existing. Call this unreal scenario “anthropic undeath”.
Optimizing the utility of the real scenario is the same as optimizing utility in anthropic undeath, because the agent choosing “do nothing” in the anthropic undeath scenario has the same physical effect as what actually happens in the real scenario when the agent is dead. I call this the “ghosts are as good as gone” principle.
The anthropic undeath scenario has no anthropic shadow, because the agent never stops existing. Thus, the optimal policy never uses anthropic shadow in its reasoning. The optimal policy in the real scenario is the same by the principle in the previous paragraph.
(Also see this comment for an explanation of why we can consider a physically dead entity to be an agent.)
Q.E.D.
A worked example: calibrated forecasts of the deadly coin
Imagine a coin with a side labelled “zero” and a side labelled “one”. It will get flipped a bunch of times. Before each flip, you will give your credence p that the coin will come up one. Letting x be the result of the coin flip, you will be given 1000 (1-(p-x)²) utils. (We can imagine the utils representing cookies being given to your friend Timmy.) Notice that this is a proper scoring rule. Also, if the coin comes up one, the game ends and you die. The utils still count if you die (because Timmy still gets the cookies). (Notice how the “you die” part has no effect on the game since the game is over anyways. We could stop here using the same argument as the previous section, but we will work it out to illustrate why the anthropic shadow fails.)
You now must come up with a policy that you will follow during the game.
You have two hypotheses about the coin, both with 50% credence apriori:
The coin always comes up zero.
The coin has an independent 50% chance of coming up zero each time.
Consider a situation in which you have observed 7 zeros in a row. What p should you choose? The anthropic shadow suggests you have gotten no information, and thus the chance of one is 25%.
However, this is incorrect as a policy. Before the game begins, you reason as follows about the state after 7 coin flips:
There is a 50% chance you are in scenario 1. This contributes 1000 (0.5 (1-p²)) to the expected value.
There is a 50% chance you are in scenario 2.
There is a 1 − 2⁻⁷ chance that one of the coin flips resulted in one. In this case, the game is over, you are dead, and nothing is contributed to the expected utility.
There is a 2⁻⁷ chance that you get 7 zeros in a row. There is a 50% chance that the next flip is a zero, and a 50% chance that it is a one. Since scenario 2 is itself a 50% chance, this contributes 1000 (2⁻⁸ (1-(0.5 p² + 0.5 (p-1)²))) to the expected utility.
Maximizing your expected utility of 1000 (0.5 (1-p²) + 2⁻⁸ (1-(0.5 p² + 0.5 (p-1)²))), you find the optimal p is 1⁄258, equivalent to odds of 1 to 257 (about 1⁄85 of the anthropic shadow estimate). This is exactly the same as the probability you get by doing normal bayesian updating!
To summarize, the anthropic shadow would have you say:
“Anthropic principle! If the coin came up one, I would have died, and I wouldn’t be here!”
And would lose you about 30 expected utils on the 7th round alone! At the beginning of the game when you are setting your policy, don’t do that!
You might say, “but what I really want is to not die, I don’t care about maximizing my calibration!”. If so, you lack faith in the Litany of Tarski. Vladimir Nesov has an example in No Anthropic Evidence where the only goal is survival. Again, the optimal policy agrees with not using anthropic shadow. If you have a clear view of your utility function (including in states where you no longer exist), it is best for your credences to calibrated!
Anthropic Angels and Lucky Fedoras
Okay, but what should we do if we observed a huge amount of evidence that weird coincidences happen around deadly things, like a zillion LHC accidents or perfect anti-correlation between pandemics and recessions in Anthropic signature: strange anti-correlations. Surely at some point I must relent and go “okay, the anthropic shadow is real”. And if that is so, than even a little bit of evidence should at least make us a little worried about the anthropic shadow.
No.
Treating ideal reasoning as an approximation to Solomonoff Induction, we find that there is no anthropic shadow hypothesis. However, there are what I call anthropic angel hypotheses. These are hypotheses that posit that there is some mysterious force that protects you from death, perhaps via rejection sampling. One such hypothesis is quantum immortality[1].
An important thing to understand about anthropic angels though is that they typically don’t stop on the next observation. If the LHC would destroy humanity but accidents keep happening, will I protest to stop the LHC? No, because there is no reason to think that the accidents will stop.
Of course, if you are worried that the anthropic angel might fail in the future, you still might be cautious. However, the more times you get saved, the more you can trust the angel. This is the exact opposite of the anthropic shadow!
Keep in mind also that the type of reasoning behind anthropic angels also applies to things other than death. Death isn’t special in this context! Suppose that you are a forgetful person, but you have a lucky fedora. You notice that there are weird coincidences that prevent you from losing your fedora. Is there a “fedora shadow”, whereby the version of you currently wearing the fedora can’t observe scenarios where the fedora is missing, and thus you must adjust your bayesian updating? No. Given enough evidence, you would need to conclude that there is a “fedora angel” that influences events to save your lucky fedora for some reason, instead of a fedora shadow whereby from your luck you make fearful inferences.
What would convince me that the anthropic shadow is real?
So—taking into account the previous cancellation of the Superconducting Supercollider (SSC) - how many times does the LHC have to fail before you’ll start considering an anthropic explanation? 10? 20? 50? - How Many LHC Failures Is Too Many?
Since the anthropic shadow is not reflectively consistent, I am convinced that there is no object-level evidence that would persuade me. No amount of LHC weirdness, Nuclear close calls, strange anti-correlations, etc… would change my mind. The evidence that is normally presented for the anthropic shadow is instead (extremely weak thus far) evidence for an anthropic angel.
However, to make my belief pay rent, I should specify what it excludes. Here is what I would count as evidence for the anthropic shadow: if people applying the concept of anthropic shadow to personal risk of death, such as car crashes, consistently make better decisions than those who do not. Note that to be persuasive, there shouldn’t be simpler explanations (like it cancelling out some other bias, or in an extreme case them actually using the anthropic angel).
Curiously, I have not seen anyone apply the anthropic shadow in this way (except ironically). If anyone tries, I strongly anticipate it will be systematically worse.
Applications
LHC failures, Nuclear close calls, Strange anti-correlations, etc...
These are basically the same as the worked out example above. A string of identical coin flips might seem unusual, but they do not mean we should use the anthropic shadow.
For example, for the strange anti-correlations, the probability mass of “x-risk and we are alive and unexplained anti-correlation” is the same as “no x-risk and unexplained anti-correlation”, so good policy does not use anthropic shadow.
Also see this comment (and the ones around it) for a more indepth discussion of how it applies to things like the LHC (with many independent parts that could fail).
The Hero With A Thousand Chances
“Allow me to make sure I have this straight,” the hero said. “The previous hero said I would definitely fail for what reason?”
“Shades of Ahntharhapik, very serious!” said Aerhien.
“And how did he fare?” asked the hero?
“Pretty typically. He tried to destroy the Dust with something called a ‘Nuclear bomb’. Didn’t get very far though. In the uranium mine we discovered a magical artifact that turned half of the miners into zombies who started eating the Dust.” replied Aerhien.
Ghandhol chipped in “The hero then said that the shades of Ahntharhapik saved us, but it wouldn’t next time, and thus we should summon a hero from the same world to continue the advanced weaponry program he started.”
“🤦 so the previous hero was trying to imply that each time you survived, that was evidence that you were bad at survival.” sighed the hero.
“Yes, there is no other explanation!” exclaimed Aerhien.
The hero mocked “Just like if you see a thousand coin flips come up heads, the only explanation is that you got really really lucky (instead of checking if both sides of the coin are heads)?”
The whole council went silent.
“Look, out of all the possible worlds that could’ve summoned me, the ones that weren’t good at surviving passed long ago. And the worlds that already succeeded wouldn’t be summoning me to fight the Dust either. Your world genuinely does have a kind of luck. We can’t agree to disagree.” said the hero.
“What kind of cruel luck is this? Is it the Counter-Force?” replied Aerhien.
“So to speak. Hmm. If your world was generated by resampling, you would’ve defeated the Dust long ago (or if that also caused a resample, a more sensible stalemate). If your world branches off at the moment of death, the anomalies would happen much later. If someone tried to shoot you, the gun would go off, but the bullet would bounce.” thought the hero.
The hero then had an insight, saying “its simple really. We are in a fantasy novel of some sort (or maybe a fanfic? something in this vicinity). Normally fictional characters don’t figure this out, but as a good bayesian reasoner, even these kinds of conclusions can’t escape me! Especially when the alternative is believing in a thousand identical coin flips.”
The Doomsday argument and Grabby Aliens
When I was first writing this, I thought that my argument ruled out the Doomsday argument, but permitted Grabby aliens. Turns out, it vaguely argues against both, but not in a strong enough way to conclude they are reflectively inconsistent. It is quite similar to the Self-Indication Assumption Doomsday argument rebuttal (the main difference being that the likelihood of being born is a sublinear function of the number of humans under my argument).
Let p be that probability that a randomly generated human will be me, Christopher King (as defined by my experiences). Let q be the probability that a randomly generated civilization in the universe will contain Christopher King. What policy should I choose so that Christopher King is well calibrated?
For the doomsday argument, the example hypotheses are:
There will be 120 billion humans.
There will be 10ˡ⁰⁰ humans.
And for grabby aliens:
The universe will be taken over by grabby aliens.
The universe is and will be filled with quiet aliens (for simplicity, we will say that it has the same frequency f of intelligent civilizations per year per meter cubed of unconquered space as in hypothesis 3).
Both arguments posit that we need to explain earliness. However, hypotheses 2 and 4 also have early humans. For the doomsday argument, the probability of Christopher King being among the first ~60 billion humans is (1-(1-p)^(60 billion)) under both hypotheses. So being early is not evidence either way for 1 or 2. For grabby aliens, the probability of Christopher King being present in the first ~14 billion years of the universe is (1-(1-q)^(f * 14 billion years * (4 x 10⁸⁰ m³))) under hypothese 4, and slightly less under hypothese 3 because some space is already conquered by grabby aliens. So the likelihood ratio favors hypothesis 4.
The problem is that this technically isn’t a case of reflective inconsistency, because I wouldn’t be able to remember and reflect before the universe started, of course. I worry in particular that there is no reason for “pre-existence” Christopher King to have the same priors as “embodied” Christopher King.[2] See also Where Recursive Justification Hits Bottom.
However, also see SSA rejects anthropic shadow, too for ways in which popular anthropic theories handle the pre-existence anthropic shadow.
Conclusion
So in summary, we can update on the fact that we have survived. Counter-intuitively, we should treat the fact that it is “humanity 1, x-risk 0” or “survive 1, death 0″ the same as we would treat any other statistic of that form. For example, we can update normally on the fact that we survived the Cold War, the fact that nothing has randomly killed humanity (like pandemics) yet, and on a personal level the fact that we survive things like car crashes. This was shown on the basis of reflective consistency.
- ↩︎
As far as I know, our current Everett branch is best explained by Born’s rule without quantum immortality (QI). There are infinitely many branches where, in the future, we start seeing evidence of it, but in each such branch Solomonoff induction (SI) will only require a finite amount of evidence to update (bounded by a constant determined by how complicated it is to implement QI in the programming language). That is what it means for SI to be universal: it works in every Everett branch, not just the typical ones.
On the other hand, QI can have infinitely bad calibration in finite time. If they are in a normal Born’s rule branch and they die, their prediction for if they would die (and any consequences thereof) would have infinitely many bits of surprisal. This could be quite bad if you cared about the state of the world in the Born’s rule branch after your death!
- ↩︎
My argument works fine under SIA and SSA (assuming that physicists are correct about the universe being infinite), but there are more exotic sampling assumptions like “SSA but only in the observable universe and also no Boltzmann brains” where it can fail. This hypothesis would have positive probability under UDASSA, for example. Even though its weird, I don’t see a reflective inconsistency.
- SSA rejects anthropic shadow, too by (27 Jul 2023 17:25 UTC; 61 points)
- Learning as you play: anthropic shadow in deadly games by (12 Aug 2023 7:34 UTC; 37 points)
- 's comment on Arguments for optimism on AI Alignment (I don’t endorse this version, will reupload a new version soon.) by (17 Oct 2023 23:05 UTC; 2 points)
- 's comment on Practical anthropics summary by (29 Jun 2023 2:52 UTC; 1 point)
I think the application to the Hero With A Thousand Chances is partly incorrect because of a technicality. Consider the following hypothesis: there is a huge number of “parallel worlds” (not Everett branches, just thinking of different planets very far away is enough) each fighting the Dust. Every fight each of those worlds summons a randomly selected hero. Today that hero happened to be you. The world that happened to summon you has survived the encounter with the Dust 1079 times before you. The world next to it has already survived 2181, and the other one was destroyed during the 129th attempt.
This hypothesis explains the observation of the hero pretty well—you can’t get summoned to a world that’s destroyed or has successfully eliminated the Dust, so of course you get summoned to a world that is still fighting. As for the 1079 attempts before you, you can’t consider that a lot of evidence for fighting the Dust being easy, maybe you’re just 1080th entry in their database and can only be summoned for the 1080th attempt, there was no way for you to observe anything else. Under this hypothesis, you personally still have a pretty high chance of dying—there’s no angel helping you, that specific world really did get very lucky, as did lots of other worlds.
So, anthropic angel/”we’re in a fanfic” hypothesis explains observations just as well as this “many worlds, and you’re 1079th on their database” hypothesis, so they’re updated by the same amount, and at least for me the “many worlds” hypothesis has much higher prior than “I’m a fanfic character” hypothesis.
Note, this only holds from hero’s perspective: from Aerhien’s POV she really has observed herself survive 1079 times, which counts as a lot of evidence for the anthropic angel.
Yeah, the hero with a thousand chances is a bit weird since you and Aerhien should technically have different priors. I didn’t want to get too much into it since it’s pretty complicated, but technically you can have hypotheses where bad things only start happening after the council summons you.
This has weird implications for the cold war case. Technically I can’t reflect against the cold war anthropic shadow since it was before I was born. But a hypothesis where things changed when I was born seems highly unnatural and against the Copernican principle.
In your example though, the hypothesis that things are happening normally is still pretty bad to other hypotheses we can imagine. That’s because there will be a much larger number of worlds that are in a more sensible stalemate with the Dust, instead of “incredibly improbable stuff happens all the time”. Like even “the hero defeats the Dust normally each time” seems more likely. The less things that need to go right, the more survivors there are! So in your example, it is still a more likely hypothesis that there is some mysterious Counter-Force that just seems like it is a bunch of random coincides, and this would be a type of anthropic angel.
I think the real debate about anthropic shadows is whether anthropic angels exist.
When I think of the archetypal example of anthropic shadows, I think something like disease or war where destruction follows a positively skewed distribution, and some naive maximum likelihood fits that permit long-tailedness suggest that there’s a serious likelihood of world-ending events.
Here, as I understand it, the counterargument is that there is a gap in observations around the size that would be world-ending, so we should fit a model fit smaller tails to match this gap. Such a model seems like “anthropic angels” to me.
You’re probably right that Solomonoff induction recommends anthropic angels. But Solomonoff induction is weird and incapable of reasoning about its own death, so I’m not convinced it tells us anything.
I would be inclined to say that correct anthropic reasoning does normal Bayesian updates but avoids priors that postulate anthropic angels.
To be clear, Anthropic angels aren’t necessary for this argument to work. My deadly coin example didn’t have one, for example.
The reason I introduce Anthropic angels is to avoid a continuity counter-argument. “If you saw a zillion LHC accidents, you’d surely have to agree with the Anthropic shadow, no matter how absurd you claim it is! Thus, a small amount of LHC accidents is a little bit for evidence for it.” Anthropic angels show the answer is no, because LHC accidents are not evidence for the anthropic shadow.
Like, it seems unnatural to give it literally 0% probability (see 0 And 1 Are Not Probabilities).
If there are weird acasual problems that the Anthropic angel can cause, I’m guessing you can just change your decisions without changing your beliefs. I haven’t thought too hard about it though.
No, anthropic angels would literally be some mechanism that saves us from disasters. Like if it turned out Superman is literally real, thinks the LHC is dangerous, and started sabotaging it. Or it could be some mechanism “outside the universe” that rewinds the universe.
Keep in mind that the problems with maximum likelihood have nothing to do with death. That should be the main takeaway from my article, that we shouldn’t use special reasoning to reason about our demise.
In the case of maximum likelihood, it is also bad for:
Estimating when we will meet aliens
Forecasting the stock market
Being a security guard
etc...
Which is why you should use bayesian reasoning with a good prior instead.
Suppose you have N coins. If all N coins come up 1, you die. For each coin, you have 50:50 credence about whether it always comes up 0, or if it can also come up 1.
For N=1, it reduces to your case. For N>1, you get an anthropic shadow, which means that even if you’ve had a bunch of flips where you’ve survived, you might actually have to conclude that you’ve got a 1-in-4 chance of dying on your next flip.
Okay, I think our crux comes from the slight ambiguity from the term “anthropic shadow”.
I would not consider that anthropic shadow, because the reasoning has nothing to do with anthropics. Your analysis is correct, but so is the following:
The “ghosts are as good as gone” principle implies that death has no special significance when it becomes to bayesian reasoning.
Going back to the LHC example, if the argument worked for vacuum collapse, it would also work for the LHC doing harmless things (like discovering the Higg’s boson or permanently changing the color of the sky or getting a bunch of physics nerds stoked or granting us all immortality or what not) because of this principle (or just directly adapting the argument for vacuum collapse to other uncertain consequences of the LHC).
In the bird example, why would the baguette dropping birds be evidence of “LHC causes vacuum collapse” instead of, say, “LHC does not cause vacuum collapse”? What are the probabilities for the four possible combinations?
I think we basically agree now. I think my original comments were somewhat confused, but also the deadly coin model was somewhat confused. I think the best model is a variant of the N-coin model where one or more of the coins are obscured, and I think in this model your proof goes through to show that you should independently perform Bayesian updates on each coin, and that since you don’t get information about the obscured coin, you should not update on it in a anthropic shadow style way.
I think the thing that makes deadly coins different from LHC accidents and skew-distributed death counts is that deadly coins lack variance, so you cannot extrapolate the upper tail from the bulk of the distribution.
We could think of the LHC as having a generating process similar to a log-normal distribution. There are a number of conditions that are necessary before it can be turned on (which would be Bernoulli-distributed variables), as well as settings which affect the extent to which it is turned on (based on the probed energy levels). These all interact multiplicatively, with e.g. it only being turned on if all of the conditions are met. Of course it is not literally lognormally distributed because it is possible for it to not be turned on at all, but otherwise multiplicative interactions are the same as what happens in log-normal distributions. Multiplicatively interacting variables include:
The vacuum must be maintained against leaks
Machinery must be protected against outside forces
Power must be present
It must be configured to probe some energy level
etc.
(Disclaimer: I know little about the LHC.)
If all of the LHC accidents go the same way, say a cleaning company keeps breaking things, then it is reasonable to infer an anthropic angel such as “the cleaning company is incompetent/malevolent and will keep breaking things until you replace them”. On the other hand, if the LHC accidents are highly varied in an anticorrelated way as predicted by the anthropic shadow (e.g. birds become more likely to drop baguettes onto machinery as you increase the energy levels), then I think you can infer an anthropic shadow, because this shows that the latent generator of LHC activation is highly varied, which according to the model would lead to highly varied outputs if not for anthropic shadows.
Not wrong. I haven’t thought in detail about this, maybe there are some better solutions.
… which in turn would result in a distribution of damage with shorter tails, no?
A further thought:
For diseases or war or asteroid impacts where the means by which greater scales causes greater damage, I think an argument along these lines basically goes through.
But for things like the LHC or AI or similar, there is an additional parameter: what is the relationship between scale and damage? (E.g. what energy level does the LHC need to probe before it destroys the earth?)
Some people might want to use anthropic shadow reasoning to conclude things about this parameter. I think something along the lines of OP’s argument goes through to show that you can’t use anthropic shadow reasoning to infer things about this parameter beyond the obvious.
The equivalent coin example would be if you were uncertain about whether the coin will kill you.
Or one could use my other multi-coin example, but where there is one of the coins that you can’t see. So you can’t update about whether it is 50:50 or 100:0.
I have an impression is that what you call here ‘anthropic shadow’ is the idea of quantum immortality. One can’t ’use anthropic shadow”.
In that case your central thesis is reasonable for most altruistic policies. If I play a series of Russian roulette rounds and “won”, it will be still a bad policy to continue playing Russian roulette.
More over, antithesis is correct: if I survive long enough, I am probably underestimating the background risks. This idea—in my view—is true anthropic shadow. And for policy it means that I should be even more cautious about risks which look benign.
Also, survival in QI/AS is “timeless”. As in, rather than surviving a Russian Roulette round, you’ll junt find yourself as having avoided playing to begin with. This is because your 5D measure is greater when you avoid such activities (plus survive for free) than if you don’t and survive. Such activities aren’t begun in a counterfactual vacuum.
One more thing, QI invokes a notion of your consciousness “shifting” to youn surviving timeline, but that’s unnecessary. Since survival is timeless, you’ll simply find yourself in the longer timeline from the get go already. No shifting necessary.
This model explains secularly how I find myself as being a Christian. I’m on my longest (and thickest) counterfactual timeline already from my conception (eternal life stuff). Pretty neat stuff. I shouldn’t be so surprised though since secular reasoning isn’t so far fallen that it concludes as it does!
Thoughts?
There is a possible bad decision theory consequence of this.
Imagine a person suffers a bad event like having a serious accident in which he damages other people. If he assumes ‘5D measure’ or “retrospective selection of thickest timelines” to be valid, he decides to commit suicide.
As a result he-tomorrow will find himself only in timelines where the accident didn’t happen.
This actually happens rather often (e.g. Epstein committed suicide after going to jail)
I mean, wouldn’t that timeline be more like “my consciousness keeps somehow hanging onto a husk of a body out of increasingly unlikely coincidences as the universe approaches heat death”? Quantum Immortality strikes me as a terrible world that I really hope isn’t true.
No. Husks have very little 5d measure. How? The same reason you find yourself as a man and not an ant. More precisely, as stated above previously, 5d measure is timeless. Since husks take a hit in both linear scope (4th-d) and counterfactual scope (5th-d), they occupy an infinitesimally-small to epsilon-large area of your anthropic measure landscape. This affects self-location even from the beginning. Read up on UDASSA for a similar formulation of timelessness in self-location. Though UDASSA is more environmental-affecting than this. Here, you can still find yourself as a nobody who’s fully reliant on civilization to protect him against an unbounded counterfactual scope with 0 producer-participation, only consumer-participation. Also, this implies your ontological substrate is a pereto-ideal of simultaneously very full, and very simple (eg, you won’t find yourself as a sim npc).
This only holds if there is any alternative to husks which is also compatible with the laws of physics as a whole. A whole civilisation that just happens to keep drawing its energy from an ever higher entropy universe seems a lot less likely, for example, because the improbabilities compound for the whole size. Seems to me that the most likely state would always be one that requires the bare minimum.
Anyway I didn’t know about UDASSA so I’ll check that out, seems interesting. I’ll maybe get back at this after having gone through it properly.
Yes that’s what I take would happen too unless I’m misunderstanding something? Because it would seem far more probable for *just* your consciousness to somehow still exist, defying entropy, than for the same thing to happen to an entire civilization (same argument why nearly all Boltzmann brains would be just a bare “brain”).
In a sense, our observations are also well explained by Copenhagen, etc., since all interpretations make the same prediction. The supposed advantage of MWI is simplicity, but SI can’t prove it without shenanigans.
https://www.lesswrong.com/posts/6Lg8RWL9pEvoAeEvr/raising-safety-consciousness-among-agi-researchers?commentId=QtwqxiwnCSksCc566
Related: Against Anthropic Shadow, contains pretty direct counterarguments.
I either am not following, or disagree. You’re just making this anthropic ghost a non-agent, but not unable to experience (dis)utility. If I know that a certain choice leads to death, but I expect that after death I’ll still experience stuff, what I decide to do depends on what I imagine death to be like—is it going to be anthropic Heaven or anthropic Hell? Or just a Purgatory? This tangibly affects expected utility before death and thus my behaviour. It also changes the odds of there being a “me” that experiences a past with a deadly event in it—it will be a powerless me, but still a sentient one. The anthropic shadow approach requires death to be the end of sensation, not just of agency.
Anthropic undeath by definition begins when your sensory experience ends. If you end up in an afterlife, the anthropic undeath doesn’t begin until the real afterlife ends. That’s because anthropic undeath is a theoretical construct I defined, and that’s how I defined it.
So why does this even count as not just simple death? Are you assuming this is kind of a cut off state of existence where you still have self-awareness but only that? That still leaves room for utility from internal states.
I probably should’ve expanded on this more in the post, so let me explain.
“Anthropic shadow”, if it were to exist, seems like it should be a general principle of how agents should reason, separate from how they are “implemented”.
Abstractly, all an agent is is a tree of decisions. It’s basically just game theory. We might borrow the word “death” for the end of the game, but this is just an analogy. For example, a reinforcement learning agent “dies” when the training episode is over, even though its source code and parameters still exist. It is “dead” in the sense that the agent isn’t planning its actions past this horizon. This is when Anthropic shadow would apply if it were abstract.
But the idea of “anthropically undead” shows that the actual point of “death” is arbitrary; we can create a game with identical utility where the agent never “dies”. So if the only thing the agent cares about is utility, the agent should reason as if there was no anthropic shadow. And this further suggests that the anthropic shadow must’ve been flawed in the first place; good reasoning principles should hold up under reflection.
I’m still not convinced. Suppose an agent has to play a game: two rounds of Russian roulette with a revolver with one bullet and N chambers. The agent doesn’t know N, and will get a reward R if they survive both rounds; if they give up after one round, they get a consolation prize C1 < R. If they give up immediately, they get one C0 < C1 < R.
The optimal strategy depends on N, and the agent’s belief about it. The question is, suppose the agent already passed the first round, and survived, does this give them any information about the distribution P(N)? Anthropic shadow says “no” because, conditioned on them having any use for the information, they must also have survived the first round. So I suppose you could reframe the anthropic shadow as a sort of theorem about how much useful information you can extract out of the outcomes of a series of past rounds of a game in which loss limits your future actions (death being one extreme outcome). I need to think about formalizing this.
And it is wrong because the anthropic principle is true: we learned that N ≠ 1.
There is the idea of Anthropic decision theory which is related, but I’m still guessing it still has no shadow.
Fair, but no more. And any additional survival doesn’t teach us anything else. All meaningful information only reaches us once we have no more use for it.
I think the same argument applies against quantum immortality, provided you care about different branches in a way that respects the Born rule (which you should).
Okay, basic sanity-check: I sit in Shrödinger box with a radioactive nucleus with half-life one minute for an hour and come out alive from it. I have two possible explanations:
Quantum mechanics is wrong.
Dead mes are unobservable.
And I am pretty damn sure that quantum mechanics work.
I suppose there are other possible explanations like:
3. Someone messed up the experiment so there was no radiactive nucleus in the box
4. You are mind experiment, simulating exteremely improbable scenario
You don’t observe that. You just die.
OK, technically, there is some tiny chance (i.e. small-weight branch) that you would survive, and conditional on this tiny chance (i.e. conditional on that we are considering your observations only in this small-weight branch), you would have observed it not decaying. But the weight of this branch is no larger than the weight of the branch where it didn’t decay if you did the experiment without the dying part.
When physicists outside the box see you come out, they just observed something that is far greater significance than 5 sigmas. It is almost 9 sigmas, in fact. This is enough to make physicists reject QM (or at least the hypothesis “everything happened as you described and QM is true”). And you can’t agree to disagree once you get outside of the box and meet them. So you’d be a physics crank in this scenario if you tell people the experiment’s result was compatible with QM.
The trick is that, from my perspective, everything is going according to QM every time my death doesn’t depend on it. So I can put someone else inside the box and see them die in 10 minutes.
Right, so this an anthropic angel hypothesis, not anthropic shadow.
I mean, there’s plenty of yous that suddenly hear a hiss, see gas filling in the room, start desperately scratching at the sealed door, regretting their stupidity to be coaxed for some reason in this absurd experiment just for the sake of LW cred, grasp at their throat, and eventually slide to the floor spasming as the poison chokes the life out of them. Plenty of experience there! They just don’t get to tell the story.
(it’s interesting though that in a way things DO feel different if instead of poison the decaying atom triggers a nuclear bomb at point blank which obliterates you before any chances to perceive anything at all. But if that meant a real, physical difference that would imply some freaky things about consciousness and quantum mechanics)
Yes! Very good points. A clever trick with an anthropic ghost. I have a vague feeling that there may be a non-sequitur hidden there somewhere, but can’t exactly point it out.
As I’ve mentioned here updating on “anthropic evidence” is the refusal to update on regular evidence.
Adding here the fact that core assumption of both SSA and SIA that I’m selected from a random sample are often wrong, I think it’s possible to make a general case against anthropic reasoning.
Eh, don’t get too cocky. There are definitely some weird bits of anthropics. See We need a theory of anthropic measure binding for example.
But I do think in cases where you exist before the anthropic weirdness goes down, you can use reflection to eliminate much of the mysteriousness of it (just pick an optimal policy and commit that your future selves will follow it). What’s currently puzzling me is what to do when the anthropic thought experiments start before you even existed.
I don’t think I accept the premise. As with any other situation where either A or B can happen but I don’t have any other relevant information I default to an equiprobable prior.
All the attempts to guess what the “universe cares more about” seem completely unsubstantiated and confused. Likewise with the equivalence of anthropic and compassion binding based on pure vibes and not any evidence that our universe has ethics encoded in its fabric in some way.
I believe my approach doesn’t lead to any paradoxes or bizarre scenarios even in cases when the thought experiment started before my existence. Can you think of a counterexample?
Example1: Timmie gets cookies and I am glad he did even if I am dead and cant see him.
Example2: Timmie sucked into blackhole but I don’t care as I cant see him.