It’s ironic that Wei originally invented UDT in an attempt to figure out anthropic reasoning, but now we can see that it mostly pushes the problem under the rug.
I did point out
In general, Updateless Decision Theory converts anthropic reasoning problems into ethical problems.
Thanks for that link, I’d read it but then forgot about it for some reason. it definitely belongs here :-)
The problem is, I just can’t accept the idea that anthropic/ethical problems are “indeterminate” and that we should rely on our “intuitions”. It seems that some anthropic problems must have well-defined solutions. If you take part in many successive experiments with observer-splitting, you will observe some limiting frequencies, but we have no clue what these frequencies will be. Do they depend on the thickness of the silicon wafers used to build our computers? Or do they depend on the algorithms running within our brains?
If you take part in many successive experiments with observer-splitting, you will observe some limiting frequencies, but we have no clue what these frequencies will be. Do they depend on the thickness of the silicon wafers used to build our computers? Or do they depend on the algorithms running within our brains?
I’m confused by this comment. From the outside, I can tell you which observers will have remembered which limiting frequencies in their histories. What’s an example of an experiment that would tell you something besides this?
The problem is, I just can’t accept the idea that anthropic/ethical problems are “indeterminate” and that we should rely on our “intuitions”.
I’m not saying that we should rely on our intuitions in this matter (which we can’t anyway since they are contradictory, or at least mine are, as I pointed out in the above linked post), but they do perhaps offer some starting points for thinking about the problem.
It seems that some anthropic problems must have well-defined solutions.
Well, that’s your intuition. :)
I’m not sure what you mean by “limiting frequencies” here, but here’s a thought experiment attempting to show that they may not be very relevant.
Suppose we take a subject and label him A0. In each round, we take the copy that was previously labeled A0, make two more copies of him, which we’ll label B0 and C0, then ask the three of them to guess A or Not-A (without letting them observe their labels). After that, we tell them their labels, and make one more copy of each of the three, to be labeled A1, B1, C1. If A0 guessed correctly, we give A1 a nice experience, for example eating a cake. Do the same for B1 if B0 guessed correctly, and similarly for C1/C0. (The reason for making extra copies after the guess is so A0 doesn’t get sick of eating cakes.) At the end of the round, delete B0, C0, A1, B1, C1.
Again, I’m not sure how you define “observed limiting frequency”, but it seems that F(A)=1. However you’re better off always betting on Not-A, since that results in twice as many of your copies eating cake.
Killing observers to change the limiting frequency is cheating :-)
Consider a simpler example: I flip a coin and show you the result. Then if it came up heads, I kill you, otherwise I repeat the experiment. I think you’d be correct (in some yet-undiscovered sense) to have a “subjective anticipation” of 50% heads and 50% tails before the throw, but counting the surviving branches after many trials gives a “limiting frequency” of mostly tails. This doesn’t look to me like a fair interpretation of “limiting frequency”, because it arbitrarily throws away all observations made by those of you who ended up dying. If I could resurrect them and include them in the poll, I’d get a different result.
If you resurrect them and include them in the poll, and assuming you average their observed frequencies, don’t you still get F(A)=1? As I said, I’m not sure what you mean by “limiting frequency”, but I don’t see how you can get something other than F(A)=1 in my example.
If I count all observer-moments that get told their labels, the fraction of observer-moments that get told A is 1⁄3. If each observer has a fixed amount of “reality fluid” that gets split in equal parts when copies are made and disappears when copies die, the total fraction of “reality fluid” in observer-moments that get told A is also 1⁄3, but by a different calculation. Maybe both these methods of counting are wrong, but the answer 1 is still far from certain.
If you take part in many successive experiments with observer-splitting, you will observe some limiting frequencies
No. Different instances of the agent that processed different sequences of observations will observe different limiting frequencies. Some will even observe secret messages from Lords of the Matrix encoded in Morse.
And once again, I cannot understand whether your comment expresses some deep idea or you’re missing some obvious point. Flipping a coin is also observer-splitting (in a sense), and we do observe something pretty damn similar to “limiting frequencies” instead of the utter chaos that you seem to predict. Yeah it’s true that different instances of you will see different sequences of heads and tails. But it’s not the whole truth.
An observer-splitting setup that did not give rise to subjective limiting frequencies would be something quite new under the sun. I have no idea if it’s even possible, yet you sound so certain...
Observing certain frequencies is probable and anticipated. Normative probability is given by prior, and normative anticipation is calculated from the prior, possibly along the lines of what I described here. The resulting probabilities explain the observations we’re likely to see in the trivial sense of being the theories which hold these observations probable. It is an example of a circular justification, an answer to a heuristic “why” question that short-circuits in this particular case where you ask not about a phenomenon with non-trivial definition, but the whole of your experience.
I think you’ll agree that there are other versions of yourself that observed all the chaos allowed by the laws of physics. What is the sense in which you’re special, compared to them, what is the regularity that wants explaining? You’re much, much more probable and hence more relevant for decision-making heuristics. You remember expecting normality and not chaos, and remember having that expectation met. That expectation was formed under the same considerations that define the corresponding past experiences as probable, even if that probability is logically non-transparent in mere psychological expectation, and becomes apparent mostly in retrospect and on reflection. But there are other instances of yourself out there, unimportant in their measure, that have had some strange experiences not explained by their normative anticipation.
Your explanation still doesn’t work for me, I’m afraid.
Do you mean “prior” as part of my mind’s software, or “prior” as something ethereal and universal? If the former, how can my tiny brain have beliefs about all elementary particles in the universe, why did evolution build such a thing if robots using ordinary software can survive just fine, and where should I tweak my mind if I want to win the lottery? If the latter, what makes you believe that there is such a prior, and isn’t this “measure” just reality-fluid by another name, which is a well-known antipattern? Or is there some third alternative that I missed?
The disparity between the level of detail in reality/prior, and imprecision and mutability of psychological anticipation was an open problem for my attack at the problem which I made in autumn (and discussed previously here).
This problem is solved by identifying prior (notion of reality) not with explicit data given by psychological anticipation, but with normative anticipation. That is, reality is explained as that which we should expect, where the shouldness of expectation is not a line from Litany of Tarski, suggesting how one ought to keep an accurate map of reality, but literally explanation of what reality is.
The multi-level conceptual models that humans build are models of uncertainty, expressing logical uncertainty about the conclusions that should be drawn from past observations. There is only one level of reality in the same sense there is only one mathematical structure behind the many axiomatic definitions that specify it. Reality is, in a sense, what a Bayesian superintelligence would conclude given the knowledge and observations that humans have. But as with morality, we don’t have that definition explicitly anywhere, and can only learn more and more detail, and as with morality, the notion is normative, so you can’t solve any problems by changing the question (“where should I tweak my mind if I want to win the lottery”).
A big question remaining is how do we learn from observations, in what sense do the observations confer knowledge, what is the difference between such knowledge and other kinds of knowledge. This requires facing some problems that UDT avoided by refusing to treat observations as knowledge.
This problem is solved by identifying prior (notion of reality) not with explicit data given by psychological anticipation, but with normative anticipation. That is, reality is explained as that which we should expect, where the shouldness of expectation is not a line from Litany of Tarski, suggesting how one ought to keep an accurate map of reality, but literally explanation of what reality is.
I don’t understand how this is different from believing in reality-fluid. If it’s the same thing, I cannot accept that. If it’s different, could you explain how?
This is an explanation of reality in terms of decision-theoretic heuristics we carry in our heads, as a notion similar to morality and platonic truth. This is of course a mere conceptual step, it doesn’t hand you much explanatory power, but I hope it can make reality a bit less mysterious. Like saying that Boeing 747 is made out of atoms, but not pointing out any specific details about its systems.
I don’t understand what exactly you refer to by reality-fluid, in what sense you see an analogy, and what problem that points out. The errors and confusions of evaluating one’s anticipation in practice have little bearing on how anticipation should be evaluated.
This problem is solved by identifying prior (notion of reality) not with explicit data given by psychological anticipation, but with normative anticipation. That is, reality is explained as that which we should expect, where the shouldness of expectation is not a line from Litany of Tarski, suggesting how one ought to keep an accurate map of reality, but literally explanation of what reality is.
I don’t understand how this is different from believing in reality-fluid. If it’s the same thing, I cannot accept that. If it’s different, could you explain how?
I did point out
in Torture vs. Dust vs. the Presumptuous Philosopher: Anthropic Reasoning in UDT. So I’m not trying it hide it or anything. :)
Thanks for that link, I’d read it but then forgot about it for some reason. it definitely belongs here :-)
The problem is, I just can’t accept the idea that anthropic/ethical problems are “indeterminate” and that we should rely on our “intuitions”. It seems that some anthropic problems must have well-defined solutions. If you take part in many successive experiments with observer-splitting, you will observe some limiting frequencies, but we have no clue what these frequencies will be. Do they depend on the thickness of the silicon wafers used to build our computers? Or do they depend on the algorithms running within our brains?
I’m confused by this comment. From the outside, I can tell you which observers will have remembered which limiting frequencies in their histories. What’s an example of an experiment that would tell you something besides this?
I’m not saying that we should rely on our intuitions in this matter (which we can’t anyway since they are contradictory, or at least mine are, as I pointed out in the above linked post), but they do perhaps offer some starting points for thinking about the problem.
Well, that’s your intuition. :)
I’m not sure what you mean by “limiting frequencies” here, but here’s a thought experiment attempting to show that they may not be very relevant.
Suppose we take a subject and label him A0. In each round, we take the copy that was previously labeled A0, make two more copies of him, which we’ll label B0 and C0, then ask the three of them to guess A or Not-A (without letting them observe their labels). After that, we tell them their labels, and make one more copy of each of the three, to be labeled A1, B1, C1. If A0 guessed correctly, we give A1 a nice experience, for example eating a cake. Do the same for B1 if B0 guessed correctly, and similarly for C1/C0. (The reason for making extra copies after the guess is so A0 doesn’t get sick of eating cakes.) At the end of the round, delete B0, C0, A1, B1, C1.
Again, I’m not sure how you define “observed limiting frequency”, but it seems that F(A)=1. However you’re better off always betting on Not-A, since that results in twice as many of your copies eating cake.
Killing observers to change the limiting frequency is cheating :-)
Consider a simpler example: I flip a coin and show you the result. Then if it came up heads, I kill you, otherwise I repeat the experiment. I think you’d be correct (in some yet-undiscovered sense) to have a “subjective anticipation” of 50% heads and 50% tails before the throw, but counting the surviving branches after many trials gives a “limiting frequency” of mostly tails. This doesn’t look to me like a fair interpretation of “limiting frequency”, because it arbitrarily throws away all observations made by those of you who ended up dying. If I could resurrect them and include them in the poll, I’d get a different result.
If you resurrect them and include them in the poll, and assuming you average their observed frequencies, don’t you still get F(A)=1? As I said, I’m not sure what you mean by “limiting frequency”, but I don’t see how you can get something other than F(A)=1 in my example.
Averaging observed frequencies sounds weird...
If I count all observer-moments that get told their labels, the fraction of observer-moments that get told A is 1⁄3. If each observer has a fixed amount of “reality fluid” that gets split in equal parts when copies are made and disappears when copies die, the total fraction of “reality fluid” in observer-moments that get told A is also 1⁄3, but by a different calculation. Maybe both these methods of counting are wrong, but the answer 1 is still far from certain.
No. Different instances of the agent that processed different sequences of observations will observe different limiting frequencies. Some will even observe secret messages from Lords of the Matrix encoded in Morse.
And once again, I cannot understand whether your comment expresses some deep idea or you’re missing some obvious point. Flipping a coin is also observer-splitting (in a sense), and we do observe something pretty damn similar to “limiting frequencies” instead of the utter chaos that you seem to predict. Yeah it’s true that different instances of you will see different sequences of heads and tails. But it’s not the whole truth.
An observer-splitting setup that did not give rise to subjective limiting frequencies would be something quite new under the sun. I have no idea if it’s even possible, yet you sound so certain...
Observing certain frequencies is probable and anticipated. Normative probability is given by prior, and normative anticipation is calculated from the prior, possibly along the lines of what I described here. The resulting probabilities explain the observations we’re likely to see in the trivial sense of being the theories which hold these observations probable. It is an example of a circular justification, an answer to a heuristic “why” question that short-circuits in this particular case where you ask not about a phenomenon with non-trivial definition, but the whole of your experience.
I think you’ll agree that there are other versions of yourself that observed all the chaos allowed by the laws of physics. What is the sense in which you’re special, compared to them, what is the regularity that wants explaining? You’re much, much more probable and hence more relevant for decision-making heuristics. You remember expecting normality and not chaos, and remember having that expectation met. That expectation was formed under the same considerations that define the corresponding past experiences as probable, even if that probability is logically non-transparent in mere psychological expectation, and becomes apparent mostly in retrospect and on reflection. But there are other instances of yourself out there, unimportant in their measure, that have had some strange experiences not explained by their normative anticipation.
Your explanation still doesn’t work for me, I’m afraid.
Do you mean “prior” as part of my mind’s software, or “prior” as something ethereal and universal? If the former, how can my tiny brain have beliefs about all elementary particles in the universe, why did evolution build such a thing if robots using ordinary software can survive just fine, and where should I tweak my mind if I want to win the lottery? If the latter, what makes you believe that there is such a prior, and isn’t this “measure” just reality-fluid by another name, which is a well-known antipattern? Or is there some third alternative that I missed?
The disparity between the level of detail in reality/prior, and imprecision and mutability of psychological anticipation was an open problem for my attack at the problem which I made in autumn (and discussed previously here).
This problem is solved by identifying prior (notion of reality) not with explicit data given by psychological anticipation, but with normative anticipation. That is, reality is explained as that which we should expect, where the shouldness of expectation is not a line from Litany of Tarski, suggesting how one ought to keep an accurate map of reality, but literally explanation of what reality is.
The multi-level conceptual models that humans build are models of uncertainty, expressing logical uncertainty about the conclusions that should be drawn from past observations. There is only one level of reality in the same sense there is only one mathematical structure behind the many axiomatic definitions that specify it. Reality is, in a sense, what a Bayesian superintelligence would conclude given the knowledge and observations that humans have. But as with morality, we don’t have that definition explicitly anywhere, and can only learn more and more detail, and as with morality, the notion is normative, so you can’t solve any problems by changing the question (“where should I tweak my mind if I want to win the lottery”).
A big question remaining is how do we learn from observations, in what sense do the observations confer knowledge, what is the difference between such knowledge and other kinds of knowledge. This requires facing some problems that UDT avoided by refusing to treat observations as knowledge.
I don’t understand how this is different from believing in reality-fluid. If it’s the same thing, I cannot accept that. If it’s different, could you explain how?
This is an explanation of reality in terms of decision-theoretic heuristics we carry in our heads, as a notion similar to morality and platonic truth. This is of course a mere conceptual step, it doesn’t hand you much explanatory power, but I hope it can make reality a bit less mysterious. Like saying that Boeing 747 is made out of atoms, but not pointing out any specific details about its systems.
I don’t understand what exactly you refer to by reality-fluid, in what sense you see an analogy, and what problem that points out. The errors and confusions of evaluating one’s anticipation in practice have little bearing on how anticipation should be evaluated.
I don’t understand how this is different from believing in reality-fluid. If it’s the same thing, I cannot accept that. If it’s different, could you explain how?