You are talking about it as though it is a property of the puzzle, when it seems likely to be an interaction between the person and puzzle
Chris_Leong
Karma: 3,252
Interesting post, although I wish “reality-masking” puzzles had been defined better. Most of this post is around disorientation pattern or disabling parts of the epistemic immune system more than anything directly masking reality.
Also related: Pseudo-rationality
“That said, I don’t think we are really explaining or de-confusing anything if we appeal to backwards causation to understand Newcomb’s Problem or argue for a particular solution to it.”—How come?
“Relativity does not make the arrow of time relative to observer”—I didn’t say that. I said there was no unified notion of the present
Is backwards causation necessarily absurd?
Maybe you can’t dream the actual process of factoring a large number, but you can dream of having just finished completing such a factoring with the result having come out correct
I liked the diagrams as I think they’ll be clarifying to most people. However, in response to:
I think that many proponents of FDT fail to make this point: FDT’s advantage is that it shifts the question to what type of agent you want to be—not misleading questions of what types of “choices” you want to make
FDT involves choosing in observation-action mapping which is effectively the same as choosing an algorithm if the reward doesn’t ever depend on why you make a particular decision and the mapping space is finite
One problem with trying to model it as what algorithm to run is that you are running code to select the algorithm so if the actual algorithm ever matters as opposed to just the observation-action map, you’d need to take the selection algorithm into account.
The point about simulations was merely to show that the idea of a universe with the majority of consciousness being Boltzmann Brains isn’t absurd
I’m sure I’ll link back to this post soon. But this post is motivated by a few things such as:
a) Disagreements over consciousness—if non-materialist qualia existed, then we wouldn’t be able to know about them empirically, but the universe doesn’t have to play nice and make all phenomenon accessible to our scientific instruments, so we should have more uncertainty about this than people generally possess
b) The theories that can explain everything post—as nice as it’d be to just be able to evaluate theories empirically, there’s no reason why we can’t have a theory that is important for determining expectations, which isn’t cleanly falsifiable
The Universe Doesn’t Have to Play Nice
I thought it might be useful to give an example of when normalisation of deviance is functional. Let’s suppose that a hospital has to treat patients, but because of short-staffing there would be no way of filling out all of the paperwork properly whilst treating all the patients, so the doctors don’t fill out all of the fields.
It’s also important to mention the possibility of scapegoating—perhaps the deviance is justified and practically everyone is working in that manner, but if something goes wrong you may be blamed anyway. So it’s very important to take this small chance of an extremely harsh punishment into account.
Interesting idea. What is the use of organising beliefs without updating them?
My argument is that we ARE thinking ahead about counterfactual mugging right now, in considering the question
When we think about counterfactual muggings, we naturally imagine the possibility of facing a counterfactual mugging in the future. I don’t dispute the value of pre-committing either to take a specific action or to acting updatelessly. However, instead of imagining a future mugging, we could also imagine a present mugging where we didn’t have time to make any pre-commitments. I don’t think it is immediately obvious that we should think updatelessly, instead I believe that it requires further justification.
The role of thinking about decision theory now is to help guide the actions of my future self
This is effectively an attempt at proof-by-definition
I think the average person is going to be thinking about things like duty, honor, and consistency which can serve some of the purpose of updatelessness. But sure, updateful reasoning is a natural kind of starting point, particularly coming from a background of modern economics or bayesian decision theory
If someone’s default is already updateless reasoning, then there’s no need for us to talk them into it. It’s only people with an updateful default that we need to convince (until recently I had an updateful default).
And when we think about problems like counterfactual mugging, the description of the problem requires that there’s both the possibility of heads and tails
It requires a counterfactual possibility, not an actual possibility. And a counterfactual possibility isn’t actual, it’s counter to the factual. So it’s not clear this has any relevance.
It looks like to me that you’re tripping yourself up with verbal arguments that aren’t at all obviously true. The reason why I believe that the Counterfactual Prisoner’s Dilemma is important is because it is a mathematical result that doesn’t require much in the way of assumptions. Sure, it still has to be interpreted, but it seems hard to find an interpretations that avoids the conclusion that the updateful perspective doesn’t quite succeed on its own terms.
Have you considered cross-posting this to the EA forum?
You feel that I’m begging the question. I guess I take only thinking about this counterfactual as the default position, as where an average person is likely to be starting from. And I was trying to see if I could find an argument strong enough to displace this. So I’ll freely admit I haven’t provided a first-principles argument for focusing just on this counterfactual.
OK, but I don’t see how that addresses my argument.
Your argument is that we need to look at iterated situations to understand learning. Sure, but that doesn’t mean that we have to interpret every problem in iterated form. If we need to understand learning better, we can look at a few iterated problems beforehand, rather than turning this one into an iterated problems.
The average includes worlds that you know you are not in. So this doesn’t help us justify taking these counterfactuals into account,
Let me explain more clearly why this is a circular argument:
a) You want to show that we should take counterfactuals into account when making decisions
b) You argue that this way of making decisions does better on average
c) The average includes the very counterfactuals whose value is in question. So b depends on a already being proven ⇒ circular argument
Saying anything is possible is a prediction, but a trivial prediction. Nonetheless, it changes expectations if before only A seemed possible.
Theories That Can Explain Everything
The argument is that we have to understand learning in the first place to be able to make these arguments, and iterated situations are the easiest setting to do that in
Iterated situations are indeed useful for understanding learning. But I’m trying to abstract out over the learning insofar as I can. I care that you get the information required for the problem, but not so much how you get it.
Especially when we can see that the latter way of reasoning does better on average?
The average includes worlds that you know you are not in. So this doesn’t help us justify taking these counterfactuals into account, indeed for us to care about the average we need to already have an independent reason to care about these counterfactuals.
I kind of feel like you’re just repeatedly denying this line of reasoning. Yes, the situation in front of you is that you’re in the risk-hand world rather than the risk-life world. But this is just question-begging with respect to updateful reasoning.
I’m not saying you should reason in this way. You should reason updatelessly. But in order to get to the point of finding the Counterfactual Prisoner’s Dilemma, while I consider a satisfactory justification, I had rigorously question every other solution until I found one which could withstand the questioning. This seems like a better solution as it is less dependent on tricky to evaluate philosophical claims.
Ah, that’s kind of the first reply from you that’s surprised me in a bit
Well, thinking about a decision after you make it won’t do you much good. So you’re pretty always thinking about decisions before you make them. But timelessness involves thinking about decision before you end up facing them.
If an agent is really in a pure one-shot case, that agent can do anything at all
You can learn about a situation other than by facing that exact situation yourself. For example, you may observe other agents facing that situation or receive testimony from an agent that has proven itself trustworthy. You don’t even seem to disagree with me here as you wrote: “you can learn enough about the universe to be confident you’re now in a counterfactual mugging without ever having faced one before”
“This goes along with the idea that it’s unreasonable to consider agents as if they emerge spontaneously from a vacuum, face a single decision problem, and then disappear”—I agree with this. I asked this question because I didn’t have a good model of how to conceptualise decision theory problems, although I think I have a clearer idea now that we’ve got the Counterfactual Prisoner’s Dilemma.
One way of appealing to human moral intuition
Doesn’t work on counter-factually selfish agents
Decision theory should be reflectively endorsed decision theory. That’s what decision theory basically is: thinking we do ahead of time which is supposed to help us make decisions
Thinking about decisions before you make them != thinking about decisions timelessly
There appears to be something of a Sensemaking community developing on the internet, which could roughly be described as a spirituality-inspired attempt at epistemology. This includes Rebel Wisdom, Future Thinkers, Emerge and maybe you could even count post-rationality. While there are undoubtedly lots of critiques that could be made of their epistemics, I’d suggest watching this space as I think some interesting ideas will emerge out of it.