Decision theories are not about what kind of agent you want to be. There is no one on god’s green earth who disputes that the types of agents who one box are better off on average. Decision theory is about providing a theory of what is rational.
Taboo the word “rational”—what real-world effects are you are actually trying to accomplish? Because if FDT makes me better off on average and CDT allows me to write “look how rational I am” in my diary, then there’s a clear winner here.
You might like my recent post which also argues that FDT (or at least the sequential version of EDT, I don’t feel that FDT is well specified enough for me to talk about it) shouldn’t be considered a decision theory.
I think FDT is more practical as a decision theory for humans than you give it credit for. It’s true there are a lot of weird and uncompelling examples floating around, but how about this very practical one: the power of habits. There’s common and (I think) valuable wisdom that when you’re deciding whether to e.g. exercise today or not (assuming that’s something you don’t want to do in the moment but believe has long-term benefits), you can’t just consider the direct costs and benefits of today’s exercise session. Instead, you also need to consider that if you don’t do it today, realistically you aren’t going to do it tomorrow either because you are a creature of habit. In other words, the correct way to think about habit-driven behavior (which is a lot of human behavior) is FDT: you don’t ask “do I want to skip my exercise today” (to which the answer might be yes), instead you ask “do I want to be the kind of person who skips their exercise today” (to which the answer is no, because that kind of person also skips it every day).
I agree that this is an important consideration for humans, though I feel perhaps FDT is overkill for the example you mentioned (insofar as I understand what FDT means).
I haven’t dived into the formalism (if there is one?), but I’m roughly using FDT to mean “make your decision with the understanding that you are deciding at the policy level, so this affects not just the current decision but all other decisions that fall under this policy that will be made by you or anything sufficiently like you, as well as all decisions made by anyone else who can discern (and cares about) your policy”. Which sounds complicated, but I think often really isn’t? e.g. in the habits example, it makes everything very simple (do the habit today because otherwise you won’t do it tomorrow either). CDT can get to the same result there—unlike for some weirder examples, there is a causal though not well-understood pathway between your decision today and the prospective cost you will face when making the decision tomorrow, so you could hack that into your calculations. But if by ‘overkill’ you mean using something more complicated than necessary, then I’d say that it’s CDT that would be overkill, not FDT, since FDT can get to the result more simply. And if by ‘overkill’ you mean using something more powerful/awesome/etc than necessary, then overkill is the best kind of kill :)
Making a decision at the policy level may be useful for forming habits, but I don’t think that considering others with a similar policy or those who can discern my policy is useful in this example. Those later two are the ones I associate more closely with FDT, and such considerations seem to be very difficult to carry out in practice and perhaps sensitive to assumptions about the world. Honestly I don’t see the motivation for worrying about the decisions of “other agents sufficiently similar to oneself” at all. It doesn’t seem useful to me, right now, making decisions or adopting a policy, and it doesn’t seem useful to build into an A.I. either except in very specific cases where many copies of the A.I. are likely to interact. The heuristic arguments that this is important aren’t convincing to me because they are sufficiently elaborate that it seems other assumptions about the way the environment accesses/includes one’s policy could easily lead to completely different conclusions.
The underlying flaw I see in many pro-FDT style arguments is that they tend to uncritically accept that if adopting FDT (or policy X) is better in one example that adopting policy Y, policy X must be better than policy Y, or at least neither one is the best policy. But I strongly suspect there are no free lunch conditions here—even in the purely Decision Theoretic context of AIXI there are serious issues with the choice of prior being subjective, so I’d expect it to be even worse if one allows the environment read/write access to the whole policy. I haven’t seen any convincing argument that there is some kind of “master policy.” I suppose if you pin down a mixture rigorously defining how the environment is able to read/write the policy then there would be some Bayes optimal policy, but I’m willing to bet it would be deviously hard to find or even approximate.
Taboo the word “rational”—what real-world effects are you are actually trying to accomplish? Because if FDT makes me better off on average and CDT allows me to write “look how rational I am” in my diary, then there’s a clear winner here.
You might like my recent post which also argues that FDT (or at least the sequential version of EDT, I don’t feel that FDT is well specified enough for me to talk about it) shouldn’t be considered a decision theory.
https://www.lesswrong.com/posts/MwetLcBPvshg9ePZB/decision-theory-is-not-policy-theory-is-not-agent-theory?commentId=rZ5tpB5nHewBzAszR
I think FDT is more practical as a decision theory for humans than you give it credit for. It’s true there are a lot of weird and uncompelling examples floating around, but how about this very practical one: the power of habits. There’s common and (I think) valuable wisdom that when you’re deciding whether to e.g. exercise today or not (assuming that’s something you don’t want to do in the moment but believe has long-term benefits), you can’t just consider the direct costs and benefits of today’s exercise session. Instead, you also need to consider that if you don’t do it today, realistically you aren’t going to do it tomorrow either because you are a creature of habit. In other words, the correct way to think about habit-driven behavior (which is a lot of human behavior) is FDT: you don’t ask “do I want to skip my exercise today” (to which the answer might be yes), instead you ask “do I want to be the kind of person who skips their exercise today” (to which the answer is no, because that kind of person also skips it every day).
I agree that this is an important consideration for humans, though I feel perhaps FDT is overkill for the example you mentioned (insofar as I understand what FDT means).
I haven’t dived into the formalism (if there is one?), but I’m roughly using FDT to mean “make your decision with the understanding that you are deciding at the policy level, so this affects not just the current decision but all other decisions that fall under this policy that will be made by you or anything sufficiently like you, as well as all decisions made by anyone else who can discern (and cares about) your policy”. Which sounds complicated, but I think often really isn’t? e.g. in the habits example, it makes everything very simple (do the habit today because otherwise you won’t do it tomorrow either). CDT can get to the same result there—unlike for some weirder examples, there is a causal though not well-understood pathway between your decision today and the prospective cost you will face when making the decision tomorrow, so you could hack that into your calculations. But if by ‘overkill’ you mean using something more complicated than necessary, then I’d say that it’s CDT that would be overkill, not FDT, since FDT can get to the result more simply. And if by ‘overkill’ you mean using something more powerful/awesome/etc than necessary, then overkill is the best kind of kill :)
Making a decision at the policy level may be useful for forming habits, but I don’t think that considering others with a similar policy or those who can discern my policy is useful in this example. Those later two are the ones I associate more closely with FDT, and such considerations seem to be very difficult to carry out in practice and perhaps sensitive to assumptions about the world.
Honestly I don’t see the motivation for worrying about the decisions of “other agents sufficiently similar to oneself” at all. It doesn’t seem useful to me, right now, making decisions or adopting a policy, and it doesn’t seem useful to build into an A.I. either except in very specific cases where many copies of the A.I. are likely to interact. The heuristic arguments that this is important aren’t convincing to me because they are sufficiently elaborate that it seems other assumptions about the way the environment accesses/includes one’s policy could easily lead to completely different conclusions.
The underlying flaw I see in many pro-FDT style arguments is that they tend to uncritically accept that if adopting FDT (or policy X) is better in one example that adopting policy Y, policy X must be better than policy Y, or at least neither one is the best policy. But I strongly suspect there are no free lunch conditions here—even in the purely Decision Theoretic context of AIXI there are serious issues with the choice of prior being subjective, so I’d expect it to be even worse if one allows the environment read/write access to the whole policy. I haven’t seen any convincing argument that there is some kind of “master policy.” I suppose if you pin down a mixture rigorously defining how the environment is able to read/write the policy then there would be some Bayes optimal policy, but I’m willing to bet it would be deviously hard to find or even approximate.