Note: decision theories are theories about rationality. They tell you what decisions are wise and sensible to make. They are not theories about the desirable dispositions to have or about how you should program an AI. There are interesting questions about those sorts of things, but they aren’t what decision theory is about.
Tendentious. Certainly you are welcome to define your way to victory. At that point, why bother writing the rest of the post?
There’s a response to this that I’ve heard from a lot of functional decision theorists. Here’s the idea: you don’t really know if you are the algorithm being simulated in Newcomb’s problem or the actual person. For all you know, you might be the simulation, in which case you outputting “one-box” leads to more utility. I find this response very bizarre:
I know I’m not a simulated algorithm. The simulated algorithm isn’t conscious (we can stipulate). I am.
This disagreement probably isn’t tractable, but, uh, false. I assume it’s not a load-bearing argument for you, though.
Note that the real world contains many Newcomb-like problems. People do in fact go around making decisions that depend on their beliefs about other people’s decision algorithms. So one question I have is whether you think CDT is good either as a descriptive or normative decision theory. It seems quite obvious that it’s a bad descriptive theory of human behavior: people model other people’s decision algorithms, and people implement decision algorithms in that context. It also really leaves quite a lot to be desired as a normative theory: there are wildly fewer positive-sum trades that can be made with agents that actually implement (or approximate) CDT, compared to FDT.
Note that the real world contains many Newcomb-like problems. People do in fact go around making decisions that depend on their beliefs about other people’s decision algorithms.
I disagree, see here for why. (I think “decisions that depend on their beliefs about other people’s decision algorithms” is too weak to get you a Newcomblike structure.)
>Certainly you are welcome to define your way to victory
I mean, you‘re free to use the term “decision theory“ to refer to whatever you want. But the claim that the theory about what decisions to make is the same thing as the theory about how to program an AI or what are the most desirable dispositions to have is what actually requires an argument.
Yep. Presumably BB’s implicit claim here is something like: “People make claims that we should do such and such thing because of FDT, but those claims don’t follow from ‘we should program an AI to follow FDT’.”
For example, here’s Richard Ngo saying we should cooperate with the values of civilizations outside our lightcone because of FDT.
Tendentious. Certainly you are welcome to define your way to victory. At that point, why bother writing the rest of the post?
This disagreement probably isn’t tractable, but, uh, false. I assume it’s not a load-bearing argument for you, though.
Note that the real world contains many Newcomb-like problems. People do in fact go around making decisions that depend on their beliefs about other people’s decision algorithms. So one question I have is whether you think CDT is good either as a descriptive or normative decision theory. It seems quite obvious that it’s a bad descriptive theory of human behavior: people model other people’s decision algorithms, and people implement decision algorithms in that context. It also really leaves quite a lot to be desired as a normative theory: there are wildly fewer positive-sum trades that can be made with agents that actually implement (or approximate) CDT, compared to FDT.
I disagree, see here for why. (I think “decisions that depend on their beliefs about other people’s decision algorithms” is too weak to get you a Newcomblike structure.)
>Certainly you are welcome to define your way to victory
I mean, you‘re free to use the term “decision theory“ to refer to whatever you want. But the claim that the theory about what decisions to make is the same thing as the theory about how to program an AI or what are the most desirable dispositions to have is what actually requires an argument.
Yep. Presumably BB’s implicit claim here is something like: “People make claims that we should do such and such thing because of FDT, but those claims don’t follow from ‘we should program an AI to follow FDT’.”
For example, here’s Richard Ngo saying we should cooperate with the values of civilizations outside our lightcone because of FDT.