I don’t think the CDTers claim to Newcomb being unfair a valid one. The parallel claim is Omega saying “OK, you follow CDT, I am going to kill you.” which is a separate problem from Newcomblike problems where you’re being simulated.
And I don’t think your in-principle reasons actually hold up in practice. FDT’s output can be understood as “Output X such that, given the statement ‘FDT outputs X’, value is maximized” which in classical logic does break by the principle of explosion. This is, again, why the entire logical uncertainty agenda exists.
(My guess is that the answer is something like a modified FDT(N) outputting “Output X such that, if you feed ‘FDT outputs X’ into a logical inductor and run it for N steps, expected value is maximized” which provably[1] converge as to something we might call FDT)
A claim I would agree with is something like: “FDT as currently written down is not well-defined, but appears to be intuitively navigable and may be modifiable into something like F’DT which basically gets at the same flavour, using the language of logical uncertainty”? I get the impression your view is much stronger than that, along the lines of “FDT is so completely poorly-defined it’s pointless to ever look in that direction again, salt the earth.” based on the general vibe of this article, but that’s not justifiable from the evidence that we have!
“FDT outputs X” is importantly underspecified. If you just change what it says in your exact situation, then you have no account of how it changes the other algorithms like the simulator.
The bigger problem though isn’t the counterlogicals but analyzing how in the counterlogical world other algorithms would be different.
Newcomb’s Revenge actually is a fair problem which is not quite as much of a problem as “If you use FDT, I will kill you”. But if you accept Newcomb’s Revenge as symmetric to Newcomb’s problem then you have to also deny that any decision theory can be better than any other. For any decision problem where we simulate an agent in situation X, decide a payoff matrix based on X for a different instance which is in a different situation Y, then you can come up with a flip. Newcomb’s problem X is important because it depends on your behaviour in the situation that you are actually in! Which breaks the asymmetry.
And ok if you actually want to think about this in depth it gets super cursed super quickly, because any FDT agent will have to have a prior over being in a Newcomblike problem vs a Revengelike problem and if it turns out that in the real world, Newcomb problems are rare and Revenge problems are common, then FDT will update accordingly and start one-boxing, but actually the concept of high-fidelity simulation at some level itself breaks down because the simulator can induce any behaviour in your simulated self by giving you arbitrary inputs.
Newcomb’s problem also works if you treat the simluated agents as agents themselves who have an accurate prior over their own situation (the maths takes a while to shake out but it definitely does). While Revenge only works in this way if the simulated agents think they’re being simulated by a Newcomblike Omega (otherwise, if they know they’re 50:50 real or being simulated by a Revengelike Omega, they will two box).
I don’t understand your claim on how exactly the counterlogicals break. Normally the obstacle is Lob’s theorem, which IIUC Logical Induction fixes, but if you have a stronger argument then I would like to hear it.
I don’t think the CDTers claim to Newcomb being unfair a valid one. The parallel claim is Omega saying “OK, you follow CDT, I am going to kill you.” which is a separate problem from Newcomblike problems where you’re being simulated.
And I don’t think your in-principle reasons actually hold up in practice. FDT’s output can be understood as “Output X such that, given the statement ‘FDT outputs X’, value is maximized” which in classical logic does break by the principle of explosion. This is, again, why the entire logical uncertainty agenda exists.
(My guess is that the answer is something like a modified FDT(N) outputting “Output X such that, if you feed ‘FDT outputs X’ into a logical inductor and run it for N steps, expected value is maximized” which provably[1] converge as
A claim I would agree with is something like: “FDT as currently written down is not well-defined, but appears to be intuitively navigable and may be modifiable into something like F’DT which basically gets at the same flavour, using the language of logical uncertainty”? I get the impression your view is much stronger than that, along the lines of “FDT is so completely poorly-defined it’s pointless to ever look in that direction again, salt the earth.” based on the general vibe of this article, but that’s not justifiable from the evidence that we have!
I don’t have a proof but this is the general kind of thing that logical inductors prove.
Why is Newcomb’s any less fair than Newcomb’s revenge? https://www.umsu.de/blog/2022/772
“FDT outputs X” is importantly underspecified. If you just change what it says in your exact situation, then you have no account of how it changes the other algorithms like the simulator.
The bigger problem though isn’t the counterlogicals but analyzing how in the counterlogical world other algorithms would be different.
Newcomb’s Revenge actually is a fair problem which is not quite as much of a problem as “If you use FDT, I will kill you”. But if you accept Newcomb’s Revenge as symmetric to Newcomb’s problem then you have to also deny that any decision theory can be better than any other. For any decision problem where we simulate an agent in situation X, decide a payoff matrix based on X for a different instance which is in a different situation Y, then you can come up with a flip. Newcomb’s problem X is important because it depends on your behaviour in the situation that you are actually in! Which breaks the asymmetry.
And ok if you actually want to think about this in depth it gets super cursed super quickly, because any FDT agent will have to have a prior over being in a Newcomblike problem vs a Revengelike problem and if it turns out that in the real world, Newcomb problems are rare and Revenge problems are common, then FDT will update accordingly and start one-boxing, but actually the concept of high-fidelity simulation at some level itself breaks down because the simulator can induce any behaviour in your simulated self by giving you arbitrary inputs.
Newcomb’s problem also works if you treat the simluated agents as agents themselves who have an accurate prior over their own situation (the maths takes a while to shake out but it definitely does). While Revenge only works in this way if the simulated agents think they’re being simulated by a Newcomblike Omega (otherwise, if they know they’re 50:50 real or being simulated by a Revengelike Omega, they will two box).
I don’t understand your claim on how exactly the counterlogicals break. Normally the obstacle is Lob’s theorem, which IIUC Logical Induction fixes, but if you have a stronger argument then I would like to hear it.