One Doubt About Timeless Decision Theories
Timeless Decisions Theories (including variants like FDT, UDT, ADT, ect.) provide a rather elegant method of solving a broader class of problems than CDT. While CDT requires the outcomes of decisions to be independent of the individual making the decision (in such a way that causal surgery on a single node is valid), timeless decisions theories can handle any problem where the outcome is a function of the choice selected (even if this occurs indirectly as a result of a prediction).
(Epistemic Status: Thoughts for further investigation)
This is an excellent reason to investigate these decision theories, yet we need to make sure that we don’t get blinded by insight. Before we immediately jump to conclusions by taking this improvement, it is worthwhile considering what we give up. Perhaps there are other classes that we might which to optimise over which we can no longer optimise over once we have included this whole class?
After all, there is a sense in which there is no free lunch. As discussed in the TDT paper, for any algorithm, we could create a situation where there is an agent that specifically punished that algorithm. The usual response is that these situations are unfair, but a) the universe is often unfair b) there are plausible situations where the algorithm chosen influences the outcome is slightly less unfair ways.
Expanding on b), there are times when you want to be predictable to simulators. Indeed, I can even imagine agents that wish to eliminate agents that they can’t predict. Further, rather than facing a perfect predictor, it seems like it’ll be at least a few orders of magnitude more likely that you’ll face an imperfect predictor. Modelling these as X% perfect predictor, 100-X% random predictor will usually be implausible as predictors won’t have a uniform success rate over all algorithms. These situations are slightly more plausible for scenarios involving AI, but even if you perfectly know an agent’s source code, you are unlikely to know its exact observational state due to random noise.
It therefore seems that the “best” decision theory algorithm might be dominated by factors other than optimal performance on the narrow class of problems TDT operates on. It may very well be the case that TDT is ultimately taking the right approach, but even if this is the case, I thought it was worthwhile sketching out these concerns so that they can be addressed.