@Steven Byrnes I suspect that free markets are an environment which also does things like incentivizing treachery or races to the bottom. What I would expect is UDT-following agents being more likely to coordinate with each other, since the author didn’t actually propose an alternative decision theory. I suspect that the UDT should’ve been derived from a version of superrationality, but I don’t understand how one embeds big neural nets or agents involving randomness into predictors.
I’m not sure I buy this post’s assertion that UDT violates independence. It seems more like it violates “common sense independence”, in the same way it violates “common sense choosing the best option” when it one-boxes on Newcomb’s problem.
An agent locally acting according to a good policy might violate what a CDT agent would call independence, but it still obeys independence when choosing a policy, i.e. it has a numerical utility function, just not over the same stuff as the CDT agent.
One-boxing does not violate “common sense best option”. More people would one box than would two box (although it’s pretty close to 50⁄50). https://www.youtube.com/watch?v=Ol18JoeXlVI in the steelman for one boxing the math is favored by an expected utility approach anyways, as long as you think the genie has >50% probability of predicting you correctly. Plus, if two boxing is common sense, why ain’tcha rich?
I think you’re right that a UDT agent who chooses policies could have preferences over those policies that satisfy independence, with the apparent node-level independence violations being just an artifact of projecting policy-level optimization down to individual nodes.
The core argument is against the standard application of independence as it is done in practice: to individual lotteries and individual choices, with branch-by-branch evaluation, which is how it’s used in the economics literature and in most discussions of rational agency. Both UDT and EE move away from this level.
Or, in other words: for the things over which people actually attempt to maximize expected utility or model it, expected utility maximization is a wrong strategy, because independence doesn’t hold for these things.
Also, for EE specifically, the ergodic mapping makes the effective utility function dependent on the dynamics of the process, which means that choices across different dynamic environments (where the dynamics themselves are part of what you’re choosing) may violate independence even at the trajectory or policy level. Whether this is a real violation or can be further absorbed by moving to yet another meta-level is, I think, an open question.
Overall, it looks like there always may be some “meta” level on which independence holds, but is it useful/meaningfull in practice? You may be able to rescue independence by climbing to higher and higher levels of abstraction, but at the level where EUT is really deployed, where economists fit CRRA parameters to observed choices, where financial advisors recommend portfolio allocations, where rationalists tell each other to shut up and multiply, independence fails and EU maximization gives the wrong answers.
@Steven Byrnes I suspect that free markets are an environment which also does things like incentivizing treachery or races to the bottom. What I would expect is UDT-following agents being more likely to coordinate with each other, since the author didn’t actually propose an alternative decision theory. I suspect that the UDT should’ve been derived from a version of superrationality, but I don’t understand how one embeds big neural nets or agents involving randomness into predictors.
I’m not sure I buy this post’s assertion that UDT violates independence. It seems more like it violates “common sense independence”, in the same way it violates “common sense choosing the best option” when it one-boxes on Newcomb’s problem.
An agent locally acting according to a good policy might violate what a CDT agent would call independence, but it still obeys independence when choosing a policy, i.e. it has a numerical utility function, just not over the same stuff as the CDT agent.
One-boxing does not violate “common sense best option”. More people would one box than would two box (although it’s pretty close to 50⁄50). https://www.youtube.com/watch?v=Ol18JoeXlVI in the steelman for one boxing the math is favored by an expected utility approach anyways, as long as you think the genie has >50% probability of predicting you correctly. Plus, if two boxing is common sense, why ain’tcha rich?