I don’t think that the updateful EDT behaviour in e.g. the calculator example is obviously problematic. Certainly not clearly worse than the alternative of just optimizing relative to the prior (cf. Anthony’s post).
I do think that the buy-and-copy behaviour from your example is bad, but it is bad because of how EDT manages the news, not because of the combination of EDT and anthropic updating per se. A counterfactual theory like FDT or TDT doesn’t manage the news and so doesn’t use the buy-and-copy strategy, AIUI. (Maybe similar cases could be constructed for counterfactual theories, though?)
To me, 1 and 2 suggest that we should consider a counterfactual theory (without going updateless relative to the prior), not just EDT + updateless relative to the prior.
In any case, I’m sympathetic to the ideal reflection principle, i.e., we should optimize our subjective expectation of the ideal agent’s expected utilities. So if you think the ideal agent is updateless relative to the prior, then you should make decisions based on your expectations of your expectations relative to this prior if you could compute it. (This includes the expected value of policies for handling reasoning/logical learning.) Of course it’s very unclear how to form such beliefs, but that doesn’t seem like a problem specific to updatelessness (i.e., it’s also unclear how to form beliefs about an updateful ideal agent).
Are there cases where EDT manages the news for reasons other than anthropic updating? I’m not aware of any, and if not then it this is exactly because of the interaction of EDT and anthropic updating.
To me, “managing the news” is just a description of how EDT works in general (i.e., EDT is definitionally about picking the action that gives us the best news). And I think EDT is problematic for that fundamental reason. I just think that Lukas’ case makes the silliness of news-management particularly vivid. (Other cases which arguably do so are XOR blackmail and this case.)
(I do think that Lukas’ case gets some of its counterintuitiveness from the fact that SIA has us weight worlds in proportion to how many copies of us they contain. But, again, that is just a counterintuitive property of SIA in general, which I think we ought to evaluate independently of how it interacts with decision theory.)
If “managing the news” just means “making a decision in situation X such that you are glad to hear the news that you made that decision in situation X,” then I agree that’s a description of EDT. I think it’s a priori reasonable to manage news about what you decide to do, so I don’t see this as a fundamental reason that EDT is problematic. I usually associate the phrase with various intuitive mistakes that EDT might make, and then I want to discuss concrete cases (like Lukas’) in which it appears an agent did something wrong.
I don’t think that the updateful EDT behaviour in e.g. the calculator example is obviously problematic. Certainly not clearly worse than the alternative of just optimizing relative to the prior (cf. Anthony’s post).
I do think that the buy-and-copy behaviour from your example is bad, but it is bad because of how EDT manages the news, not because of the combination of EDT and anthropic updating per se. A counterfactual theory like FDT or TDT doesn’t manage the news and so doesn’t use the buy-and-copy strategy, AIUI. (Maybe similar cases could be constructed for counterfactual theories, though?)
To me, 1 and 2 suggest that we should consider a counterfactual theory (without going updateless relative to the prior), not just EDT + updateless relative to the prior.
In any case, I’m sympathetic to the ideal reflection principle, i.e., we should optimize our subjective expectation of the ideal agent’s expected utilities. So if you think the ideal agent is updateless relative to the prior, then you should make decisions based on your expectations of your expectations relative to this prior if you could compute it. (This includes the expected value of policies for handling reasoning/logical learning.) Of course it’s very unclear how to form such beliefs, but that doesn’t seem like a problem specific to updatelessness (i.e., it’s also unclear how to form beliefs about an updateful ideal agent).
Are there cases where EDT manages the news for reasons other than anthropic updating? I’m not aware of any, and if not then it this is exactly because of the interaction of EDT and anthropic updating.
To me, “managing the news” is just a description of how EDT works in general (i.e., EDT is definitionally about picking the action that gives us the best news). And I think EDT is problematic for that fundamental reason. I just think that Lukas’ case makes the silliness of news-management particularly vivid. (Other cases which arguably do so are XOR blackmail and this case.)
(I do think that Lukas’ case gets some of its counterintuitiveness from the fact that SIA has us weight worlds in proportion to how many copies of us they contain. But, again, that is just a counterintuitive property of SIA in general, which I think we ought to evaluate independently of how it interacts with decision theory.)
If “managing the news” just means “making a decision in situation X such that you are glad to hear the news that you made that decision in situation X,” then I agree that’s a description of EDT. I think it’s a priori reasonable to manage news about what you decide to do, so I don’t see this as a fundamental reason that EDT is problematic. I usually associate the phrase with various intuitive mistakes that EDT might make, and then I want to discuss concrete cases (like Lukas’) in which it appears an agent did something wrong.