On the Smoking Lesion, P(Cancer|Smoking) != P(Cancer), by hypothesis- since you don’t know whether or not you have the gene, your decision to smoke does provide you with new evidence about whether or not you do, and therefore about whether or not you are likely to get cancer. (Crucially, it doesn’t cause you to get cancer- all along you either had the gene or didn’t. But EDT works based on correlation, not causation.) For example, one way to rig the probabilities is that half of people smoke, and half of those people get cancer, while nobody who doesn’t smoke gets cancer. (And, again, smoking doesn’t cause cancer.) Then P(Cancer|Smoking)=.5, P(Cancer)=.25, and choosing to smoke is still the right decision.
The one way you can get a correct correlation between your decision and the utility gained is if you refuse to update your belief that you have the gene based on the evidence of your decision. (This is what CDT does.) But this method fails on Newcomb’s Problem. EDT wins on Newcomb’s Problem precisely because it has you update on who you are based on your decision: P(Kind of Person Who Would One-Box|Decision to One-Box) != P(Kind of Person Who Would One-Box). But if you decide not to update your beliefs about your character traits based on your decision, then this is lost.
One way to solve this is to give the agent access to its own code/knowledge about itself, and have it condition on that knowledge. That is essentially what Wei Dai’s Updateless Decision Theory (UDT) does- it doesn’t update its knowledge about its code based on its decision (thus the name). UDT still works on the Prisoner’s Dilemma, because the agent doesn’t know what decision its code will output until it has made the decision, so it still needs to update Omega’s (by assumption, accurate) prediction of its output based on its actual output. (Decision theories, in general, can’t condition on their output- doing so leads to the five-and-ten problem. So UDT doesn’t start out knowing whether it is the kind of agent that one-boxes or two-boxes- though it does start out knowing its own (genetic or computer) code.)
(This comment would probably be significantly improved by having diagrams. I may add some at some point.)
On the Smoking Lesion, P(Cancer|Smoking) != P(Cancer), by hypothesis
Correct: P(Cancer|Smoking) > P(Cancer). When I said P(O|A) = P(O), I was using A to denote “the decision to not smoke, for the purpose of avoiding cancer.” And this is given by the hypothesis of Smoker’s lesion. The whole premise is that once we correct for the presence of the genetic lesion (which causes both love of smoking and cancer) smoking is not independently associated with cancer. This also suggests that once we correct for love of smoking, smoking is not independently associated with cancer. So if you know that your reason for (not) smoking has nothing to do with how much you like smoking, then the knowledge that you (don’t) smoke doesn’t make it seem any more (or less) likely that you’ll get cancer.
Unfortunately, “the decision to not smoke, for the purpose of avoiding cancer” and “the decision not to smoke, for any other reason” are not distinct actions. The actions available are simply “smoke” or “not smoke”. EDT doesn’t take prior information, like motives or genes, into account.
You can observe your preferences and hence take them into account.
Suppose that most people without the lesion find smoking disgusting, while most people with the lesion find it pleasurable. The lesion doesn’t affect your probability of smoking other than by affecting that taste.
The EDT says that should smoke if you find it pleasurable and you shouldn’t if you find it disgusting.
Figuring out what your options are is a hard problem for any decision theory, because it goes to the heart of what we mean by “could”. In toy problems like this, agents just have their options spoon-fed to them. I was trying to show that EDT makes the sensible decision, if it has the right options spoon-fed to it. This opens up at least the possibility that a general EDT agent (that figures out what its options are for itself) would work, because there’s no reason, in principle, why it can’t consider whether the statement “I decided to not smoke, for the purpose of avoiding cancer” would be good news or bad news.
Recognizing this as an option in the first place is a much more complicated issue. But recognizing “smoke” as an option isn’t trivial either. After all, you can’t smoke if there are no cigarettes available. So it seems to me that, if you’re a smoker who just found out about the statistics on smoking and cancer, then the relevant choice you have to make is whether to “decide to quit smoking based on this information about smoking and cancer.”
On the Smoking Lesion, P(Cancer|Smoking) != P(Cancer), by hypothesis- since you don’t know whether or not you have the gene, your decision to smoke does provide you with new evidence about whether or not you do, and therefore about whether or not you are likely to get cancer. (Crucially, it doesn’t cause you to get cancer- all along you either had the gene or didn’t. But EDT works based on correlation, not causation.) For example, one way to rig the probabilities is that half of people smoke, and half of those people get cancer, while nobody who doesn’t smoke gets cancer. (And, again, smoking doesn’t cause cancer.) Then P(Cancer|Smoking)=.5, P(Cancer)=.25, and choosing to smoke is still the right decision.
The one way you can get a correct correlation between your decision and the utility gained is if you refuse to update your belief that you have the gene based on the evidence of your decision. (This is what CDT does.) But this method fails on Newcomb’s Problem. EDT wins on Newcomb’s Problem precisely because it has you update on who you are based on your decision: P(Kind of Person Who Would One-Box|Decision to One-Box) != P(Kind of Person Who Would One-Box). But if you decide not to update your beliefs about your character traits based on your decision, then this is lost.
One way to solve this is to give the agent access to its own code/knowledge about itself, and have it condition on that knowledge. That is essentially what Wei Dai’s Updateless Decision Theory (UDT) does- it doesn’t update its knowledge about its code based on its decision (thus the name). UDT still works on the Prisoner’s Dilemma, because the agent doesn’t know what decision its code will output until it has made the decision, so it still needs to update Omega’s (by assumption, accurate) prediction of its output based on its actual output. (Decision theories, in general, can’t condition on their output- doing so leads to the five-and-ten problem. So UDT doesn’t start out knowing whether it is the kind of agent that one-boxes or two-boxes- though it does start out knowing its own (genetic or computer) code.)
(This comment would probably be significantly improved by having diagrams. I may add some at some point.)
Correct: P(Cancer|Smoking) > P(Cancer). When I said P(O|A) = P(O), I was using A to denote “the decision to not smoke, for the purpose of avoiding cancer.” And this is given by the hypothesis of Smoker’s lesion. The whole premise is that once we correct for the presence of the genetic lesion (which causes both love of smoking and cancer) smoking is not independently associated with cancer. This also suggests that once we correct for love of smoking, smoking is not independently associated with cancer. So if you know that your reason for (not) smoking has nothing to do with how much you like smoking, then the knowledge that you (don’t) smoke doesn’t make it seem any more (or less) likely that you’ll get cancer.
Ah, I see.
Unfortunately, “the decision to not smoke, for the purpose of avoiding cancer” and “the decision not to smoke, for any other reason” are not distinct actions. The actions available are simply “smoke” or “not smoke”. EDT doesn’t take prior information, like motives or genes, into account.
You can observe your preferences and hence take them into account.
Suppose that most people without the lesion find smoking disgusting, while most people with the lesion find it pleasurable. The lesion doesn’t affect your probability of smoking other than by affecting that taste.
The EDT says that should smoke if you find it pleasurable and you shouldn’t if you find it disgusting.
Is that explicitly forbidden by some EDT axiom? It seems quite natural for an EDT agent to know its own motives for its decision.
Figuring out what your options are is a hard problem for any decision theory, because it goes to the heart of what we mean by “could”. In toy problems like this, agents just have their options spoon-fed to them. I was trying to show that EDT makes the sensible decision, if it has the right options spoon-fed to it. This opens up at least the possibility that a general EDT agent (that figures out what its options are for itself) would work, because there’s no reason, in principle, why it can’t consider whether the statement “I decided to not smoke, for the purpose of avoiding cancer” would be good news or bad news.
Recognizing this as an option in the first place is a much more complicated issue. But recognizing “smoke” as an option isn’t trivial either. After all, you can’t smoke if there are no cigarettes available. So it seems to me that, if you’re a smoker who just found out about the statistics on smoking and cancer, then the relevant choice you have to make is whether to “decide to quit smoking based on this information about smoking and cancer.”
(edited for clarity)