Sounds like there could be at least two approaches here. One would be CEV. The other would be to consider the smoker as wanting to smoke, or at least to avoid withdrawal cravings, and also to avoid the downsides of smoking. A sufficiently powerful agent operating on this model would try to suppress withdrawals, cure lung cancer or otherwise act in the smoker’s interests. On the other hand, a less powerful agent with this model might try to simply keep the smoker smoking. There’s an interesting question here about to what extent revealed preferences are a person’s true preferences, or whether addictions and the like should be considered an unwanted addition to one’s personality.
Sounds like there could be at least two approaches here. One would be CEV. The other would be to consider the smoker as wanting to smoke, or at least to avoid withdrawal cravings, and also to avoid the downsides of smoking. A sufficiently powerful agent operating on this model would try to suppress withdrawals, cure lung cancer or otherwise act in the smoker’s interests. On the other hand, a less powerful agent with this model might try to simply keep the smoker smoking. There’s an interesting question here about to what extent revealed preferences are a person’s true preferences, or whether addictions and the like should be considered an unwanted addition to one’s personality.