″ Also the lifespan dilemma shows that for many people the answer can’t be just a matter of expected value, otherwise everyone would agree on reducing the probability of success to values near to 0. ”
This is a delusion that stems from using the formulation of “expected value” without understanding it. The basic idea of expected value derives from a utility function, which is the effect of being able to give consistent answers to every question of the form, “Would you prefer X to Y, or Y to X? Or does it just not matter?” Once you have such a set of consistent answers, a natural way to use a real numbering measuring system falls out, which is such that something is called “twice as good” when it just doesn’t matter whether you get a 50% chance of the “twice as good” thing, or 100% chance of the “half as good” thing.
But this idea of “twice as good” is just defined in this way because it works well. There is no reason whatsoever to assume “twice as much stuff by some other quantitative measure” is twice as good in this sense; and the lifespan dilemma and all sorts of other things definitively proves that it is not. Twice as much stuff (in the measurement sense) is just not twice as good (in the expected value sense), period, not even when you are talking about lifespan or lives or whatever else.
In this sense, if you can give consistent answers to every pair of preference options, you will want to maximize expected utility. There is no reason whatsoever to think you will be willing to drive the probability of success down to values near zero, however, since there is no reason to believe that value of lifespan scales in that way, and definitive reason to believe otherwise.
If I have understood correctly, your utility function is asymptotic. I wonder if an asymptote in an utility function can be consider as a sort of arbitrary limit.
Anyway, I agree with you, an asymptotic utility function can work and maintain consistency.
The limit of the logarithm of x when x approaches infinity is infinity, so: if someone wants to live forever, and doesn’t care about a minimum amount of safety, he should accept the deal.
What would constitute a non-arbitrary limit? A utility function just describes the behavior of a physical being. It is not surprising that a physical being has limits in its behavior—that follows from the fact that physical law imposes limits on what a thing does. This is why e.g. transhumanist hope for real immortality is absurd. Even if you could find a way around entropy, you will never change the fact that you are following physical laws. The only way you will exist forever is if current physical laws extrapolated from your current situation imply that you will exist forever. There is no reason to think this is the case, and extremely strong reasons to think that it is not.
In the same way, everything I do is implied by physical laws, including the fact that I express a preference for one thing rather than another. It may be that a good conman will be able to persuade some people to order their preferences in a way that gets them to commit suicide (e.g. by accepting the lifespan offer), but they will be very unlikely to be able to persuade me to order my preferences that way. This is “arbitrary” only in the sense that in order for this to be true, there have to be physical differences between me and the people who would be persuaded that in theory neither of us is responsible for. I don’t have a problem with that. I still don’t plan to commit suicide.
Then we agree. I too have limits that are defined by my physiology. This is why I think that I couldn’t stand 50 years of torture but I could stand 3^^^3 dust speck diluted among 3^^^3 lives.
″ Also the lifespan dilemma shows that for many people the answer can’t be just a matter of expected value, otherwise everyone would agree on reducing the probability of success to values near to 0. ”
This is a delusion that stems from using the formulation of “expected value” without understanding it. The basic idea of expected value derives from a utility function, which is the effect of being able to give consistent answers to every question of the form, “Would you prefer X to Y, or Y to X? Or does it just not matter?” Once you have such a set of consistent answers, a natural way to use a real numbering measuring system falls out, which is such that something is called “twice as good” when it just doesn’t matter whether you get a 50% chance of the “twice as good” thing, or 100% chance of the “half as good” thing.
But this idea of “twice as good” is just defined in this way because it works well. There is no reason whatsoever to assume “twice as much stuff by some other quantitative measure” is twice as good in this sense; and the lifespan dilemma and all sorts of other things definitively proves that it is not. Twice as much stuff (in the measurement sense) is just not twice as good (in the expected value sense), period, not even when you are talking about lifespan or lives or whatever else.
In this sense, if you can give consistent answers to every pair of preference options, you will want to maximize expected utility. There is no reason whatsoever to think you will be willing to drive the probability of success down to values near zero, however, since there is no reason to believe that value of lifespan scales in that way, and definitive reason to believe otherwise.
If I have understood correctly, your utility function is asymptotic. I wonder if an asymptote in an utility function can be consider as a sort of arbitrary limit.
Anyway, I agree with you, an asymptotic utility function can work and maintain consistency.
No, the utility function just needs to be sublinear. A popular example is that many toy models assign log(X) utility to X dollars.
But then there would still be a very high value that could make you change idea.
The limit of the logarithm of x when x approaches infinity is infinity, so: if someone wants to live forever, and doesn’t care about a minimum amount of safety, he should accept the deal.
“arbitrary limit”
What would constitute a non-arbitrary limit? A utility function just describes the behavior of a physical being. It is not surprising that a physical being has limits in its behavior—that follows from the fact that physical law imposes limits on what a thing does. This is why e.g. transhumanist hope for real immortality is absurd. Even if you could find a way around entropy, you will never change the fact that you are following physical laws. The only way you will exist forever is if current physical laws extrapolated from your current situation imply that you will exist forever. There is no reason to think this is the case, and extremely strong reasons to think that it is not.
In the same way, everything I do is implied by physical laws, including the fact that I express a preference for one thing rather than another. It may be that a good conman will be able to persuade some people to order their preferences in a way that gets them to commit suicide (e.g. by accepting the lifespan offer), but they will be very unlikely to be able to persuade me to order my preferences that way. This is “arbitrary” only in the sense that in order for this to be true, there have to be physical differences between me and the people who would be persuaded that in theory neither of us is responsible for. I don’t have a problem with that. I still don’t plan to commit suicide.
Then we agree. I too have limits that are defined by my physiology. This is why I think that I couldn’t stand 50 years of torture but I could stand 3^^^3 dust speck diluted among 3^^^3 lives.