Thanks, again for your help :) That makes me feel a lot better. I have the twin difficulties of having severe OCD-related anxiety about weird decision theory problems, and being rather poor at the math required to understand them.
The case of the immortal who becomes uncertain of the reality of their experiences is I think what that “Pascal’s Mugging for Bounded Utilities” article I linked to the the OP was getting at. But it’s a relief to see that it’s just a subset of decisions under uncertainty, rather than a special weird problem.
the importance to the immortal of the welfare of one particular region of any randomly selected planet of those 10^30 might be less than that of Ancient Egypt. Even if they’re very altruistic.
Ok, thanks, I get that now, I appreciate your help. The thing I am really wondering is, does this make any difference at all to how that immortal would make decisions once Ancient Egypt is in the past and cannot be changed? Assuming that they have one of those bounded utility functions where their utility is asymptotic to the bound, but never actually reaches it, I don’t feel like it necessarily would.
If Ancient Egypt is in the past and can’t be changed, the immortal might, in some kind of abstract sense, value that randomly selected planet of those 10^30 worlds less than they valued Egypt. But if they are actually in a situation where they are on that random planet, and need to make altruistic decisions about helping the people on that planet, then their decisions shouldn’t really be affected. Even if the welfare of that planet is less valuable to them than the welfare of Ancient Egypt, that shouldn’t matter if their decisions don’t affect Ancient Egypt and only affect the planet. They would be trading less valuable welfare off against other less valuable welfare, so it would even out. Since their utility function is asymptotic to the bound, they would still act to increase their utility, even if the amount of utility they can generate is very small.
I am totally willing to accept the Egyptology argument if all it is saying is that past events that cannot be changed might affect the value of present-day events in some abstract sense (at least if you have a bounded utility function). Where I have trouble accepting it is if those same unchangeable past events might significantly affect what choices you have to make about future events that you can change. If future welfare is only 0.1x as valuable as past welfare, that doesn’t really matter, because future welfare is the only welfare you are able to affect. If it’s only possible to make a tiny difference, then you might as well try, because a tiny difference is better than no difference. The only time when the tininess seems relevant to decisions is Pascal’s Mugging type scenarios where one decision can generate tiny possibilities of huge utility.
The phrasing here seems to be a confused form of decision making under uncertainty. Instead of the agent saying “I don’t know what the distribution of outcomes will be”, it’s phrased as “I don’t know what my utility function is”.
I think part of it is that I am conflating two different parts of the Egyptology problem. One part is uncertainty: it isn’t possible to know certain facts about the welfare of Ancient Egyptians that might affect how “close to the bound” you are. The other part is that most people have a strong intuition that those facts aren’t relevant to our decisions, whether we are certain of them or not. But there’s this argument that those facts are relevant if you have an altruistic bounded utility function because they affect how much diminishing returns your function has.
For example, I can imagine that if I was an altruistic immortal who was alive during ancient Egypt, I might be unwilling to trade a certainty of a good outcome in ancient Egypt for an uncertain amazingly terrific outcome in the far future because of my bounded utility function. That’s all good, it should help me avoid Pascal’s Mugging. But once I’ve lived until the present day, it feels like I should continue acting the same way I did in the past, continue to be altruistic, but in a bounded fashion. It doesn’t feel like I should conclude that, because of my achievements as an altruist in Ancient Egypt, that there is less value to being an altruist in the present day.
In the case of the immortal, I do have all the facts about Ancient Egypt, but they don’t seem relevant to what I am doing now. But in the past, in Egypt, I was unwilling to trade certain good outcomes for uncertain terrific ones because my bounded utility function meant I didn’t value the larger ones linearly. Now that the events of Egypt are in the past and can’t be changed, does that mean I value everything less? Does it matter if I do, if the decrease in value is proportionate? If I treat altruism in the present day as valuable, does that contradict the fact that I discounted that same value back in Ancient Egypt?
I think that’s why I’m phrasing it as being uncertain of what my utility function is. It feels like if I have a bounded utility function, I should be unwilling (within limits) to trade a sure thing for a small possibility of vast utility, thereby avoiding Pascal’s Mugging and similar problems. But it also feels like, once I have that sure thing, and the fact that I have it cannot be changed, I should be able to continue seeking more utility, and how many sure things I have accumulated in the past should not change that.
I really still don’t know what you mean by “knowing how close to the bound you are”.
What I mean is, if I have a bounded utility function where there is some value, X, and (because the function is bounded) X diminishes in value the more of it there is, what if I don’t know how much X there is?
For example, suppose I have a strong altruistic preference that the universe have lots of happy people. This preference is not restricted by time and space, it counts the existence of happy people as a good thing regardless of where or when they exist. This preference is also agent neutral, it does not matter whether I, personally, am responsible for those people existing and being happy, it is good regardless. This preference is part of a bounded utility function, so adding more happy people starts to have diminishing returns the closer one gets to a certain bound. This allows me to avoid Pascal’s Mugging.
However, if adding more people has diminishing returns because the function is bounded, and my preference is not restricted by time, space, or agency, that means that I have no way of knowing what those diminishing returns are unless I know how many happy people have ever existed in the universe. If there are diminishing returns based on how many people there are, total, in the universe, then the value of adding more people in the future might change depending on how many people existed in the past.
That is what I mean by “knowing how close to the bound” I am. If I value some “X”, what if it isn’t possible to know how much X there is? (like I said before, a version of this for egoistic preferences might be if the X is happiness over your lifetime, and you don’t know how much X there is because you have amnesia or something).
I was hoping that I might be able to fix this issue by making a bounded utility function where X diminishes in value smoothly and proportionately. So a million happy people in ancient Egypt has proportional diminishing returns to a billion and so on. So when I am making choices about maximizing X in the present, the amount of X I get is diminished in value, but it is proportionately diminished, so the decisions that I make remain the same. If there was a vast population in the past, the amount of X I can generate has very small value according to a bounded utility function. But that doesn’t matter because it’s all that I can do.
That way, even if X decreases in value the more of it there is, it will not effect any choices I make where I need to choose between different probabilities of getting different amounts of X in the future.
I suppose I could also solve it by making all of my preferences agent-relative instead of agent-neutral, but I would like to avoid that. Like most people I have a strong moral intuition that my altruistic preferences should be agent-neutral. I suppose it might also get me into conflict with other agents with bounded agent-relative utility functions if we value the same act differently.
If I am explaining this idea poorly, let me try directing you to some of the papers I am referencing. Besides the one I mentioned in the OP, there is this one by Beckstead and Thomas (pages 16, 17, and 18 are where it discusses it).
REA doesn’t help at all there, though. You’re still computing U(2X days of torture) - U(X days of torture)
I think I see my mistake now, I was treating a bounded utility function using REA as subtracting the “unbounded” utilities of the two choices and then comparing the post-subtraction results using the bounded utility function. It looks like you are supposed to judge each one’s utility by the bounded function before subtracting them.
Unfortunately REA doesn’t change anything at all for bounded utility functions. It only makes any difference for unbounded ones.
That’s unfortunate. I was really hoping that it could deal with the Egyptology scenario by subtracting the unknown utility value of Ancient Egypt and only comparing the difference in utility between the two scenarios. That way the total utilitarian (or some other type of altruist) with a bounded utility function would not need to research how much utility the people of Ancient Egypt had in order to know how good adding happy people to the present day world is. That just seems insanely counterintuitive.
I suppose there might be some other way around the Egyptology issue. Maybe if you have a bounded or nonlinear utility function that is sloped at the correct rate it will give the same answer regardless of how happy the Ancient Egyptians were. If they were super happy then the value of whatever good you do in the present is in some sense reduced. But the value of whatever resources you would sacrifice in order to do good is reduced as well, so it all evens out. Similarly, if they weren’t that happy, the value of the good you do is increased, but the value of whatever you sacrifice in order to do that good is increased proportionately. So a utilitarian can go ahead and ignore how happy the ancient Egyptians were when doing their calculations.
It seems like this might work if the bounded function has adding happy lives have diminishing returns at a reasonably steady and proportional rate (but not so steady that it is effectively unbounded and can be Pascal’s Mugged).
With the “long lived egoist” example I was trying to come up with a personal equivalent to the Egyptology problem. In the Egyptology problem, a utilitarian does not know how close they are to the “bound” of their bounded utility function because they do not know how happy the ancient Egyptians were. In the long lived egoist example, they do not know how close to the bound they are because they don’t know exactly how happy and long lived their past self was. It also seems insanely counterintuitive to say that, if you have a bounded utility function, you need to figure out exactly how happy you were as a child in order to figure out how good it is for you to be happy in the future. Again, I wonder if a solution might be to have a bounded utility function with returns that diminish at a steady and proportional rate.
Thank you for your reply. That was extremely helpful to have someone crunch the numbers. I am always afraid of transitivity problems when considering ideas like this, and I am glad it might be possible to avoid the Egyptology objection without introducing any.
Thanks a lot for the reply. That makes a lot of sense and puts my mind more at ease.
To me this sounds more like any non-linear utility, not specifically bounded utility.
You’re probably right, a lot of my math is shaky. Let me try to explain the genesis of the example I used. I was trying to test REA for transitivity problems because I thought that it might have some further advantages to conventional theories. In particular, it seemed to me that by subtracting before averaging, REA could avoid the two examples those articles I references:
1. The total utilitarian with a bounded utility function who needs to research how many happy people lived in ancient Egypt to establish how “close to the bound” they were and therefore how much they should discount future utility.
2. The very long lived egoist with a bounded utility function who vulnerable to Pascal’s mugging because they are unsure of how many happy years they have lived already (and therefore how “close to the bound” they were).
It seemed like REA, by subtracting past utility that they cannot change before doing the calculation, could avoid both those problems. I do not know if those are real problems or if a non-linear/bounded utility with a correctly calibrated discount rate could avoid them anyway, but it seemed worthwhile to find ways around them. But I was really worried that REA might create intransitivity issues with bounded utility functions, the lottery example I was using was an example of the kind of intransitivity problem that I was thinking of.
It also occurred to me that REA might avoid another peril of bounded utility functions that I read about in this article. Here is the relevant quote:
“if you have a bounded utility function and were presented with the following scary situation: “Heads, 1 day of happiness for you, tails, everyone is tortured for a trillion days” you would (if given the opportunity) increase the stakes, preferring the following situation: “Heads, 2 days of happiness for you, tails, everyone is tortured forever. (This particular example wouldn’t work for all bounded utility functions, of course, but something of similar structure would.)”
It seems like REA might be able to avoid that. If we imagine that the person is given a choice between two coins, since they have to pick one, the “one day of happiness+trillion days of torture” is subtracted beforehand, so all the person needs to do is weigh the difference. Even if we get rid of the additional complications of computing infinity that “tortured forever” creates, by replacing it with some larger number like “2 trillion days”, I think it might avoid it.
But I might be wrong about that, especially if REA always gives the same answers in finite situations. If that’s the case it just might be better to find a formulation of an unbounded utility function that does its best to avoid Pascal’s Mugging and also the “scary situations” from the article, even if it does it imperfectly.
That aside, relative expected value is purely a patch that works around some specific problems with infinite expected values, and gives exactly the same results in all cases with finite expected values.
That’s what I thought as well. But then it occurred to me that REA might not give exactly the same results in all cases with finite expected values if one has a bounded utility function. If I am right, this could result in scenarios where someone could have circular values or end up the victim of a money pump.
For example, imagine there is a lottery that costs $1 to for a ticket and generates x utility for odds of y. The value for x is very large, the value for y is quite small, like in Pascal’s mugging. A person with a bounded utility function does not enter it. However, imagine that there is another lottery that costs a penny for a ticket, and generates 0.01x utility for odds of y. Because this person’s utility function is bounded, y odds of 0.01x utility is worth a penny to them, even though y odds of x utility is not worth a dollar to them. The person buys a ticket for a penny. Then they are offered a chance to buy another. Because they are using REA, they only count the difference in utility from buying the new ticket, and do not count the ticket they already have, so they buy another. Eventually they buy 100 tickets. Then someone offers to buy the tickets from them for a dollar. Because they have a bounded utility function, y odds of winning x are less valuable than a dollar, so they take the trade. They are now back where they started.
Does that make sense? Or have I made some sort of error somewhere? (maybe pennies become more valuable the less you have, which would interrupt the cycle?) It seems like with a bounded utility function, REA might have transitivity problems like that. Or have I made a mistake and misunderstood how to discount using REA?
I am really concerned about this, because REA seems like a nice way to address the Egyptology objection to bounded utility functions. You don’t need to determine how much utility already exists in the world by studying Ancient Egypt, because you only take into account the difference in utility, not the total utility, when calculating where the bound is in your utility function. Ditto for the Pascal’s mugging example. So I really want there to be a way to discount the Egyptology stuff, without also generating intransitive preferences.
That can be said about any period in life. It’s just a matter of perspective and circumstances. The best years are never the same for different people.
That’s true, but I think that for the overwhelming majority of people, their childhoods and young adulthoods were at the very least good years, even if they’re not always the best. They are years that contain significantly more good than bad for most people. So if you create a new adult who never had a childhood, and whose lifespan is proportionately shorter, they will have a lower total amount of wellbeing over their lifetime than someone who had a full-length life that included a childhood.
I took a crack at figuring it out here.
I basically take a similar approach to you. I give animals a smaller -u0 penalty if they are less self-aware and less capable of forming the sort of complex eudaimonic preferences that human beings can. I also treat complex eudaimonic preferences as generating greater moral value when satisfied in order to avoid incentivizing creating animals over creating humans.
I think another good way to look at u0 that compliments yours is to look at it as the “penalty for dying with many preferences left unsatisfied.” Pretty much everyone dies with some things that they wanted to do left undone. I think most people have a strong moral intuition that being unable to fulfill major life desires and projects is tragic, and think a major reason death is bad is that it makes us unable to do even more of what we want to do with our lives. I think we could have u0 represent that intuition.
If we go back to Peter Singer’s original formulation of this topic, we can think of unsatisfied preferences as “debts” that are unpaid. So if we have a choice between creating two people who live x years, or 1 person who lives 2x years, assuming their total lifetime happiness is otherwise the same, we should prefer the one person for 2x years. This is because the two people living x years generate the same amount of happiness, but twice the amount of “debt” from unfulfilled preferences. Everyone will die with some unfulfilled preferences because everyone will always want more, and that’s fine and part of being human.
Obviously we need to calibrate this idea delicately in order to avoid any counterintuitive conclusions. If we treat creating a preference as a “debt” and satisfying it as merely “paying the debt” to “break even” then we get anti-natalism. We need to treat the “debt” that creating a preference generates as an “investment” that can “pay off” by creating tremendous happiness/life satisfaction when it is satisfied, but occasionally fails to “pay off” if its satisfaction is thwarted by death or something else.
I think that this approach could also address Isnasene’s question below of figuring out the -u0 penatly for nonhuman animals. Drawing from Singer again, since nonhuman animals are not mentally capable of having complex preferences for the future, they generate a smaller u0 penalty. The preferences that they die without having satisfied are not as strong or complex. This fits nicely with the human intuition that animals are more “replaceable” than humans and are of lesser (although nonzero) moral value. It also fits the intuition that animals with more advanced, human-like minds are of greater moral value.
Using that approach for animals also underscores the importance of treating creating preferences as an “investment” that can “pay off.” Otherwise it generates the counterintuitive conclusion that we should often favor creating animals over humans, since they have a lower u0 penalty. Treating complex preference creation as an “investment” means that humans are capable of generating far greater happiness/satisfaction than animals, which more than outweighs our greater u0 penalty.
We would also need some sort of way to avoid incentivizing the creation of intelligent creatures with weird preferences that are extremely easy to satisfy, or a strong preference for living a short life as an end in itself. This is a problem pretty much all forms of utilitarianism suffer from. I’m comfortable with just adding some kind of hack massively penalizing the creation of creatures with preferences that do not somehow fit with some broad human idea of eudaimonia.
Point taken, but for the average person, the time period of growing up isn’t just a joyless period where they do nothing but train and invest in the future. Most people remember their childhoods as a period of joy and their college years as some of the best of their lives. Growing and learning isn’t just preparation for the future, people find large portions of it to be fun. So the “existing” person would be deprived of all that, whereas the new person would not be.
If someone is in a rut and could either commit suicide or take the reprogramming drug (and expects to have to take it four times before randomizing to a personality that is better than rerolling a new one), why is that worse than killing them and allowing a new human to be created?
If such a drug is so powerful that the new personality is essentially a new person, then you have created a new person whose lifespan will be a normal human lifespan minus however long the original person lived before they got in a rut. By contrast, if they commit suicide and you create a new human, you have created a new person who will likely live a normal human lifespan. So taking the drug even once is clearly worse than suicide + replacement since, all else being equal, it is better to create a someone with a longer lifespan than a shorter one (assuming their lifespan is positive, overall, of course).
So if they don’t want to be killed, that counts as a negative if we do that, even if we replace them with someone happier.
I have that idea as my “line of retreat.” My issue with it is that it is hard to calibrate it so that it leaves as big a birth-death asymmetry as I want without degenerating into full-blown anti-natalism. There needs to be some way to say that the new happy person’s happiness can’t compensate for the original person’s death without saying that the original person’s own happiness can’t compensate for their own death, which is hard. If I calibrate it to avoid anti-natalism it becomes such a small negative that it seems like it could easily be overcome by adding more people with only a little more welfare.
There’s also the two step “kill and replace” method, where in step one you add a new life barely worth living without affecting anyone. Since the new person exists now, they count the same as everyone else, so then in the second step you kill someone and transfer their resources to the new person. If this process gives the new person the same amount of utility as the old one, it seems neutral under total utilitarianism. I suppose under total preference utilitarianism its somewhat worse, since you now have two people dying with unsatisfied preferences instead of one, but it doesn’t seem like a big enough asymmetry for me.
I feel like in order to reject the two step process, and to have as big an asymmetry as I want, I need to be able to reject “mere addition” and accept the Sadistic Conclusion. But that in turn leads to “galaxy far far away issues” where it becomes wrong to have children because of happy people in some far off place. Or “Egyptology” issues where its better for the world to end than for it to decline so future people have somewhat worse lives, and we are obligated to make sure the Ancient Egyptians didn’t have way better lives than ours before we decide on having children. I just don’t know. I want it to stop hurting my brain so badly, but I keep worrying about how there’s no solution that isn’t horrible or ridiculous.
This has degenerate solutions too—it incentivises producing beings that are very easy to satisfy and that don’t mind being killed.
For this one, I am just willing to just decree that creating creatures with a diverse variety of complex human-like psychologies is good, and creating creatures with weird minmaxing unambitious creatures is bad (or at least massively sub-optimal). To put it another way, Human Nature is morally valuable and needs to be protected.
Another resource that helped me on this was Derek Parfit’s essay “What Makes Someone’s Life Go Best.” You might find it helpful, it parallels some of your own work on personal identity and preferences. The essay describes which of our preferences we feel count as part of our “self interest” and which do not. It helped me understand things, like why people general feel obligated to respect people’s “self interest” preferences (i.e. being happy, not dying), but not their “moral preferences” (i.e. making the country a theocracy, executing heretics).
Parfit’s “Success Theory,” as he calls it, basically argues that only preferences that are “about your own life” count as “welfare” or “self interest.” So that means that we would not be making the world a better place by adding lives who prefer that the speed of light stay constant, or that electrons keep having negative charges. That doesn’t defuse the problem entirely, you could still imagine creating creatures with super unambitious life goals. But it gets it part of the way, the rest, again, I deal with by “defending Human Nature.”
I’m more wanting to automate the construction of values from human preferences
I had a question about that. It is probably a silly question since my understanding of decision and game theory is poor. When you were working on that you said that there was no independence of irrelevant alternatives. I’ve noticed that IIA is something that trips me up a lot when I think about population ethics. I want to be able to say something like “Adding more lives might be bad if there is still the option to improve existing ones instead, but might be good if the existing ones have already died and that option is foreclosed.” This violates IIA because I am conditioning whether adding more lives is good on whether there is another alternative or not.
I was wondering if my brain might be doing the thing you described in your post on no IIA, where it is smashing two different values together and getting different results if there are more alternatives. It probably isn’t I am probably just being irrational, but reading that post just felt familiar.
You can always zero out those utilities by decree, and only consider utilities that you can change. There are other patches you can apply. By talking this way, I’m revealing the principle I’m most willing to sacrifice: elegance.
It’s been a long time since you posted this, but if you see my comment, I’d be curious about what some others patches one could apply are. I have pretty severe scrupulosity issues around population ethics and often have trouble functioning because I can’t stop thinking about them. I dislike pure total utilitarianism, but I have trouble rejecting it precisely because of “galaxy far far away” type issues. I spend a lot of time worrying about the idea that I am forced to choose between two alternatives:
1) That (to paraphrase what you said in your critique of total utilitarianism) it is a morally neutral act to kill someone if you replace them with someone whose lifetime utility is equal to the first person’s remaining lifetime utility (and on a larger scale, the Repugnant Conclusion), or
2.That the human race might be obligated to go extinct if it turns out there is some utopia in some other branch of the multiverse, or the Andromeda Galaxy, or in some ancient, undiscovered fallen civilization in the past. Or that if the Earth was going to explode and I could press a button to save it, but it would result in future generations living slightly lower quality lives than present generations, I shouldn’t push the button.
I’d really like to know some ways that I can reject both 1 and 2. I really admire your work on population ethics and find that your thinking on the subject is really closely aligned with my own, except that you’re better at it than me :)
You can get mind states that are ambiguous mixes of awake and asleep.
I am having trouble parsing this statement. Does it mean that when simulating a mind you could also simulate ambiguous awake/asleep in addition to simulating sleep and wakefulness? Or does it mean that a stored, unsimulated mind is ambiguously neither awake or asleep?
That makes a lot of sense, thank you.
Thanks for the reply. It sounds like maybe my mistake was assuming that unsimulated brain data was functionally and morally equivalent to an unconscious brain. From what you are saying it sounds like the data would need to be simulated even to generate unconsciousness.