That’s even less tractable a problem than summing over the utility functions of all existing agents, but that’s not necessarily a game-changer. There are some other odd features of this idea, though:
It only seems to work with preference utilitarianism; pleasure/pain utilitarianism would still treat the painless death of an agent with neutral expected utility as neutral. Fair enough; preference utilitarianism seems less broken than conventional utilitarianism anyway.
Contingent on using preference utilitarianism, certain ways of doing the summing lead to odd features regarding changing cultural values: if future preferences are unbounded in time, a big enough stack of dead ancestors with strong enough preferences could render arbitrary social changes unethical. This could be avoided by summing only over potential lifespan, time-discounting in some way, or using some kind of nonstandard aggregation function that takes new information into account.
Let’s say we’re now at a point in time T_{n}. We can plan for T_{n1} using only the preferences of existing or previous agents; all very intuitive so far. But let’s say we consider a time T_{k} further in the future. New agents will have been introduced between T_{n} and T_{k}, and there’s no obvious way to take their preferences into account; every option gives us potential inconsistencies between optimal actions planned at T_{n} and optimal actions taken at time T_{k}. The least bad option seems to be doing a probability-weighted average over agents extant in all possible futures, but (besides being just ridiculously intractable) that seems to introduce some weird acausal effects that I’m not sure I want to deal with. Taking the average at least avoids some of the crazier possible consequences, like the utilitarian go forth and multiply that I’m sure you’ve thought of already.
Yeah this only makes sense for preference utilitarianism, I should have mentioned that.
It is strange to be sure. I wonder what the aggregated preferences of humanity would look like. I wouldn’t be to surprised if it ended up being really similar to the aggregated preferences of current humans. Also, adding some sort of EV to this would probably make any issue here go away. But in any case, it seems to be an open problem on how to chose the starting set of utility functions in a moral way. Once things were running, it might work pretty well, especially once death is solved.
Why not just plan for whatever the current set of utility functions is?
In the context of a FAI, it probably wouldn’t want the aggregate utility function to change anyway. But again, deciding which functions to aggregate seems to be unsolved.
Aren’t utility functions kind of… invariant to scaling and addition of a constant value?
That is, you can say that “I would like A more than B” but not “having A makes me happier than you would be having it”. Neither “I’m neither happy or unhappy, so me not existing wouldn’t change anything”. It’s just not defined.
Actually, the only place different people’s utility functions can be added up is in a single person’s mind, that is, “I value seeing X and Y both feeling well twice as much as just X being in such a state”. So “destroying beings with less than average utility” would appeal to those who tend to average utilities instead of summing them. And, of course, it also depends on what they think of those utility functions.
(that is, do we count the utility function of the person before or after giving them antidepressants?)
Of course, the additional problem is that no one sums up utility functions the same way, but there seems to be just enough correllation between individual results that we can start debates over the “right way of summing utiliity functions”.
It’s hard to do utilitarian ethics without commensurate utility functions, and so utilitarian ethical calculations, in the comparatively rare cases where they’re implemented with actual numbers, often use a notion of cardinal utility. (The Wikipedia article’s kind of a mess, unfortunately.) As far as I can tell this has nothing to do with cardinal numbers in mathematics, but it does provide for commensurate utility scales; in this case, you’d probably be mapping preference orderings over possible world-states onto the reals in some way.
There do seem to be some interesting things you could do with pure preference orderings, analogous to decision criteria for ranked-choice voting in politics. As far as I know, though, they haven’t received much attention in the ethics world.
That’s even less tractable a problem than summing over the utility functions of all existing agents, but that’s not necessarily a game-changer. There are some other odd features of this idea, though:
It only seems to work with preference utilitarianism; pleasure/pain utilitarianism would still treat the painless death of an agent with neutral expected utility as neutral. Fair enough; preference utilitarianism seems less broken than conventional utilitarianism anyway.
Contingent on using preference utilitarianism, certain ways of doing the summing lead to odd features regarding changing cultural values: if future preferences are unbounded in time, a big enough stack of dead ancestors with strong enough preferences could render arbitrary social changes unethical. This could be avoided by summing only over potential lifespan, time-discounting in some way, or using some kind of nonstandard aggregation function that takes new information into account.
Let’s say we’re now at a point in time T_{n}. We can plan for T_{n 1} using only the preferences of existing or previous agents; all very intuitive so far. But let’s say we consider a time T_{k} further in the future. New agents will have been introduced between T_{n} and T_{k}, and there’s no obvious way to take their preferences into account; every option gives us potential inconsistencies between optimal actions planned at T_{n} and optimal actions taken at time T_{k}. The least bad option seems to be doing a probability-weighted average over agents extant in all possible futures, but (besides being just ridiculously intractable) that seems to introduce some weird acausal effects that I’m not sure I want to deal with. Taking the average at least avoids some of the crazier possible consequences, like the utilitarian go forth and multiply that I’m sure you’ve thought of already.
Yeah this only makes sense for preference utilitarianism, I should have mentioned that.
It is strange to be sure. I wonder what the aggregated preferences of humanity would look like. I wouldn’t be to surprised if it ended up being really similar to the aggregated preferences of current humans. Also, adding some sort of EV to this would probably make any issue here go away. But in any case, it seems to be an open problem on how to chose the starting set of utility functions in a moral way. Once things were running, it might work pretty well, especially once death is solved.
Why not just plan for whatever the current set of utility functions is? In the context of a FAI, it probably wouldn’t want the aggregate utility function to change anyway. But again, deciding which functions to aggregate seems to be unsolved.
Aren’t utility functions kind of… invariant to scaling and addition of a constant value?
That is, you can say that “I would like A more than B” but not “having A makes me happier than you would be having it”. Neither “I’m neither happy or unhappy, so me not existing wouldn’t change anything”. It’s just not defined.
Actually, the only place different people’s utility functions can be added up is in a single person’s mind, that is, “I value seeing X and Y both feeling well twice as much as just X being in such a state”. So “destroying beings with less than average utility” would appeal to those who tend to average utilities instead of summing them. And, of course, it also depends on what they think of those utility functions.
(that is, do we count the utility function of the person before or after giving them antidepressants?)
Of course, the additional problem is that no one sums up utility functions the same way, but there seems to be just enough correllation between individual results that we can start debates over the “right way of summing utiliity functions”.
It’s hard to do utilitarian ethics without commensurate utility functions, and so utilitarian ethical calculations, in the comparatively rare cases where they’re implemented with actual numbers, often use a notion of cardinal utility. (The Wikipedia article’s kind of a mess, unfortunately.) As far as I can tell this has nothing to do with cardinal numbers in mathematics, but it does provide for commensurate utility scales; in this case, you’d probably be mapping preference orderings over possible world-states onto the reals in some way.
There do seem to be some interesting things you could do with pure preference orderings, analogous to decision criteria for ranked-choice voting in politics. As far as I know, though, they haven’t received much attention in the ethics world.