I’m curious—would you say DNT is a good approximate model of what we ought to do (assuming we were ideally virtuous), or of what you would actually want done? Where ‘should’ selfishness come into things?
For instance, let’s say we’re in a universe with a finite limit on computation, and plan (a) involves setting up an optimal reachable-universe-wide utopia as fast as possible, with the side effect of killing all current humans. Plan (b) involves ensuring that all current humans have utopian futures, at the cost of a one second delay to spreading utopia out into the universe.
From the point of view of DNT or standard total utilitarianism, plan (a) seems superior here. My intuition says it’s preferable too: that’s an extra second for upwards of 10^35 patients. Next to that, the deaths (and optimised replacement) of a mere 10^10 patients hardly registers.
However, most people would pick plan (b); I would too. This amounts to buying my survival at the cost of 10^17 years of others’ extreme happiness. It’s a waste of one second, and it’s astronomically selfish.
[EDIT: I now believe the limiting factor is energy rather than time, so I don’t think this argument works with these numbers; the salient point seems to be that delay loses access to energy; a similar argument still applies, but I think you have to knock off quite a few orders of magnitude (disclaimer: not a physicist)]
It’s hard to see how we could preserve or convert current human lives without being astronomically selfish moral monsters. If saving current humans costs even one nanosecond, then I’m buying my survival for 10^8 years of others’ extreme happiness; still morally monstrous.
Is there a reasonable argument for plan (b) beyond, “Humans are selfish”?
Of course time discounting can make things look different, but I see no moral justification to discount based on time. At best that seems to amount to “I’m more uncertain about X, so I’m going to pretend X doesn’t matter much” (am I wrong on this?). (Even in the infinite case, which I’m not considering above, time discounting doesn’t seem morally justified—just a helpful simplification.)
Excellent question! I think that my actual preferences are some combination of selfish and altruistic (and the same is probably true of most people), and DNT only tries to capture the altruistic part. It is therefore interesting to try writing down a model of how selfish utility aggregates with altruistic utility. A simple “agnostic” formula such as a linear combination with fixed coefficients works poorly, because for any given coefficients it’s easy to come up with a hypothetical where it’s either way too selfish or way too altruistic.
I think that it’s more reasonable to model this aggregation as bargaining between two imaginary agents: a selfish agent that only values you and people close to you, and an altruistic agent with impartial (DNT-ish) preferences. This bargaining can work, for example, according to the Kalai-Smorodinksy solution, with the disagreement point being “purely selfish optimization with probability p and purely altruistic optimization with probability 1−p”, where p is a parameter reflecting your personal level of altruism. Of course, the result of bargaining can be expressed as a single “effective” utility function, which is just a linear combination between the two, but the coefficients depend on the prior and strategy space.
Something of the same nature should apply when a group of people act cooperatively. In this case we can imagine bargaining between an agent that only cares about this group and an impartial agent. Even if the group includes all living people, the two agents will be different since the second assigns value to animals and future people as well.
Of course time discounting can make things look different, but I see no moral justification to discount based on time.
Actually I think time discount is justified and necessary. Without time discount, you get a divergent integral over time and utility is undefined. Another question is, what kind of time discount exactly. One possibility I find alluring is using the minimax-regret decision rule for exponential time discount with a half-life that is allowed to vary between something of the order of τ0 to ∞.
That bargaining approach is indeed interesting, thanks.
On discounting, I need to read more. I’m currently looking through Pareto Principles in Infinite Ethics (other useful suggestions welcome). While I can see that a naive approach gives you divergent integrals and undefined utility, it’s not yet clear to me that there’s no approach which doesn’t (without discounting).
If time discounting truly is necessary, then of course no moral justification is required. But to the extent that that’s an open question (which in my mind, it currently is—perhaps because I lack understanding), I don’t see any purely moral justification to time discount. From an altruistic view with a veil of ignorance, it seems to arbitrarily favour some patients over others.
That lack of a moral justification motivates me to double-check that it really is necessary on purely logical/mathematical grounds.
I’m curious—would you say DNT is a good approximate model of what we ought to do (assuming we were ideally virtuous), or of what you would actually want done? Where ‘should’ selfishness come into things?
For instance, let’s say we’re in a universe with a finite limit on computation, and plan (a) involves setting up an optimal reachable-universe-wide utopia as fast as possible, with the side effect of killing all current humans. Plan (b) involves ensuring that all current humans have utopian futures, at the cost of a one second delay to spreading utopia out into the universe.
From the point of view of DNT or standard total utilitarianism, plan (a) seems superior here. My intuition says it’s preferable too: that’s an extra second for upwards of 10^35 patients. Next to that, the deaths (and optimised replacement) of a mere 10^10 patients hardly registers.
However, most people would pick plan (b); I would too. This amounts to buying my survival at the cost of 10^17 years of others’ extreme happiness. It’s a waste of one second, and it’s astronomically selfish.
[EDIT: I now believe the limiting factor is energy rather than time, so I don’t think this argument works with these numbers; the salient point seems to be that delay loses access to energy; a similar argument still applies, but I think you have to knock off quite a few orders of magnitude (disclaimer: not a physicist)]
It’s hard to see how we could preserve or convert current human lives without being astronomically selfish moral monsters. If saving current humans costs even one nanosecond, then I’m buying my survival for 10^8 years of others’ extreme happiness; still morally monstrous.
Is there a reasonable argument for plan (b) beyond, “Humans are selfish”?
Of course time discounting can make things look different, but I see no moral justification to discount based on time. At best that seems to amount to “I’m more uncertain about X, so I’m going to pretend X doesn’t matter much” (am I wrong on this?). (Even in the infinite case, which I’m not considering above, time discounting doesn’t seem morally justified—just a helpful simplification.)
Excellent question! I think that my actual preferences are some combination of selfish and altruistic (and the same is probably true of most people), and DNT only tries to capture the altruistic part. It is therefore interesting to try writing down a model of how selfish utility aggregates with altruistic utility. A simple “agnostic” formula such as a linear combination with fixed coefficients works poorly, because for any given coefficients it’s easy to come up with a hypothetical where it’s either way too selfish or way too altruistic.
I think that it’s more reasonable to model this aggregation as bargaining between two imaginary agents: a selfish agent that only values you and people close to you, and an altruistic agent with impartial (DNT-ish) preferences. This bargaining can work, for example, according to the Kalai-Smorodinksy solution, with the disagreement point being “purely selfish optimization with probability p and purely altruistic optimization with probability 1−p”, where p is a parameter reflecting your personal level of altruism. Of course, the result of bargaining can be expressed as a single “effective” utility function, which is just a linear combination between the two, but the coefficients depend on the prior and strategy space.
It’s interesting to speculate about the relation between this model and multiagent models of the mind.
Something of the same nature should apply when a group of people act cooperatively. In this case we can imagine bargaining between an agent that only cares about this group and an impartial agent. Even if the group includes all living people, the two agents will be different since the second assigns value to animals and future people as well.
Actually I think time discount is justified and necessary. Without time discount, you get a divergent integral over time and utility is undefined. Another question is, what kind of time discount exactly. One possibility I find alluring is using the minimax-regret decision rule for exponential time discount with a half-life that is allowed to vary between something of the order of τ0 to ∞.
That bargaining approach is indeed interesting, thanks.
On discounting, I need to read more. I’m currently looking through Pareto Principles in Infinite Ethics (other useful suggestions welcome). While I can see that a naive approach gives you divergent integrals and undefined utility, it’s not yet clear to me that there’s no approach which doesn’t (without discounting).
If time discounting truly is necessary, then of course no moral justification is required. But to the extent that that’s an open question (which in my mind, it currently is—perhaps because I lack understanding), I don’t see any purely moral justification to time discount. From an altruistic view with a veil of ignorance, it seems to arbitrarily favour some patients over others.
That lack of a moral justification motivates me to double-check that it really is necessary on purely logical/mathematical grounds.