A more general problem with utilitarianisms including those that evade the critique in that article:
Suppose we have a computer running a brain sim (along with VR environment). The brain sim works as following: given current state, next state is calculated (using multiple cpus in parallel); the current state is read only, the next state is write only. Think arrays of synaptic values. After all of the next state is calculated, the arrays are switched and the old state data is written over . This is a reductionist model of ‘living’ that is rather easy to think about. Suppose that this being is reasonably happy.
We really want to sacrifice the old state for sake of the new state. If we are to do so based on maximizing utility (rather than seeing the update as a virtue in it’s own right), the utility of new state data has to be greater than utility of the current state data. The utility has to keep rising with each simulator step. That’s clearly not what anyone expects the utility to do. And it clearly has a lot of problems; e.g. when you have multiple brain sims, face a risk of hardware failure, and may want to erase some sim to use freed up memory as backup for some much older sim (whose utility grew over time to a larger value).
I’m very unconvinced that there even exist any ‘utilitarian’ solution here. If you want to maximize some metric over experience-moments that ever happen, then you need to keep track of the experience moments that already happened, to avoid re-doing it (you don’t want to be looping sims over some happy moment). And it is still entirely immoral because you are going to want to destroy everything and create utilitronium.
Why assume that utility is a function of individual states in this model, rather than processes? Can’t a utilitarian deny that instantaneous states, considered apart from context, have any utility?
What is “processes” ? What’s about not switching state data in above example? (You keep re-calculating same state from previous state; if it’s calculation of the next state that is the process then the process is all right)
Also, at that point you aren’t rescuing utilitarianism, you’re going to some sort of virtue ethics where particular changes are virtuous on their own.
Bottom line is, if you don’t define what is processes then you just plug in something undefined through which our intuitions can pour in and make it look all right even if the concept is still fundamentally flawed.
We want to overwrite the old state with the new state. But we would like to preserve old state in a backup if we had unlimited memory. It thus follows that there is a tradeoff decision between worth of old state, worth of new state, and cost of backup. You can proclaim that instantaneous states considered apart from context don’t have any utility. Okay you have what ever context you want, now what are the utilities of the states and the backup, so that we can decide when to do the backup? How often to do the backup? Decide on optimal clock rate? etc.
A process, at a minimum, takes some time (dt > 0). Calculating the next state from previous state would be a process. If you make backups, you could also make additional calculation processes working from those backed-up states. Does that count as “creating more people”? That’s a disputed philosophy of mind question on which reasonable utilitarians might differ, just like anyone else. But if they do say that it creates more people, then we just have yet another weird population ethics question. No more and no less a problem for utilitarianism than the standard population ethics questions, as far as I can see. Nothing follows about each individual’s life having to have ever-increasing utility lest putting that person in stasis be considered better.
I actually would be very curious as of any ideas how ‘utilitarianism’ could be rescued from this. Any ideas?
I don’t believe direct utilitarianism works as a foundation; declaring that the intelligence is about maximizing ‘utility’ just trades one thing (intelligence) that has not been reduced to elementary operations but we at least have good reasons to believe it should be reducible (we are intelligent and laws of physics are, in relevant approximation, computable), for something (“utility”) that not only hasn’t been shown reducible but for which we have no good reason to think it is reducible or works on reductionist models (observe how there’s suddenly a problem with utility of life once I consider a mind upload simulated in a very straightforward way; observe how number of paperclips in the universe is impossible or incredibly difficult to define as a mathematical function).
edit: Note: the model-based utility-based agent does not have real world utility function and as such, no matter how awesomely powerful is the solver it uses to find maximums of mathematical functions, won’t ever care if it’s output gets disconnected from the actuators, unless such condition was explicitly included into model; furthermore it will break itself if model includes itself and it is to modify the model, once again no matter how powerful is it’s solver. The utility is defined within very specific non-reductionist model where e.g. a paperclip is a high level object, and ‘improving’ model (e.g. finding out that paperclip is in fact made of atoms) breaks utility measurement (it was never defined how to recognize when those atoms/quarks/what ever novel physics the intelligence came up with, constitute a paperclip). This is not a deficiency when it comes to solving practical problems other than ‘how do we destroy mankind by accident’.
A more general problem with utilitarianisms including those that evade the critique in that article:
Suppose we have a computer running a brain sim (along with VR environment). The brain sim works as following: given current state, next state is calculated (using multiple cpus in parallel); the current state is read only, the next state is write only. Think arrays of synaptic values. After all of the next state is calculated, the arrays are switched and the old state data is written over . This is a reductionist model of ‘living’ that is rather easy to think about. Suppose that this being is reasonably happy.
We really want to sacrifice the old state for sake of the new state. If we are to do so based on maximizing utility (rather than seeing the update as a virtue in it’s own right), the utility of new state data has to be greater than utility of the current state data. The utility has to keep rising with each simulator step. That’s clearly not what anyone expects the utility to do. And it clearly has a lot of problems; e.g. when you have multiple brain sims, face a risk of hardware failure, and may want to erase some sim to use freed up memory as backup for some much older sim (whose utility grew over time to a larger value).
I’m very unconvinced that there even exist any ‘utilitarian’ solution here. If you want to maximize some metric over experience-moments that ever happen, then you need to keep track of the experience moments that already happened, to avoid re-doing it (you don’t want to be looping sims over some happy moment). And it is still entirely immoral because you are going to want to destroy everything and create utilitronium.
Why assume that utility is a function of individual states in this model, rather than processes? Can’t a utilitarian deny that instantaneous states, considered apart from context, have any utility?
What is “processes” ? What’s about not switching state data in above example? (You keep re-calculating same state from previous state; if it’s calculation of the next state that is the process then the process is all right)
Also, at that point you aren’t rescuing utilitarianism, you’re going to some sort of virtue ethics where particular changes are virtuous on their own.
Bottom line is, if you don’t define what is processes then you just plug in something undefined through which our intuitions can pour in and make it look all right even if the concept is still fundamentally flawed.
We want to overwrite the old state with the new state. But we would like to preserve old state in a backup if we had unlimited memory. It thus follows that there is a tradeoff decision between worth of old state, worth of new state, and cost of backup. You can proclaim that instantaneous states considered apart from context don’t have any utility. Okay you have what ever context you want, now what are the utilities of the states and the backup, so that we can decide when to do the backup? How often to do the backup? Decide on optimal clock rate? etc.
A process, at a minimum, takes some time (dt > 0). Calculating the next state from previous state would be a process. If you make backups, you could also make additional calculation processes working from those backed-up states. Does that count as “creating more people”? That’s a disputed philosophy of mind question on which reasonable utilitarians might differ, just like anyone else. But if they do say that it creates more people, then we just have yet another weird population ethics question. No more and no less a problem for utilitarianism than the standard population ethics questions, as far as I can see. Nothing follows about each individual’s life having to have ever-increasing utility lest putting that person in stasis be considered better.
I actually would be very curious as of any ideas how ‘utilitarianism’ could be rescued from this. Any ideas?
I don’t believe direct utilitarianism works as a foundation; declaring that the intelligence is about maximizing ‘utility’ just trades one thing (intelligence) that has not been reduced to elementary operations but we at least have good reasons to believe it should be reducible (we are intelligent and laws of physics are, in relevant approximation, computable), for something (“utility”) that not only hasn’t been shown reducible but for which we have no good reason to think it is reducible or works on reductionist models (observe how there’s suddenly a problem with utility of life once I consider a mind upload simulated in a very straightforward way; observe how number of paperclips in the universe is impossible or incredibly difficult to define as a mathematical function).
edit: Note: the model-based utility-based agent does not have real world utility function and as such, no matter how awesomely powerful is the solver it uses to find maximums of mathematical functions, won’t ever care if it’s output gets disconnected from the actuators, unless such condition was explicitly included into model; furthermore it will break itself if model includes itself and it is to modify the model, once again no matter how powerful is it’s solver. The utility is defined within very specific non-reductionist model where e.g. a paperclip is a high level object, and ‘improving’ model (e.g. finding out that paperclip is in fact made of atoms) breaks utility measurement (it was never defined how to recognize when those atoms/quarks/what ever novel physics the intelligence came up with, constitute a paperclip). This is not a deficiency when it comes to solving practical problems other than ‘how do we destroy mankind by accident’.