It seems to me that there are multiple types of utility, and wireheading doesn’t satisfy all of them. Things like “life satisfaction” and “momentary pleasure” are empirically separable (as described in e.g. Thinking, Fast and Slow), and seem to point to entirely different kinds of preferences, different terms in the utility function… such that maximizing one at the complete expense of all others is just not going to give you what you want.
How can there be multiple types of utility? There’s going to be a trade-off, so you have to have some system for how much you value each kind. This gives you some sort of meta-utility. Why not throw away the individual utilities and stick with the meta-utility?
That could work… if you take into account the behavior where, if you don’t get enough of one kind of utility, your meta-utility might actually go down.
Utility is whatever you’re trying to maximize the expected value of. If you act in a way that maximizes the expected value of log(happiness) + log(preference fulfillment), for example, this doesn’t mean that you’re risk averse and you have two different kinds of utility. It means that your utility function is log(happiness) + log(preference fulfillment).
That’s true. I’ve been using the term “utility” in a way that is probably wrong. What I really meant is that humans have different kinds of things they want to maximize, and get unhappy if some of them are fulfilled and others aren’t… so their utility functions are complicated.
The obvious problem with standard wireheading is that it only maximizes one of the things humans want.
It seems to me that there are multiple types of utility, and wireheading doesn’t satisfy all of them. Things like “life satisfaction” and “momentary pleasure” are empirically separable (as described in e.g. Thinking, Fast and Slow), and seem to point to entirely different kinds of preferences, different terms in the utility function… such that maximizing one at the complete expense of all others is just not going to give you what you want.
How can there be multiple types of utility? There’s going to be a trade-off, so you have to have some system for how much you value each kind. This gives you some sort of meta-utility. Why not throw away the individual utilities and stick with the meta-utility?
That could work… if you take into account the behavior where, if you don’t get enough of one kind of utility, your meta-utility might actually go down.
Utility is whatever you’re trying to maximize the expected value of. If you act in a way that maximizes the expected value of log(happiness) + log(preference fulfillment), for example, this doesn’t mean that you’re risk averse and you have two different kinds of utility. It means that your utility function is log(happiness) + log(preference fulfillment).
That’s true. I’ve been using the term “utility” in a way that is probably wrong. What I really meant is that humans have different kinds of things they want to maximize, and get unhappy if some of them are fulfilled and others aren’t… so their utility functions are complicated.
The obvious problem with standard wireheading is that it only maximizes one of the things humans want.
True, but it seems to me that there’s no reason you can’t have good wireheading which stimulates all the different types of fuzzies.
Also true, and I’m not sure that’s not ethically sound.