On most of the existing things in the modern universe, it outputs ‘don’t care’, like for dirt.
What do you mean, you don’t care about dirt? I care about dirt! Dirt is where we get most of our food, and humans need food to live. Maybe interstellar hydrogen would be a better example of something you’re indifferent to? 10^17 kg of interstellar hydrogen disappearing would be an inconsequential flicker if we noticed it at all, whereas the loss of an equal mass of arable soil would be an extinction-level event.
Instrumental values are just subgoals that appear when you form plans to achieve your terminal values. They aren’t supposed to be reflected in your utility function. That is a type error plain and simple.
For agents with bounded computational resources, I’m not sure that’s the case. I don’t terminally value money at all, but I pretend I do as a computational approximation because it’d be too expensive for me to run an expected utility calculation over all things I could possibly buy whenever I’m consider gaining or losing money in exchange for something else.
That one is. Instrumental values do not go in utility function. You use instrumental values to shortcut complex utility calculations, but utility calculating shortcut != component of utility function.
What do you mean, you don’t care about dirt? I care about dirt! Dirt is where we get most of our food, and humans need food to live. Maybe interstellar hydrogen would be a better example of something you’re indifferent to? 10^17 kg of interstellar hydrogen disappearing would be an inconsequential flicker if we noticed it at all, whereas the loss of an equal mass of arable soil would be an extinction-level event.
I care about the future consequences of dirt, but not the dirt itself.
(For the love of Belldandy, you people...)
He means that he doesn’t care about dirt for its own sake (e.g. like he cares about other sentient beings for their own sakes).
Yes, and I’m arguing that it has instrumental value anyway. A well-thought-out utility function should reflect that sort of thing.
Instrumental values are just subgoals that appear when you form plans to achieve your terminal values. They aren’t supposed to be reflected in your utility function. That is a type error plain and simple.
For agents with bounded computational resources, I’m not sure that’s the case. I don’t terminally value money at all, but I pretend I do as a computational approximation because it’d be too expensive for me to run an expected utility calculation over all things I could possibly buy whenever I’m consider gaining or losing money in exchange for something else.
I thought that was what I just said...
An approximation is not necessarily a type error.
No, but mistaking your approximation for the thing you are approximating is.
That one is. Instrumental values do not go in utility function. You use instrumental values to shortcut complex utility calculations, but utility calculating shortcut != component of utility function.