What I took away from this post is that confusing a decision-theoretic utility function with hedonic utility will make you very sick, and you might have to go to the hospital. I like this.
It would be interesting to see more exposition and discussion of hedonic utility. For example, why is there a distinction between positive and negative hedonic utility (i.e., hedons vs dolors), which do not seem to make decision theoretic sense? Has anyone proposed a design for an AI or reinforcement learning agent that can be said to make use of hedonic utility, which might help explain its evolutionary purpose?
I think I’m not quite understanding your question.
If I’ve understood you correctly, you’re asking why we’re wired to respond differently to avoiding changes that make us less-happy (that is, avoiding dolors) than to seeking out changes which make us more-happy (that is, seeking out hedons) even if the magnitude of the change is the same. For example, why avoiding a loss motivates us differently than gaining something of equivalent value.
If that’s right, can you clarify why you expect an answer beyond “historical reasons”?
That is, we have a lot of independent systems for measuring “hedons” and “dolors” in different modalities; we respond to grief with different circuits than we respond to pain, for example. We create this hypothetical construct of an inter-modal “hedon/dolor” based on people’s lottery behavior… do I prefer a 50% chance of losing my husband or having an elephant jump up and down on my leg for ten minutes, and so forth. And we know that people have inconsistent lottery behaviors and can be Dutch booked, so a “hedon/dolor” is at best an idealization of what are in humans several different inconsistently-commensurable units of happiness and unhappiness.
Is there anything else that needs to be explained, here?
It sounds like you’re assuming that this jury-rigged system was specifically selected for, and you want to know what exerted the selection pressure, when there doesn’t seem to be any reason to assume that it’s anything more than the best compromise available between a thousand independently-selected-for motivational systems operating on the same brain.
I’s not clear that the two can be reconciled. It’s also not clear that the two can’t be reconciled.
Suppose for simplicity there are just hedons and dolors into which every utilitarian reaction can be resolved and which are independent. Then every event occupies a point in a plane. Now, ordering real numbers (hedons with no dolorific part or dolors with no hedonic part) is easy and more or less unambiguous. However, it’s not immediately obvious whether there’s a useful way to specify an order over all events. A zero hedon, one dolor event clearly precedes a one hedon, zero dolor event in the suck--->win ordering. But what about a one hedon, one dolor event vs. a zero hedon, zero dolor event?
It might seem like that we can simply take the signed difference of the parts (so in that last example, 1-1=0-0 so the events are ‘equal’), but the stipulation of independence seems like it forbids such arithmetic (like subtracting apples from oranges).
Orders on the complex numbers that have been used for varying applications (assuming this has been done) might shed some light on the matter.
Clearly a CEV over all complex (i.e. consisting of exactly a possibly-zero hedonic part and possibly-zero dolorific part) utilities would afford comparison between any two events, but this doesn’t seem to help much at this point.
Beyond knowledge of the physical basis of pleasure and pain, brain scans of humans experiencing masochistic pleasure might be a particularly efficient insight generator here. Even if, say, pure pleasure and pure pain appear very differently on an MRI, it might be possible to reduce them to a common unit of utilitarian experience that affords direct comparison. On the other hand, we might have to conclude that there are actually millions of incommensurable ‘axes’ of utilitarian experience.
Orders on the complex numbers that have been used for varying applications (assuming this has been done) might shed some light on the matter.
It can be proven that there is no ordering of complex numbers which is compatible with the normal conventions of multiplication and addition. It’s not even possible to reliably seperate complex numbers into “positive” and “negative”, such that multiplying two positive numbers gives a positive number, multiplying a positive number by −1 gives a negative number, multiplying a negative number by −1 gives a positive number, and −1 is negative.
To further complicate the matter, I don’t think that hedons and dolors are fully independant; if you place the ‘hedons’ line along the x-axis, the ‘dolors’ line may be a diagonal. Or a curve.
Then I suppose the next question in this line would be: To what extent can we impose useful orders on R^2? (I’d need to study the proof in more detail, but it seems that the no-go theorem on C arises from its ring structure, so we have to drop it.) I’m thinking the place to start is specifying some obvious properties (e.g. an outcome with positive hedonic part and zero dolorific part always comes after the opposite, i.e. is better), though I’m not sure if there’d be enough of them to begin pinning something down.
Edit: Or, oppositely, chipping away at suspect ring axioms and keeping as much structure as possible. Though if it came down to case-checking axioms, it might explode.
The most useful order on R^2 seems to be an order by the absolute value. (That is to say, the distance from the origin.) This is an ordering that has many uses, and gives us certain insights into the structure of the space. (Note though that it is only a partial order, not a complete one, as you can have two different points with the same absolute value.)
Yeah, absolute value is the second-most obvious one, but I think it breaks down:
It seems that if we assume utility to be a function of exactly (i.e. no more and no less than) hedons and dolors in R^2, we might as well stipulate that each part is nonnegative because it would then seem that any sense of dishedons must be captured by dolors and vice versa. So it seems that we may assume nonegativity WLOG. Then given nonnegativity of components, we can actually compare outcomes with the same absolute value:
Given nonnegativity, we can simplify (I’m pretty sure, but even if not, I think a slightly modified argument still goes through) our metric from sqrt(h^2+d^2) (where h,d are the hedonic and dolorific parts) to just d+h. Now suppose that (h1,d1) and (h2,d2) are such that h1+d1=h2+d2. Then:
1) If h1d2 and so (h1,d1) is clearly worse than (h2,d2) 2) If h1=h2, then d1=d2 and equipreferable 3) If h1>h2, then d1<d2 and so (h1,d1) is clearly better than (h2,d2)
So within equivalence classes there will be differing utilities.
Moreover, (0,2)<<(0,0)<<(2,0) but the LHS and RHS fall in the same equivalence classs under absolute value. So the intervals of utility occupied by equivalence classes can overlap. (Where e.g. ‘A<<B’ means ‘B is preferable over A’.)
Hence absolute value seems incompatible with the requirements of a utility ordering.
~
The most obvious function of (h,d) to form equivalence classes is h minus d as in my earlier comment, but that seems to break down (if we assume every pair of elements in a given equivalence class has the same utility) by its reliance on fungibility of hedons and dolors. A ‘marginal dolor function’ that gives the dolor-worth of the next hedon after already having x hedons seems like it might fix this, but it also seems like it would be a step away from practicality.
You are correct, it does break down like that. Actually, for some reason I wasn’t thinking of a space where you want to maximize one value and minimize another, but one where you want to maximize both. That is a reasonable simplification, but it does not translate well to our problem.
Another potential solution if you want to maximize hedons and dolors, you could try sorting by the arguments of points. (i.e. maximize tan(hedons/dolors) or in other words, (given that both hedons and dolors are positive), maximize hedons/dolors itself.)
Ultimately, I think you need some relation between hedons and dolors, something like “one hedon is worth −3.67 dolors” or similar. In the end, you do have have to choose whether (1 hedon, 1 dolor) is preferable to (0 hedons, 0 dolors). (And also whether (2 hedons, 1 dolor) is preferable to (1 hedon, 0 dolors), and whether (1 hedon, 2 dolors) is preferable to (0 hedons, 1 dolor), and so forth.)
I suspect this relation would be linear, as the way we have defined hedons and dolors seems to suggest this, but more than that has to be left up to the agent who this utility system belongs to. And on pain of lack of transitivity in his or her preferences, that agent does seem to need to have one relation like this or another.
Then 0.002 hedons and 0.00001 dolors is 20 times better than 10 hedons and 1 dolor. This would be surprising.
Ultimately, I think you need some relation between hedons and dolors, something like “one hedon is worth −3.67 dolors” or similar.
That’s linear, with a scaling factor. If it is linear, then the scaling factor doesn’t really matter much (‘newdolors’ can be defined as ‘dolors’ times the scaling factor, then one hedon is equal to one newdolor). But if it’s that simple, then it’s basically a single line that we’re dealing with, not a plane.
There are any number of possible alternative (non-linear) functions; perhaps the fifth power of the total number of dolors is equivalent to the fourth power of the number of hedons? Perhaps, and this I consider far more likely, the actual relationship between hedons and dolors is nowhere near that neat...
I would suspect that there are several different, competing functions at use here; many of which may be counter-productive. For example; very few actions produce ten billion hedons. Therefore, if I find a course of action that seems (in advance) to produce ten billion or more hedons, then it is more likely that I am mistaken, or have been somehow fooled by some enemy, than that my estimations are correct. Thus, I am automatically suspicious of such a course of action. I don’t dismiss it out-of-hand, but I am extremely cautious in proceeding towards that outcome, looking out for the hidden trap.
we might have to conclude that there are actually millions of incommensurable ‘axes’ of utilitarian experience.
Yeah, I guess I more or less take this for granted. Or, rather, not that they’re incommensurable, exactly, but that the range of correspondences—how many Xs are worth a Y—is simply an artifact of what set of weighting factors was most effective, among those tested, in encouraging our ancestors to breed, which from our current perspective is just an arbitrary set of historical factors.
I think it might be due to the type of problem we are facing as living entities. We have a consistent never ending goal of “not killing ourselves” and “not mucking up our chances of reproduction”. Pain is one of the signs that we might be near doing these things. Every day we manage not to do these things is in some way a good day. This presents a baseline of utility where anything less than it is considered negative and anything more than that positive. So it just might be what this type of algorithm feels like from the inside.
What I took away from this post is that confusing a decision-theoretic utility function with hedonic utility will make you very sick, and you might have to go to the hospital. I like this.
Stay safe!
It would be interesting to see more exposition and discussion of hedonic utility. For example, why is there a distinction between positive and negative hedonic utility (i.e., hedons vs dolors), which do not seem to make decision theoretic sense? Has anyone proposed a design for an AI or reinforcement learning agent that can be said to make use of hedonic utility, which might help explain its evolutionary purpose?
I think I’m not quite understanding your question.
If I’ve understood you correctly, you’re asking why we’re wired to respond differently to avoiding changes that make us less-happy (that is, avoiding dolors) than to seeking out changes which make us more-happy (that is, seeking out hedons) even if the magnitude of the change is the same. For example, why avoiding a loss motivates us differently than gaining something of equivalent value.
If that’s right, can you clarify why you expect an answer beyond “historical reasons”?
That is, we have a lot of independent systems for measuring “hedons” and “dolors” in different modalities; we respond to grief with different circuits than we respond to pain, for example. We create this hypothetical construct of an inter-modal “hedon/dolor” based on people’s lottery behavior… do I prefer a 50% chance of losing my husband or having an elephant jump up and down on my leg for ten minutes, and so forth. And we know that people have inconsistent lottery behaviors and can be Dutch booked, so a “hedon/dolor” is at best an idealization of what are in humans several different inconsistently-commensurable units of happiness and unhappiness.
Is there anything else that needs to be explained, here?
It sounds like you’re assuming that this jury-rigged system was specifically selected for, and you want to know what exerted the selection pressure, when there doesn’t seem to be any reason to assume that it’s anything more than the best compromise available between a thousand independently-selected-for motivational systems operating on the same brain.
Or have I misunderstood your question?
I’s not clear that the two can be reconciled. It’s also not clear that the two can’t be reconciled.
Suppose for simplicity there are just hedons and dolors into which every utilitarian reaction can be resolved and which are independent. Then every event occupies a point in a plane. Now, ordering real numbers (hedons with no dolorific part or dolors with no hedonic part) is easy and more or less unambiguous. However, it’s not immediately obvious whether there’s a useful way to specify an order over all events. A zero hedon, one dolor event clearly precedes a one hedon, zero dolor event in the suck--->win ordering. But what about a one hedon, one dolor event vs. a zero hedon, zero dolor event?
It might seem like that we can simply take the signed difference of the parts (so in that last example, 1-1=0-0 so the events are ‘equal’), but the stipulation of independence seems like it forbids such arithmetic (like subtracting apples from oranges).
Orders on the complex numbers that have been used for varying applications (assuming this has been done) might shed some light on the matter.
Clearly a CEV over all complex (i.e. consisting of exactly a possibly-zero hedonic part and possibly-zero dolorific part) utilities would afford comparison between any two events, but this doesn’t seem to help much at this point.
Beyond knowledge of the physical basis of pleasure and pain, brain scans of humans experiencing masochistic pleasure might be a particularly efficient insight generator here. Even if, say, pure pleasure and pure pain appear very differently on an MRI, it might be possible to reduce them to a common unit of utilitarian experience that affords direct comparison. On the other hand, we might have to conclude that there are actually millions of incommensurable ‘axes’ of utilitarian experience.
It can be proven that there is no ordering of complex numbers which is compatible with the normal conventions of multiplication and addition. It’s not even possible to reliably seperate complex numbers into “positive” and “negative”, such that multiplying two positive numbers gives a positive number, multiplying a positive number by −1 gives a negative number, multiplying a negative number by −1 gives a positive number, and −1 is negative.
To further complicate the matter, I don’t think that hedons and dolors are fully independant; if you place the ‘hedons’ line along the x-axis, the ‘dolors’ line may be a diagonal. Or a curve.
That settled that quickly. Thanks.
Then I suppose the next question in this line would be: To what extent can we impose useful orders on R^2? (I’d need to study the proof in more detail, but it seems that the no-go theorem on C arises from its ring structure, so we have to drop it.) I’m thinking the place to start is specifying some obvious properties (e.g. an outcome with positive hedonic part and zero dolorific part always comes after the opposite, i.e. is better), though I’m not sure if there’d be enough of them to begin pinning something down.
Edit: Or, oppositely, chipping away at suspect ring axioms and keeping as much structure as possible. Though if it came down to case-checking axioms, it might explode.
The most useful order on R^2 seems to be an order by the absolute value. (That is to say, the distance from the origin.) This is an ordering that has many uses, and gives us certain insights into the structure of the space. (Note though that it is only a partial order, not a complete one, as you can have two different points with the same absolute value.)
Yeah, absolute value is the second-most obvious one, but I think it breaks down:
It seems that if we assume utility to be a function of exactly (i.e. no more and no less than) hedons and dolors in R^2, we might as well stipulate that each part is nonnegative because it would then seem that any sense of dishedons must be captured by dolors and vice versa. So it seems that we may assume nonegativity WLOG. Then given nonnegativity of components, we can actually compare outcomes with the same absolute value:
Given nonnegativity, we can simplify (I’m pretty sure, but even if not, I think a slightly modified argument still goes through) our metric from sqrt(h^2+d^2) (where h,d are the hedonic and dolorific parts) to just d+h. Now suppose that (h1,d1) and (h2,d2) are such that h1+d1=h2+d2. Then:
1) If h1d2 and so (h1,d1) is clearly worse than (h2,d2)
2) If h1=h2, then d1=d2 and equipreferable
3) If h1>h2, then d1<d2 and so (h1,d1) is clearly better than (h2,d2)
So within equivalence classes there will be differing utilities.
Moreover, (0,2)<<(0,0)<<(2,0) but the LHS and RHS fall in the same equivalence classs under absolute value. So the intervals of utility occupied by equivalence classes can overlap. (Where e.g. ‘A<<B’ means ‘B is preferable over A’.)
Hence absolute value seems incompatible with the requirements of a utility ordering.
~
The most obvious function of (h,d) to form equivalence classes is h minus d as in my earlier comment, but that seems to break down (if we assume every pair of elements in a given equivalence class has the same utility) by its reliance on fungibility of hedons and dolors. A ‘marginal dolor function’ that gives the dolor-worth of the next hedon after already having x hedons seems like it might fix this, but it also seems like it would be a step away from practicality.
You are correct, it does break down like that. Actually, for some reason I wasn’t thinking of a space where you want to maximize one value and minimize another, but one where you want to maximize both. That is a reasonable simplification, but it does not translate well to our problem.
Another potential solution if you want to maximize hedons and dolors, you could try sorting by the arguments of points. (i.e. maximize tan(hedons/dolors) or in other words, (given that both hedons and dolors are positive), maximize hedons/dolors itself.)
Ultimately, I think you need some relation between hedons and dolors, something like “one hedon is worth −3.67 dolors” or similar. In the end, you do have have to choose whether (1 hedon, 1 dolor) is preferable to (0 hedons, 0 dolors). (And also whether (2 hedons, 1 dolor) is preferable to (1 hedon, 0 dolors), and whether (1 hedon, 2 dolors) is preferable to (0 hedons, 1 dolor), and so forth.)
I suspect this relation would be linear, as the way we have defined hedons and dolors seems to suggest this, but more than that has to be left up to the agent who this utility system belongs to. And on pain of lack of transitivity in his or her preferences, that agent does seem to need to have one relation like this or another.
Then 0.002 hedons and 0.00001 dolors is 20 times better than 10 hedons and 1 dolor. This would be surprising.
That’s linear, with a scaling factor. If it is linear, then the scaling factor doesn’t really matter much (‘newdolors’ can be defined as ‘dolors’ times the scaling factor, then one hedon is equal to one newdolor). But if it’s that simple, then it’s basically a single line that we’re dealing with, not a plane.
There are any number of possible alternative (non-linear) functions; perhaps the fifth power of the total number of dolors is equivalent to the fourth power of the number of hedons? Perhaps, and this I consider far more likely, the actual relationship between hedons and dolors is nowhere near that neat...
I would suspect that there are several different, competing functions at use here; many of which may be counter-productive. For example; very few actions produce ten billion hedons. Therefore, if I find a course of action that seems (in advance) to produce ten billion or more hedons, then it is more likely that I am mistaken, or have been somehow fooled by some enemy, than that my estimations are correct. Thus, I am automatically suspicious of such a course of action. I don’t dismiss it out-of-hand, but I am extremely cautious in proceeding towards that outcome, looking out for the hidden trap.
Yeah, I guess I more or less take this for granted. Or, rather, not that they’re incommensurable, exactly, but that the range of correspondences—how many Xs are worth a Y—is simply an artifact of what set of weighting factors was most effective, among those tested, in encouraging our ancestors to breed, which from our current perspective is just an arbitrary set of historical factors.
I think it might be due to the type of problem we are facing as living entities. We have a consistent never ending goal of “not killing ourselves” and “not mucking up our chances of reproduction”. Pain is one of the signs that we might be near doing these things. Every day we manage not to do these things is in some way a good day. This presents a baseline of utility where anything less than it is considered negative and anything more than that positive. So it just might be what this type of algorithm feels like from the inside.