What utility function? There isn’t one answer, and I’m not certain it really is a function since different values within it may be incommensurable. It isn’t the easy question, it’s the impossible one. More concretely, what utility function should you use? Yours. Don’t know what it is? Find out. The necessary introspection is actual incredibly difficult, error prone, and full of special cases, so, good luck. Personally, I’d look into the values and virtues you truly hold dear, rather than utilities per se. (That does make the math not as useful, so you’ll have to train your intuition for it.)
Is suffering bad? According to most people, including me, yes, but it is hardly the most important thing. Many people’s utility function (still including me) says people should be willing to undergo extreme torture to save the people they truly love from lesser, but still bad, torture.
Is pleasure good? Sure, I prefer it, as do most people, but I’m fully anti-wireheading.
I see suffering and pleasure as simple, but useful proxies for the actual goodness of the state of the world according to that agent. Like any proxy, focusing too hard on the proxy itself (Goodharting) causes the proxy to no longer be useful as it stops representing that which it is a proxy of.
What utility function? There isn’t one answer, and I’m not certain it really is a function since different values within it may be incommensurable. It isn’t the easy question, it’s the impossible one. More concretely, what utility function should you use? Yours. Don’t know what it is? Find out. The necessary introspection is actual incredibly difficult, error prone, and full of special cases, so, good luck. Personally, I’d look into the values and virtues you truly hold dear, rather than utilities per se. (That does make the math not as useful, so you’ll have to train your intuition for it.)
Is suffering bad? According to most people, including me, yes, but it is hardly the most important thing. Many people’s utility function (still including me) says people should be willing to undergo extreme torture to save the people they truly love from lesser, but still bad, torture.
Is pleasure good? Sure, I prefer it, as do most people, but I’m fully anti-wireheading.
I see suffering and pleasure as simple, but useful proxies for the actual goodness of the state of the world according to that agent. Like any proxy, focusing too hard on the proxy itself (Goodharting) causes the proxy to no longer be useful as it stops representing that which it is a proxy of.