The more I think about it, the more I’m tempted to just bite the bullet and accept that my “empirically observed utility function” (to the degree that such a thing even makes sense) may be bounded, finite, with a lot of its variation spent measuring relatively local things like the prosaic well being of myself and my loved ones, so that there just isn’t much left over to cover anyone outside my monkey sphere except via a generic virtue-ethical term for “being a good citizen n’stuff”.
A first order approximation might be mathematically modeled by taking all the various utilities having to do with “weird infinite utilities”, normalizing all those scenarios by “my ability to affect those outcomes” (so my intrinsic concern for things decreased when I “gave up” on affecting them… which seems broken but also sorta seems like how things might actually work) and then run what’s left through a sigmoid function so their impact on my happiness and behavior is finite and marginal… claiming maybe 1% of my consciously strategic planning time and resource expenditures under normal circumstances.
Under this model, the real meat of my utility function would actually be characterized by the finite number of sigmoid terms that sum together, what each one handles, and the multiplicative factors attached to each term. All the weird “infinity arguments” are probably handled by a generic term for “other issues” that is already handling political tragedies playing out on different continents and the ongoing mass extinction event and looming singularity issues and so on. In comparison, this scheme would need quite a few terms for things like “regular bowel movements”, that are usually near optimal and have multiplicative factors such that any of them can dominate pretty much the entire utility function if anything goes wrong in these domains.
Spelling this out as “a possible description of how my ‘utility function’ actually works” it occurs to me to wonder how far from optimal an agent that was built to work this way would be?
...
“No one can save the world with an upset tummy!”
Compare and contrast from Adam Smith’s Theory of Moral Sentiments: “And when all this fine philosophy was over, when all these humane sentiments had been once fairly expressed, he would pursue his business or his pleasure, take his repose or his diversion, with the same ease and tranquillity, as if no such accident had happened. The most frivolous disaster which could befall himself would occasion a more real disturbance. If he was to lose his little finger to-morrow, he would not sleep to-night; but, provided he never saw them, he will snore with the most profound security over the ruin of a hundred millions of his brethren, and the destruction of that immense multitude seems plainly an object less interesting to him, than this paltry misfortune of his own.”
I largely buy the framework of this comment, as I’ve said elsewhere.
It does still leave the question of how to go about “being a good citizen n’stuff” with the limited portion of your efforts you want to invest in doing so. Most of multifoliaterose’s questions could be reframed in those terms.
I agree that it’s unclear that it makes sense to talk about humans having utility functions; my use of the term was more a manner of speaking than anything else.
It sounds like you’re going with something like Counterargument #5 with something like the Dunbar number determining the point at which your concern for others caps off; this augmented by some desire to “be a good citizen n’stuff”.
Something similar may be true of me, but I’m not sure. I know that I derive a lot of satisfaction from feeling like I’m making the world a better place and am uncomfortable with the idea that I don’t care about people who I don’t know (in light of my abstract belief in space and time independence of moral value); but maybe the intensity of the relevant feelings are sufficiently diminished when the magnitude of uncertainty becomes huge so that other interests predominate.
I feel like if I could prove that course X maximizes expected utility then my interest in pursuing course X would increase dramatically (independently of how small the probabilities are and of the possibility of doing more harm than good) but that having a distinctive sense that I’ll probably change my mind about whether pursuing course X was a good idea significantly decreases my interest in pursuing course X. Finding it difficult to determine whether this reflects my “utility function” or whether there’s a logical argument coming from utilitarianism against pursuing courses that one will probably regret (e.g. probable burnout and disillusionment repelling potentially utilitarian bystanders).
Great Adam Smith quotation; I’ve seen it before, but it’s good to have a reference.
I agree that it’s unclear that it makes sense to talk about humans having utility functions; my use of the term was more a manner of speaking than anything else.
It would be convenient if we could show that all O-maximizers have some
characteristic behavior pattern, as we do with reward maximizers in Appendix
B. We cannot do this, though, because the set of O-maximizers coincides with
the set of all agents; any agents can be written in O-maximizer form.
To prove this, consider an agent A whose behavior is specied by yk = A(yx<k).
Trivially, we can construct an O-maximizer whose utility is 1 if each yn in its
interaction history is equal to A(yx<n), and 0 otherwise. This O-maximizer will
maximize its utility by behaving as A does at every time n. In this way, any
agent can be rewritten as an O-maximizer.
I think the use of both DALYs and dollars in the main article is worth talking about, in context of some of the things you have mentioned. Being a stupid human, I find that it is generally useful for me to express utility to myself in dollars, because I possess a pragmatic faculty for thinking about dollars. I might not bend over to pick up one dollar. I might spend a couple of hours working for $100. There isn’t much difference between one billion and two billion dollars, from my current perspective.
When you ask me how many dollars I would spend to avert the deaths of a million people, the answer can’t be any larger than the amount of dollars I actually have. If you ask me how many dollars I would spend to avoid the suffering associated with a root canal, it could be some noticeable percentage of my net worth.
When we start talking about decisions where thousands of DALYs hang in the balance, my monkey brain has no intuitive sense of the scope of this, and no pragmatic way of engaging with it. I don’t have the resources or power to purchase even one DALY-equivalent under my own valuation!
If the net utility of the universe is actually being largely controlled by infinitesimal probabilities of enormous utilities, then my sense of scale for both risk and value is irrelevant. It hardly matters how many utilons I attribute to a million starving people when I have only so much time and so much money.
I don’t know what, if anything, to conclude from this, except to say that it makes me feel unsuited to reasoning about anything outside the narrow human scope of likelihoods and outcomes.
The more I think about it, the more I’m tempted to just bite the bullet and accept that my “empirically observed utility function” (to the degree that such a thing even makes sense) may be bounded, finite, with a lot of its variation spent measuring relatively local things like the prosaic well being of myself and my loved ones, so that there just isn’t much left over to cover anyone outside my monkey sphere except via a generic virtue-ethical term for “being a good citizen n’stuff”.
A first order approximation might be mathematically modeled by taking all the various utilities having to do with “weird infinite utilities”, normalizing all those scenarios by “my ability to affect those outcomes” (so my intrinsic concern for things decreased when I “gave up” on affecting them… which seems broken but also sorta seems like how things might actually work) and then run what’s left through a sigmoid function so their impact on my happiness and behavior is finite and marginal… claiming maybe 1% of my consciously strategic planning time and resource expenditures under normal circumstances.
Under this model, the real meat of my utility function would actually be characterized by the finite number of sigmoid terms that sum together, what each one handles, and the multiplicative factors attached to each term. All the weird “infinity arguments” are probably handled by a generic term for “other issues” that is already handling political tragedies playing out on different continents and the ongoing mass extinction event and looming singularity issues and so on. In comparison, this scheme would need quite a few terms for things like “regular bowel movements”, that are usually near optimal and have multiplicative factors such that any of them can dominate pretty much the entire utility function if anything goes wrong in these domains.
Spelling this out as “a possible description of how my ‘utility function’ actually works” it occurs to me to wonder how far from optimal an agent that was built to work this way would be?
...
“No one can save the world with an upset tummy!”
Compare and contrast from Adam Smith’s Theory of Moral Sentiments: “And when all this fine philosophy was over, when all these humane sentiments had been once fairly expressed, he would pursue his business or his pleasure, take his repose or his diversion, with the same ease and tranquillity, as if no such accident had happened. The most frivolous disaster which could befall himself would occasion a more real disturbance. If he was to lose his little finger to-morrow, he would not sleep to-night; but, provided he never saw them, he will snore with the most profound security over the ruin of a hundred millions of his brethren, and the destruction of that immense multitude seems plainly an object less interesting to him, than this paltry misfortune of his own.”
I largely buy the framework of this comment, as I’ve said elsewhere.
It does still leave the question of how to go about “being a good citizen n’stuff” with the limited portion of your efforts you want to invest in doing so. Most of multifoliaterose’s questions could be reframed in those terms.
Thanks for your thoughtful comment.
I agree that it’s unclear that it makes sense to talk about humans having utility functions; my use of the term was more a manner of speaking than anything else.
It sounds like you’re going with something like Counterargument #5 with something like the Dunbar number determining the point at which your concern for others caps off; this augmented by some desire to “be a good citizen n’stuff”.
Something similar may be true of me, but I’m not sure. I know that I derive a lot of satisfaction from feeling like I’m making the world a better place and am uncomfortable with the idea that I don’t care about people who I don’t know (in light of my abstract belief in space and time independence of moral value); but maybe the intensity of the relevant feelings are sufficiently diminished when the magnitude of uncertainty becomes huge so that other interests predominate.
I feel like if I could prove that course X maximizes expected utility then my interest in pursuing course X would increase dramatically (independently of how small the probabilities are and of the possibility of doing more harm than good) but that having a distinctive sense that I’ll probably change my mind about whether pursuing course X was a good idea significantly decreases my interest in pursuing course X. Finding it difficult to determine whether this reflects my “utility function” or whether there’s a logical argument coming from utilitarianism against pursuing courses that one will probably regret (e.g. probable burnout and disillusionment repelling potentially utilitarian bystanders).
Great Adam Smith quotation; I’ve seen it before, but it’s good to have a reference.
Obligatory OB link: Bostrom and Ord’s parliamentary model for normative uncertainty/mixed motivations.
They do have them—in this sense:
I think the use of both DALYs and dollars in the main article is worth talking about, in context of some of the things you have mentioned. Being a stupid human, I find that it is generally useful for me to express utility to myself in dollars, because I possess a pragmatic faculty for thinking about dollars. I might not bend over to pick up one dollar. I might spend a couple of hours working for $100. There isn’t much difference between one billion and two billion dollars, from my current perspective.
When you ask me how many dollars I would spend to avert the deaths of a million people, the answer can’t be any larger than the amount of dollars I actually have. If you ask me how many dollars I would spend to avoid the suffering associated with a root canal, it could be some noticeable percentage of my net worth.
When we start talking about decisions where thousands of DALYs hang in the balance, my monkey brain has no intuitive sense of the scope of this, and no pragmatic way of engaging with it. I don’t have the resources or power to purchase even one DALY-equivalent under my own valuation!
If the net utility of the universe is actually being largely controlled by infinitesimal probabilities of enormous utilities, then my sense of scale for both risk and value is irrelevant. It hardly matters how many utilons I attribute to a million starving people when I have only so much time and so much money.
I don’t know what, if anything, to conclude from this, except to say that it makes me feel unsuited to reasoning about anything outside the narrow human scope of likelihoods and outcomes.