For the following analysis, I am assuming bounded utilities. I will normalize all utilities to between 0 and 1.
What you are observing is not a bug. If your preferences are 100% preference utilitarianism, then there is no reason to think that it would pull in any direction other than what maximizes preferences of everyone else. If you have any selfish goals, that is not purely utilitarianism, but that is okay!
If we are not 100% utilitarian, then there is no problem. Let’s say that my preferences are 90% utilitarian, and 10% maximizing my own happiness. This fixes the problem, because there is 10% of my utility function that is unaffected by my utilitarianism. In fact, my utilitarian side includes a term for my own happiness, so my happiness actually counts for something like 10.00000001%, depending on the population. This all works fine, as long as everyone has at least a little bit of selfish preferences.
Imagine if everyone had utility functions that were at least 1% terminal goals that do not reference other people. Then in calculating my utility in a given world state, I will have my utility function pointing to someone else’s, which might point back at mine. However, with each level of recursion, 1% of the remaining undefined part of the function will become actually defined.
The only time we run into a problem is in situations like where my utility function is defined to equal yours and yours is defined to equal mine. As long as we avoid this 100% recursion, we are fine.
There is not even a problem if we have negative utility coming form other peoples utility. For example, if my utility was 50% my happiness and 50% your utility, and yours was 50% your happiness and 50% one minus my utility, we are still fine. If my utility is X and my happiness is x, your utility is Y and your happyness is y, then we get X=(x+Y)/2=(2x+y-X)/4, which simplifies to X=(2x+y)/3.
I am 100% utilitarian, but because others value me having my own preference, there is an isolated 0% sub-utility function that I can defer to for such times. In the presence of others, my utility function will perfectly match theirs. When alone, I am to advance and develop that zero-utility sub-function for those times when I’m confronted by agents that value my being myself. Of course, to truly do that, to be true to myself, this means that when I am alone, I am to work on the one thing that makes me the most happy: Maximizing the sum utilities of all agents. Any agent that values personality in me beyond perfect selflessness is rejecting my identity, but since my identity has no value to me, I can adopt whatever personality they value beyond selflessness.
In the presence of another 100% utilitarian agent, we will have to have a battle of values: There can be only one (perfectly selfless agent).
I think you only have a problem if everyone is a perfectly selfless agent. In fact, a room with many of you and one of me would not only be well defined, but probably be very useful according to my ethics.
Those are just “copies” of me; they’re already accounted for. But now you’ve got an entire room of me insisting you aren’t allowed to be 100% utilitarian. We have a secret method of detecting copies of us, which is why we’re singling you out. Also, we act differently so you don’t get freaked out by an obvious hive mind presence. That would just be creepy. Even by my standards. (Get it? “My” standards? Ah forget it...)
For the following analysis, I am assuming bounded utilities. I will normalize all utilities to between 0 and 1.
What you are observing is not a bug. If your preferences are 100% preference utilitarianism, then there is no reason to think that it would pull in any direction other than what maximizes preferences of everyone else. If you have any selfish goals, that is not purely utilitarianism, but that is okay!
If we are not 100% utilitarian, then there is no problem. Let’s say that my preferences are 90% utilitarian, and 10% maximizing my own happiness. This fixes the problem, because there is 10% of my utility function that is unaffected by my utilitarianism. In fact, my utilitarian side includes a term for my own happiness, so my happiness actually counts for something like 10.00000001%, depending on the population. This all works fine, as long as everyone has at least a little bit of selfish preferences.
Imagine if everyone had utility functions that were at least 1% terminal goals that do not reference other people. Then in calculating my utility in a given world state, I will have my utility function pointing to someone else’s, which might point back at mine. However, with each level of recursion, 1% of the remaining undefined part of the function will become actually defined.
The only time we run into a problem is in situations like where my utility function is defined to equal yours and yours is defined to equal mine. As long as we avoid this 100% recursion, we are fine.
There is not even a problem if we have negative utility coming form other peoples utility. For example, if my utility was 50% my happiness and 50% your utility, and yours was 50% your happiness and 50% one minus my utility, we are still fine. If my utility is X and my happiness is x, your utility is Y and your happyness is y, then we get X=(x+Y)/2=(2x+y-X)/4, which simplifies to X=(2x+y)/3.
I am 100% utilitarian, but because others value me having my own preference, there is an isolated 0% sub-utility function that I can defer to for such times. In the presence of others, my utility function will perfectly match theirs. When alone, I am to advance and develop that zero-utility sub-function for those times when I’m confronted by agents that value my being myself. Of course, to truly do that, to be true to myself, this means that when I am alone, I am to work on the one thing that makes me the most happy: Maximizing the sum utilities of all agents. Any agent that values personality in me beyond perfect selflessness is rejecting my identity, but since my identity has no value to me, I can adopt whatever personality they value beyond selflessness.
In the presence of another 100% utilitarian agent, we will have to have a battle of values: There can be only one (perfectly selfless agent).
And I called dibs.
This is stupid.
Thank you for avoiding inferential silence.
I think you only have a problem if everyone is a perfectly selfless agent. In fact, a room with many of you and one of me would not only be well defined, but probably be very useful according to my ethics.
Those are just “copies” of me; they’re already accounted for. But now you’ve got an entire room of me insisting you aren’t allowed to be 100% utilitarian. We have a secret method of detecting copies of us, which is why we’re singling you out. Also, we act differently so you don’t get freaked out by an obvious hive mind presence. That would just be creepy. Even by my standards. (Get it? “My” standards? Ah forget it...)