The AI is now reflectively consistent, but is this the right outcome?
Yes.
Should the whole future of the universe be shaped only by the preferences of those who happen to be alive at some arbitrary point in time?
‘Should’? Us deciding what should be is already us pretending, hoping or otherwise counterfactually assuming for the purposes of discussion that we can choose the fate of the universe. It so happens that many people that happen to be alive at this arbitrary point in time have preferences with altruistic components that could consider future agents. Lucky them, assuming these arbitrary agents get their way.
Presumably, if you’re a utilitarian in the first place, this is probably not the kind of utilitarianism that you’d want to subscribe to.
That may explain my disagreement (or, as phrased, unexpected agreement). I tend to consider utilitarianism (as typically described) to be naive, verging on silly. The U” option you describe at least seems to have the coherency required to be implemented in practice without a catastrophic or absurd result.
Since you found our agreement unexpected, it may give you a better perspective on this post to know that while it’s mostly addressed to utilitarians, I’m not a utilitarian myself. I do have a certain amount of intellectual sympathy towards utilitarianism, and would like to see its most coherent positions, and hear its strongest arguments, so my post was written in that spirit.
I’d also be quite interested in exploring other potentially viable approaches to moral philosophy. Given that you consider utilitarianism to be naive and verging on silly, what approaches do you find promising?
Let’s say I agree with the specific statements, which would be unexpected by the context if I were a utilitarian. I wouldn’t dream of accusing you of being a utilitarian given how much of an insult that would be given my position.
I’d also be quite interested in exploring other potentially viable approaches to moral philosophy. Given that you consider utilitarianism to be naive and verging on silly, what approaches do you find promising?
“The universe should be made to maximise my utility (best satisfy my preferences over possible states of the universe) ” is my moral philosophy. From that foundation altruism and cooperation considerations come into play. Except that some people define that as not a moral philosophy.
It seems to be much more biologically realistic than utilitariaism. Utilitarianism appears to be an ethical system based on clearly signalling how unselfish and nice you are. The signal seems somewhat tarnished by being pretty unbelievable, though. Do these people really avoid nepotism and favouring themselves? Or are they kidding themselves about their motives in the hope of deceiving others?
Yes. If you want your beliefs to pay rent, then you need to choose between features of reality rather than simply choose arbitrarily. Is there anything else that you believe arbitrarily? Why make an exception for moral philosophy? Reminds me of Status Quo Bias or keeping faith even after learning about other religions. Can you name a relevant difference?
Yes.
‘Should’? Us deciding what should be is already us pretending, hoping or otherwise counterfactually assuming for the purposes of discussion that we can choose the fate of the universe. It so happens that many people that happen to be alive at this arbitrary point in time have preferences with altruistic components that could consider future agents. Lucky them, assuming these arbitrary agents get their way.
That may explain my disagreement (or, as phrased, unexpected agreement). I tend to consider utilitarianism (as typically described) to be naive, verging on silly. The U” option you describe at least seems to have the coherency required to be implemented in practice without a catastrophic or absurd result.
Since you found our agreement unexpected, it may give you a better perspective on this post to know that while it’s mostly addressed to utilitarians, I’m not a utilitarian myself. I do have a certain amount of intellectual sympathy towards utilitarianism, and would like to see its most coherent positions, and hear its strongest arguments, so my post was written in that spirit.
I’d also be quite interested in exploring other potentially viable approaches to moral philosophy. Given that you consider utilitarianism to be naive and verging on silly, what approaches do you find promising?
Let’s say I agree with the specific statements, which would be unexpected by the context if I were a utilitarian. I wouldn’t dream of accusing you of being a utilitarian given how much of an insult that would be given my position.
“The universe should be made to maximise my utility (best satisfy my preferences over possible states of the universe) ” is my moral philosophy. From that foundation altruism and cooperation considerations come into play. Except that some people define that as not a moral philosophy.
That seems to be this: http://en.wikipedia.org/wiki/Ethical_egoism—which I would classify as some kind of moral philosophy.
It seems to be much more biologically realistic than utilitariaism. Utilitarianism appears to be an ethical system based on clearly signalling how unselfish and nice you are. The signal seems somewhat tarnished by being pretty unbelievable, though. Do these people really avoid nepotism and favouring themselves? Or are they kidding themselves about their motives in the hope of deceiving others?
It sounds silly and arbitrary when you discharge the references:
“The universe should be made to maximise widrifid’s utility (best satisfy widrifid’s preferences over possible states of the universe)”
Why not replace “widfirid” with “amcknight”? The fact that you happen to be yourself doesn’t sound like a good enough reason.
Is there some reason why moral philosophy can’t be arbitrary?
Yes. If you want your beliefs to pay rent, then you need to choose between features of reality rather than simply choose arbitrarily. Is there anything else that you believe arbitrarily? Why make an exception for moral philosophy? Reminds me of Status Quo Bias or keeping faith even after learning about other religions. Can you name a relevant difference?
That sounds like a good description of moralizing to me!
http://en.wikipedia.org/wiki/Utilitarianism#Criticism_and_defense goes over some of the common issues.