This strongly resembles the argument given by Subhan in EY’s post Is Morality Preference?, with a side order of Fake Selfishness. You might enjoy reading those posts along with others in their respective sequences. (“Is Morality Preference?” was part of the original metaethics sequence but didn’t make the cut for Rationality: AI to Zombies.)
More to the point, the biggest mistake I see here is the one addressed in The Domain of Your Utility Function: yes, my moral preferences are a part of my map rather than the territory, but there’s still a damn meaningful difference between egoism (preferences that point only to the part of my map labeled “my future experiences”) and my actual moral preferences, which point to many other parts of the map as well.
This strongly resembles the argument given by Subhan in EY’s post Is Morality Preference?, with a side order of Fake Selfishness. You might enjoy reading those posts along with others in their respective sequences. (“Is Morality Preference?” was part of the original metaethics sequence but didn’t make the cut for Rationality: AI to Zombies.)
More to the point, the biggest mistake I see here is the one addressed in The Domain of Your Utility Function: yes, my moral preferences are a part of my map rather than the territory, but there’s still a damn meaningful difference between egoism (preferences that point only to the part of my map labeled “my future experiences”) and my actual moral preferences, which point to many other parts of the map as well.