It seems to me that the problems of morality as preference come from treating humans as monolithic. Real humans with internal complex ecosystems of agents embedded in larger social systems of agents. As such, they would be expected to shift dominant values, maybe even to shift terminal values as an agent builds a new agent to accomplish its goals by optimizing for some proxy to its goals and eventually finds that new agent to pursue that proxy against its explicit preferences. Plato viewed morality as well ordered relationships between agents, that is, presumably some sort of attractor in the space of possible such relationships which leads to most of the reasonably high level agents flourishing in the medium term and the very high level ones flourishing in the long term.
Consistent with the above, morality as a given can simply be part of the universe or multiverse as a given, but this is hard to express. It is a given that certain configurations are perceptions of moral “wrongness” and others are perceptions of moral “rightness”.
It seems to me that the problems of morality as preference come from treating humans as monolithic. Real humans with internal complex ecosystems of agents embedded in larger social systems of agents. As such, they would be expected to shift dominant values, maybe even to shift terminal values as an agent builds a new agent to accomplish its goals by optimizing for some proxy to its goals and eventually finds that new agent to pursue that proxy against its explicit preferences. Plato viewed morality as well ordered relationships between agents, that is, presumably some sort of attractor in the space of possible such relationships which leads to most of the reasonably high level agents flourishing in the medium term and the very high level ones flourishing in the long term.
Consistent with the above, morality as a given can simply be part of the universe or multiverse as a given, but this is hard to express. It is a given that certain configurations are perceptions of moral “wrongness” and others are perceptions of moral “rightness”.