I agree that all “ought” statements can be easily translated into “is” statements about maximizing some utility function. But in practice, should-statements often are disguised exhortations to adopt a particular utility function.
I think the question raised by the OP is something like: Why we should take the exhortation seriously if we have not already adopted the particular utility function?
One could try to create a model of agents that respond to such exhortations. Maybe such agents could be uncertain about their own utility function, like in Dewey’s value learning paper.
If agents have different utility functions / conative ambitions / “shoulds”, they will presumably need to engage in some kind of negotiation in order to compromise their values and reach an efficient outcome. Presumably, ethical disputes can function as a way of reaching such outcomes—some accounts of ethics are quite clear in describing ethical reasoning as being very much about such a balancing of “right versus right”. Even Kantian ethics can be seen in such terms, although what we woud call “rights” Kant would perhaps refer to as “principles of practical reason”.
I agree that all “ought” statements can be easily translated into “is” statements about maximizing some utility function. But in practice, should-statements often are disguised exhortations to adopt a particular utility function.
I think the question raised by the OP is something like: Why we should take the exhortation seriously if we have not already adopted the particular utility function?
One could try to create a model of agents that respond to such exhortations. Maybe such agents could be uncertain about their own utility function, like in Dewey’s value learning paper.
If agents have different utility functions / conative ambitions / “shoulds”, they will presumably need to engage in some kind of negotiation in order to compromise their values and reach an efficient outcome. Presumably, ethical disputes can function as a way of reaching such outcomes—some accounts of ethics are quite clear in describing ethical reasoning as being very much about such a balancing of “right versus right”. Even Kantian ethics can be seen in such terms, although what we woud call “rights” Kant would perhaps refer to as “principles of practical reason”.