my current thinking is that “should” usually means “better according to some shared, motivating standard or procedure of evaluation”, but occasionally it can also be used to instill such a standard or procedure of evaluation in someone
A should statement interpreted in the form “taking action A seems to be more optimal according to your values” relates an action (or its outcome) with agent’s values. If an agent is gullible and accepts a statements like that even if it’s not true, a pressure for consistency may lead to resolving the error by either disbelieving the statement or by changing the values.
A should statement interpreted in the form “taking action A seems to be more optimal according to your values” relates an action (or its outcome) with agent’s values. If an agent is gullible and accepts a statements like that even if it’s not true, a pressure for consistency may lead to resolving the error by either disbelieving the statement or by changing the values.