This is true, but thinking in terms of thinking in terms of thinking in terms of fair is not; in some situations I might want to self-modify into a fairness based agent.
I’m thinking of the classic two-person experiment where one distributes a sum of money among the two and the other accepts it or neither gets anything particularly when if the vetoer is using e.g. cold calculation or spite rather than fairness, the distributer makes different decisions than when (the distributer believes) the vetoer is using fairness criteria.
This is true, but thinking in terms of thinking in terms of thinking in terms of fair is not; in some situations I might want to self-modify into a fairness based agent.
I’m thinking of the classic two-person experiment where one distributes a sum of money among the two and the other accepts it or neither gets anything particularly when if the vetoer is using e.g. cold calculation or spite rather than fairness, the distributer makes different decisions than when (the distributer believes) the vetoer is using fairness criteria.