I think there is also a local sense in which morals are just aesthetics. The long-term consequences of moral choices mean that evolution plays a big part in determining morality, but divorced from the constraints of evolution and any sense of long-term planning, by what can we objectively compare moral systems other than their popularity? Orthogonality and all that. Are LLMs just modeling that accurately?
I think there is also a local sense in which morals are just aesthetics. The long-term consequences of moral choices mean that evolution plays a big part in determining morality, but divorced from the constraints of evolution and any sense of long-term planning, by what can we objectively compare moral systems other than their popularity? Orthogonality and all that. Are LLMs just modeling that accurately?