Agreed. CEV is very fuzzy goal, any specific implementation in terms of an AI’s models of human behavior (e.g. dividing human motivation into moral/hedonistic and factual beliefs with some learning model based on experience, then acting on average moral/hedonistic beliefs with accurate information) has plenty of room to fail on the details. But on the other hand, it’s still worth it to talk about whether the fuzzy goal is a good place to look for a specific implementation, and I think it is.
I don’t think it’s correct to say CEV is ‘our current proposal for …’ for two reasons
Anthropomorphizing groups is not generally a good idea.
From what I gather it’s more of a ‘wrong/incomplete proposal useful for communicating strong insights’.
My understanding is very superficial, though, so I may be mistaken.
Agreed. CEV is very fuzzy goal, any specific implementation in terms of an AI’s models of human behavior (e.g. dividing human motivation into moral/hedonistic and factual beliefs with some learning model based on experience, then acting on average moral/hedonistic beliefs with accurate information) has plenty of room to fail on the details. But on the other hand, it’s still worth it to talk about whether the fuzzy goal is a good place to look for a specific implementation, and I think it is.