I think there are reasonable shared values, but unless you keep things hopelessly vague (which I don’t think you can or you get other problems) then you’re sure to run into contradictions.
And yeah, some sets of values, even if seemingly benign, will have unintended dangerous consequences in the hands of a superintelligent being. Not even solving human morality (which is already essentially impossible) would suffice for sure.
Well, there has never been a proof that there is a single set of human values, or that human morality is a single entity.
And value alignment isn’t synonymous with safety.
I think there are reasonable shared values, but unless you keep things hopelessly vague (which I don’t think you can or you get other problems) then you’re sure to run into contradictions.
And yeah, some sets of values, even if seemingly benign, will have unintended dangerous consequences in the hands of a superintelligent being. Not even solving human morality (which is already essentially impossible) would suffice for sure.