If with “morality” you mean moral realism, then yes, I agree that it is scary. I’m most scared by the apparent assumption that we have solved the human alignment problem. Looking at history, I don’t feel like our current situation of relative peace is very stable. My impression is that “good” behavior is largely dependent on incentives, and so is the very definition of “good”. Perhaps markets are one of the more successful tools of creating aligned behaviour in humans, but even in that case it only seems to work if the powers of the market participants are balanced, which is not a luxury we have in alignment work.
If with “morality” you mean moral realism, then yes, I agree that it is scary.
I’m most scared by the apparent assumption that we have solved the human alignment problem.
Looking at history, I don’t feel like our current situation of relative peace is very stable.
My impression is that “good” behavior is largely dependent on incentives, and so is the very definition of “good”.
Perhaps markets are one of the more successful tools of creating aligned behaviour in humans, but even in that case it only seems to work if the powers of the market participants are balanced, which is not a luxury we have in alignment work.