‘what values to put in the AI’ problem is also very hard.
Yes, it is hard. But Eliezer isn’t just handwaving it. Here for example is a 37-page document he wrote on the subject 19 years ago:
https://intelligence.org/files/CEV.pdf
Sure, I read that a few years after he wrote it, and it’s still probably the best idea, but even if it’s feasible it needs superintelligent help! So we have to solve the alignment problem to do it.
Yes, it is hard. But Eliezer isn’t just handwaving it. Here for example is a 37-page document he wrote on the subject 19 years ago:
https://intelligence.org/files/CEV.pdf
Sure, I read that a few years after he wrote it, and it’s still probably the best idea, but even if it’s feasible it needs superintelligent help! So we have to solve the alignment problem to do it.