Also ,there’s no proof that CEV would work. Maybe values are incoherent.
The arbital article is no help.
Asking what everyone would want* if they knew what the AI knew, and doing what they’d all predictably agree on, is just about the least jerky thing you can do.
How do we know that they would agree? That just begs the question. Saying that you shouldn’t be “jerky”, ie. selfish, doesn’t tell you what kind of unselfishness to have instead. Clearly ,the left and the right don’t agree on the best kind of altruism—laying down your life to stop the spread of socialism, versus sacrificing your income to implement socialism.
Also ,there’s no proof that CEV would work. Maybe values are incoherent.
The arbital article is no help.
How do we know that they would agree? That just begs the question. Saying that you shouldn’t be “jerky”, ie. selfish, doesn’t tell you what kind of unselfishness to have instead. Clearly ,the left and the right don’t agree on the best kind of altruism—laying down your life to stop the spread of socialism, versus sacrificing your income to implement socialism.