Not quite. It’s more like Ron seeing what a more mature version of himself would want, but Dumbledore’s pushing 200 and famously wise; he’s not going to get much more mature following the path he’s taken. You could argue that his worldview isn’t self-consistent and that a smarter or less self-deluding version of him would pick up on that, but that seems like it bakes in a conclusion.
I haven’t exactly formalized this, but I have the intuition that CEV would be doing more work in aggregating extrapolated values than in extrapolating values in the first place. We can’t just have it wave a wand (har) and rid ourselves of heuristics and biases to find our true values; too much of human value is wrapped up in those same heuristics and biases, and from an internal viewpoint none of them are any “truer” than any others. We can envision an aggregation process that plays different people’s heuristics and biases against each other in some way to find a least-worst kernel of value; but to do that, it needs those data points.
Not quite. It’s more like Ron seeing what a more mature version of himself would want, but Dumbledore’s pushing 200 and famously wise; he’s not going to get much more mature following the path he’s taken. You could argue that his worldview isn’t self-consistent and that a smarter or less self-deluding version of him would pick up on that, but that seems like it bakes in a conclusion.
I haven’t exactly formalized this, but I have the intuition that CEV would be doing more work in aggregating extrapolated values than in extrapolating values in the first place. We can’t just have it wave a wand (har) and rid ourselves of heuristics and biases to find our true values; too much of human value is wrapped up in those same heuristics and biases, and from an internal viewpoint none of them are any “truer” than any others. We can envision an aggregation process that plays different people’s heuristics and biases against each other in some way to find a least-worst kernel of value; but to do that, it needs those data points.