What is it you think may not be possible? I imagine that proofs for being self-reflectively coherent, actually optimizing its goal, stable thru rewrites and ontological revolutions, and so on are totally doable. The only fuzzyness is whether CEV is the best goal, or even a friendly one. I think the “provably friendly” thing is about having proved all those nice strong properties plus giving it the best goal system we have, (which is currently CEV).
What is it you think may not be possible? I imagine that proofs for being self-reflectively coherent, actually optimizing its goal, stable thru rewrites and ontological revolutions, and so on are totally doable. The only fuzzyness is whether CEV is the best goal, or even a friendly one. I think the “provably friendly” thing is about having proved all those nice strong properties plus giving it the best goal system we have, (which is currently CEV).
In the Godel sense. For example, it might be independent in whatever mathematical framework EY would find it reasonable to start from.