I won’t get around to posting that today, but I’ll just add that I know that the intent of CEV is to solve the problems I’m complaining about. I know there are bullet points in the CEV document that say, “Renormalizing the dynamic”, “Caring about volition,” and, “Avoid hijacking the destiny of humankind.”
But I also know that the CEV document says,
Since the output of the CEV is one of the major forces shaping the future, I’m still pondering the order-of-evaluation problem to prevent this from becoming an infinite recursion.
and
It may be hard to get CEV right—come up with an AI dynamic such that our volition, as defined, is what we intuitively want. The technical challenge may be too hard; the problems I’m still working out may be impossible or ill-defined. I don’t intend to trust any design until I see that it works, and only to the extent I see that it works. Intentions are not always realized.
I think there is what you could call an order-of-execution problem, and I think there’s a problem with things being ill-defined, and I think the desired outcome is logically impossible. I could be wrong. But since Eliezer worries that this could be the case, I find it strange that Eliezer’s bulldogs are so sure that there are no such problems, and so quick to shoot down discussion of them.
I won’t get around to posting that today, but I’ll just add that I know that the intent of CEV is to solve the problems I’m complaining about. I know there are bullet points in the CEV document that say, “Renormalizing the dynamic”, “Caring about volition,” and, “Avoid hijacking the destiny of humankind.”
But I also know that the CEV document says,
and
I think there is what you could call an order-of-execution problem, and I think there’s a problem with things being ill-defined, and I think the desired outcome is logically impossible. I could be wrong. But since Eliezer worries that this could be the case, I find it strange that Eliezer’s bulldogs are so sure that there are no such problems, and so quick to shoot down discussion of them.