Just for understanding: What is the relation between V and CEV?
If you’re saying that they are different concepts and CEV is just not what humans want, then I’d shrug and say “let’s optimize for CEV anyway, so that basically V is CEV”. (You could perhaps make a philosophical discussion out of that, and I would guess my opinion would win, though I don’t know yet how and the argument would probably be brain-meltingly complicated. I haven’t understood Yudkowsky’s writings on metaethics (yet).)
Or are you saying that V and CEV are basically the same, and that CEV doesn’t exist, isn’t well-defined, or is some weird phrasing of a value that you cannot sensibly apply goodhard’s law to it?
(I still don’t see what people want to say with “we don’t have true values”. Obviously we value some things, and obviously that depends on our environment, circumstances, etc., but that shouldn’t stop us. Not that I think you say that this stops us and value learning us useless, but I don’t understand what you want to say with it.)
Just for understanding: What is the relation between V and CEV?
If you’re saying that they are different concepts and CEV is just not what humans want, then I’d shrug and say “let’s optimize for CEV anyway, so that basically V is CEV”. (You could perhaps make a philosophical discussion out of that, and I would guess my opinion would win, though I don’t know yet how and the argument would probably be brain-meltingly complicated. I haven’t understood Yudkowsky’s writings on metaethics (yet).)
Or are you saying that V and CEV are basically the same, and that CEV doesn’t exist, isn’t well-defined, or is some weird phrasing of a value that you cannot sensibly apply goodhard’s law to it?
(I still don’t see what people want to say with “we don’t have true values”. Obviously we value some things, and obviously that depends on our environment, circumstances, etc., but that shouldn’t stop us. Not that I think you say that this stops us and value learning us useless, but I don’t understand what you want to say with it.)
The most important detail is that CEV is not a utility function. It is mostly poetry.
We have values in the plain language sense of the word. But we have no singular utility fuction, any deviation from which is bad.
V is “the direction all the train tracks go” from The Tails Coming Apart as Metaphor for Life.