Seems as if there’s some sleight-of-hand going on here Yes, we can show that any policy that is invulnerable to dutch-booking is equivalent to optimizing some utility function. But you’ve also shown earlier that “equivalent to optimizing some utility function” is a nearly-vacuous concept. There are plenty of un-dutch-bookable policies which still don’t end up paving the universe in utilitronium, for ANY utility function.
Furthermore, I find it easy to imagine human-like value systems which are in fact dutch-bookable; e.g., “I like to play peekaboo with babies” is dutch-bookable between “eyes covered” and “eyes uncovered”. So the generalization at the outset of this chapter seems over-broad.
I feel like you are trying to critique something I wrote, but I’m not sure what? Could you be a bit more specific about what you think I think that you disagree with?
(In particular, the first paragraph sounds like a statement that I myself would make, so I’m not sure how it is a critique.)
Hmm. I went back and reread you carefully, and I cannot find the part where you said the thing that I was “responding” to above. So I think I’m probably actually responding to my poor model of what you would say, not to what you actually did say. Sorry. I’ll leave my above comment but strike out the parts where it refers to what “you” say.
Seems as if there’s some sleight-of-hand going on here Yes, we can show that any policy that is invulnerable to dutch-booking is equivalent to optimizing some utility function. But you’ve also shown earlier that “equivalent to optimizing some utility function” is a nearly-vacuous concept. There are plenty of un-dutch-bookable policies which still don’t end up paving the universe in utilitronium, for ANY utility function.
Furthermore, I find it easy to imagine human-like value systems which are in fact dutch-bookable; e.g., “I like to play peekaboo with babies” is dutch-bookable between “eyes covered” and “eyes uncovered”. So the generalization at the outset of this chapter seems over-broad.
I feel like you are trying to critique something I wrote, but I’m not sure what? Could you be a bit more specific about what you think I think that you disagree with?
(In particular, the first paragraph sounds like a statement that I myself would make, so I’m not sure how it is a critique.)
Hmm. I went back and reread you carefully, and I cannot find the part where you said the thing that I was “responding” to above. So I think I’m probably actually responding to my poor model of what you would say, not to what you actually did say. Sorry. I’ll leave my above comment but strike out the parts where it refers to what “you” say.