Perhaps a more down to earth example would be value conflict within an individual. Without this problem with logical uncertainty, your conflicting selves should just merge into one agent with a weighted average of their utility functions. This problem suggests that maybe you should keep those conflicting selves around until you know more logical facts.
Right. But this is also the default safety option, you don’t throw away information if you don’t have a precise understanding of its irrelevance (given that it’s not that costly to keep), and we didn’t have such understanding.
Perhaps a more down to earth example would be value conflict within an individual. Without this problem with logical uncertainty, your conflicting selves should just merge into one agent with a weighted average of their utility functions. This problem suggests that maybe you should keep those conflicting selves around until you know more logical facts.
Right. But this is also the default safety option, you don’t throw away information if you don’t have a precise understanding of its irrelevance (given that it’s not that costly to keep), and we didn’t have such understanding.