If the set of good things seems like it’s of measure zero, maybe we should choose a better measure.
If what you want is dynamics, you can have dynamics—just program them in. Of course, they’re still equivalent to some fixed goal, but so what? The universe is equivalent to the field equations, but it still moves.
Don’t get in an AI’s van just because it’s uncertain about its goals. If human-like fixed goals are supposed to be of measure zero, building an AI that’s uncertain about its own goals raises your chances of getting something basically human-like from zero to zero.
If the set of good things seems like it’s of measure zero, maybe we should choose a better measure.
This seems to be the exact problem of AI alignment in the first place. We are currently unable to construct a rigorous measure(in the space of possible values) in which the set of good things (in the cases where said values take over the world) is not of vanishingly small measure.
If the set of good things seems like it’s of measure zero, maybe we should choose a better measure.
If what you want is dynamics, you can have dynamics—just program them in. Of course, they’re still equivalent to some fixed goal, but so what? The universe is equivalent to the field equations, but it still moves.
Don’t get in an AI’s van just because it’s uncertain about its goals. If human-like fixed goals are supposed to be of measure zero, building an AI that’s uncertain about its own goals raises your chances of getting something basically human-like from zero to zero.
This seems to be the exact problem of AI alignment in the first place. We are currently unable to construct a rigorous measure(in the space of possible values) in which the set of good things (in the cases where said values take over the world) is not of vanishingly small measure.