The point is that it’s not, but making it so is a design goal of the paper.
Example: Mario immediately jumping into a pit at level 2. According to the learned utility function of the system, it’s a good idea. According to ours, it’s not.
Just as with optimizing smiling faces. But while that one was purely a thought experiment, this paper presents a practical, experimentally testable benchmark for utility function learning, and, by the way, shows a not-yet-perfect but working solution for it. (After all, Mario’s Flying Goomba Kick of High Munchkinry definitely satisfies our utility functions.)
The point is that it’s not, but making it so is a design goal of the paper.
Example: Mario immediately jumping into a pit at level 2. According to the learned utility function of the system, it’s a good idea. According to ours, it’s not.
Just as with optimizing smiling faces. But while that one was purely a thought experiment, this paper presents a practical, experimentally testable benchmark for utility function learning, and, by the way, shows a not-yet-perfect but working solution for it. (After all, Mario’s Flying Goomba Kick of High Munchkinry definitely satisfies our utility functions.)