You could do that, but my initial inspiration was just corrigibility—the AI is indifferent to the update in values. Maybe there could be a subroutine (or another agent) that gathers the information for the update, while the AI just doesn’t care about that.
You could do that, but my initial inspiration was just corrigibility—the AI is indifferent to the update in values. Maybe there could be a subroutine (or another agent) that gathers the information for the update, while the AI just doesn’t care about that.
See “pre-corriged agents” http://lesswrong.com/lw/luo/resource_gathering_and_precorriged_agents/ for setups where the AI isn’t indifferent to the process, but is indifferent to the direction of the update.