This is a surprisingly brilliant idea, which should definitely have a name. For humans, part of the benefit of this is that it appeals to risk aversion, so people wouldn’t want to completely write off one of the scenarios. It also makes it so complex to analyze, that many people would simply fall back to “doing the right thing” naturally. I could definitely foresee benefits by, for example, randomizing whether members of a team are going to be judged based on individual performance or team performance.
I’m not totally sure it would work as well for AIs, which would naturally be trying to optimize in the gaps much more than a human would, and would potentially be less risk averse than a human.
This is a surprisingly brilliant idea, which should definitely have a name. For humans, part of the benefit of this is that it appeals to risk aversion, so people wouldn’t want to completely write off one of the scenarios. It also makes it so complex to analyze, that many people would simply fall back to “doing the right thing” naturally. I could definitely foresee benefits by, for example, randomizing whether members of a team are going to be judged based on individual performance or team performance.
I’m not totally sure it would work as well for AIs, which would naturally be trying to optimize in the gaps much more than a human would, and would potentially be less risk averse than a human.
Let’s name it:
Hidden Agency Solution
Caleiro Agency Conjecture
Principal agent blind spot
Cal-agency
Stochastic principal agency solution.
(wow, this naming thing is hard and awfully awkward, whether I’m optimizing for mnemonics or for fame—is there any other thing to optimize for here?)
Fuzzy metrics?
Doesn’t refer to Principal Agency.