Now, given that you’re already talking about instrumental goals “trying not to step on each other’s toes”, what else would they need to deserve the name of “subagents”?
Note that the instrumental goal is importantly distinct from the subagent which pursues that instrumental goal. I think a big part of the insight in this post is to say “corrigibility is a property of instrumental goals, separate from the subagents which pursue those goals”; we can study the goals (i.e. problem factorization) rather than the subagents in order to understand corrigibility.
This is great.
Now, given that you’re already talking about instrumental goals “trying not to step on each other’s toes”, what else would they need to deserve the name of “subagents”?
Note that the instrumental goal is importantly distinct from the subagent which pursues that instrumental goal. I think a big part of the insight in this post is to say “corrigibility is a property of instrumental goals, separate from the subagents which pursue those goals”; we can study the goals (i.e. problem factorization) rather than the subagents in order to understand corrigibility.