Is this necessarily true? It seems like this describes what Christiano calls “delegation” in his paper, but wouldn’t apply to IDA schemes with other capability amplification methods (such as the other examples in the appendix of “Capability Amplification”).
“Depth” only applies to the canonical tree-based implementation of IDA. If you slot in other amplification or distillation procedures, then you won’t necessarily have “depths” any more. You’ll still have recursion, and that recursion will lead to more and more capability. Where it ends up depends on your initial agent and how good the amplification and distillation procedures are.
Huh, I thought that all amplification/distillation procedures were intended as a way to approximate HCH, which is itself a tree. Can you not meaningfully discuss “this amplification procedure is like an n-depth approximation of HCH at step x”, for any amplification procedure?
For example, the internal structure of the distilled agent described in Christiano’s paper is unlikely to look anything like a tree. However, my (potentially incorrect?) impression is that the agent’s capabilities at step x are identical to an HCH tree of depth x if the underlying learning system is arbitrarily capable.
It’s possible that I’m not understanding the difference between “depth”, “tree-based” and “recursion” in this context
Can you not meaningfully discuss “this amplification procedure is like an n-depth approximation of HCH at step x”, for any amplification procedure?
No, you can’t. E.g. If your amplification procedure only allows you to ask a single subagent a single question, that will approximate a linear HCH instead of a tree-based HCH. If your amplification procedure doesn’t invoke subagents at all, but instead provides more and more facts to the agent, it doesn’t look anything like HCH. The canonical implementations of iterated amplification are trying to approximate HCH though.
For example, the internal structure of the distilled agent described in Christiano’s paper is unlikely to look anything like a tree. However, my (potentially incorrect?) impression is that the agent’s capabilities at step x are identical to an HCH tree of depth x if the underlying learning system is arbitrarily capable.
Is this necessarily true? It seems like this describes what Christiano calls “delegation” in his paper, but wouldn’t apply to IDA schemes with other capability amplification methods (such as the other examples in the appendix of “Capability Amplification”).
“Depth” only applies to the canonical tree-based implementation of IDA. If you slot in other amplification or distillation procedures, then you won’t necessarily have “depths” any more. You’ll still have recursion, and that recursion will lead to more and more capability. Where it ends up depends on your initial agent and how good the amplification and distillation procedures are.
Huh, I thought that all amplification/distillation procedures were intended as a way to approximate HCH, which is itself a tree. Can you not meaningfully discuss “this amplification procedure is like an n-depth approximation of HCH at step x”, for any amplification procedure?
For example, the internal structure of the distilled agent described in Christiano’s paper is unlikely to look anything like a tree. However, my (potentially incorrect?) impression is that the agent’s capabilities at step x are identical to an HCH tree of depth x if the underlying learning system is arbitrarily capable.
It’s possible that I’m not understanding the difference between “depth”, “tree-based” and “recursion” in this context
No, you can’t. E.g. If your amplification procedure only allows you to ask a single subagent a single question, that will approximate a linear HCH instead of a tree-based HCH. If your amplification procedure doesn’t invoke subagents at all, but instead provides more and more facts to the agent, it doesn’t look anything like HCH. The canonical implementations of iterated amplification are trying to approximate HCH though.
That sounds right to me.