Another argument in favor of the introspection approach is that attention/the KV cache seems like a natural place to interpret computations that unfold sequentially across multiple layers in an LLM. This might be a limitation of meta-model approaches, since they typically only operate on a small number of activation vectors rather than the model’s full evolving computational state.
This all makes a lot of sense to me!
Another argument in favor of the introspection approach is that attention/the KV cache seems like a natural place to interpret computations that unfold sequentially across multiple layers in an LLM. This might be a limitation of meta-model approaches, since they typically only operate on a small number of activation vectors rather than the model’s full evolving computational state.