What do we see if we apply interpretability tools to the filler tokens or repeats of the problem?
I would be especially interested in how this evolves through training, perhaps by training a more accessible model to do math / code classification with many filler tokens.
Overall, these results demonstrate a case where LLMs can do (very basic) meta-cognition without CoT.
Can you clarify what you mean by meta-cognition? I’m intuiting that these LLMs are using the extra embeddings afforded by appended tokens to do more parallel ops, which does not sound like meta-cognition to me.
Interesting!
I would be especially interested in how this evolves through training, perhaps by training a more accessible model to do math / code classification with many filler tokens.
Can you clarify what you mean by meta-cognition? I’m intuiting that these LLMs are using the extra embeddings afforded by appended tokens to do more parallel ops, which does not sound like meta-cognition to me.
It requires deciding what to think about at a given token, probably in a somewhat flexible way.