I’m not sure if “models naturally do in-context reasoning in their middle layers” is true and to what degree
I’m not sure either — it seems plausible to me. Much of my comment is a collection of hunches, guesses, and speculations along with supporting evidence that caused me to guess them.
I’m not sure either — it seems plausible to me. Much of my comment is a collection of hunches, guesses, and speculations along with supporting evidence that caused me to guess them.