Humans tend to do this less in words; it’s socially embarrassing to babble nonsense, and humans have a private internal chain-of-thought in which they can hide their incoherence.
My internal monologue often includes things such as chanting “math math math” when I’m trying to think about math, which seems to invoke a “thinking about math” mode. Plausibly, LLMs associate certain forms of thinking with specific tokens, and using those tokens pushes them towards that mode, which they can learn to deliberately invoke in a similar manner.
Ha! I thought I was the only one. Mine proceeds to think “f is a function mapping from A to B” which I guess might work alright because [everything is a function](https://arxiv.org/pdf/1612.09375)
My internal monologue often includes things such as chanting “math math math” when I’m trying to think about math, which seems to invoke a “thinking about math” mode. Plausibly, LLMs associate certain forms of thinking with specific tokens, and using those tokens pushes them towards that mode, which they can learn to deliberately invoke in a similar manner.
I actually say to myself “alright, someone whose good at math come solve this” which I guess kind of puts my internal monologue into math mode.
Ha! I thought I was the only one. Mine proceeds to think “f is a function mapping from A to B” which I guess might work alright because [everything is a function](https://arxiv.org/pdf/1612.09375)