This post is bothering an itch I’ve been having for over a year.
The entire problem of AI control stems from how we treat LLMs. We look at them as the AI. Maybe a small shell of code for letting it interface with some computer system, but the LLM for all intents and purposes is what we call ‘AI’. We draw an equivalence between the two and try to solve the problem of controlling super intelligence under a constraint.
Correct me if I’m wrong but I feel that this post and paper are jabbing in a similar direction. We need a cognitive model for AI but if we work under constraint this implies we need to develop a model of the model. If we break the constraint then we can build the cognitive model from first principles, utilizing the LLM as a language processing subsystem rather than as the entire AI.
This post is bothering an itch I’ve been having for over a year.
The entire problem of AI control stems from how we treat LLMs. We look at them as the AI. Maybe a small shell of code for letting it interface with some computer system, but the LLM for all intents and purposes is what we call ‘AI’. We draw an equivalence between the two and try to solve the problem of controlling super intelligence under a constraint.
Correct me if I’m wrong but I feel that this post and paper are jabbing in a similar direction. We need a cognitive model for AI but if we work under constraint this implies we need to develop a model of the model. If we break the constraint then we can build the cognitive model from first principles, utilizing the LLM as a language processing subsystem rather than as the entire AI.
The field even seems to be crawling towards this idea. Techniques like Inference Time Intervention(https://arxiv.org/abs/2306.03341), and FUDGE(https://aclanthology.org/2021.naacl-main.276.pdf) work at runtime. The bridge between token generation modification at runtime and a feedback system with known failure signals like entropy (https://arxiv.org/pdf/2510.21623,https://arxiv.org/html/2512.12381v1) seems a lot smaller than the current hurdle of controlling LLMs as they are.