Continual learning would force what is essentially neuralese more inevitably, even very tame kinds that merely go beyond literal directories of notes in Markdown and effectively extend context length with data that acts like KV cache (in its role during forward passes), but gets computed through true recurrence (with unbounded rather than strictly capped serial depth of computation, proportional to context length; so it’s not about hybrid attention found in today’s models).
But once learning can happen in fast invention-learning-invention feedback loops (which is the ambitious sense of continual learning, or the kind of thing strong RSI enables, unlike prosaic RSI), AIs can develop a lot of technical/cultural content with high serial depth, things like novel concepts and languages. At that point even CoT without continual learning (written in standard tokens) doesn’t help with translation, and activation vectors stop making sense.
Continual learning would force what is essentially neuralese more inevitably, even very tame kinds that merely go beyond literal directories of notes in Markdown and effectively extend context length with data that acts like KV cache (in its role during forward passes), but gets computed through true recurrence (with unbounded rather than strictly capped serial depth of computation, proportional to context length; so it’s not about hybrid attention found in today’s models).
But once learning can happen in fast invention-learning-invention feedback loops (which is the ambitious sense of continual learning, or the kind of thing strong RSI enables, unlike prosaic RSI), AIs can develop a lot of technical/cultural content with high serial depth, things like novel concepts and languages. At that point even CoT without continual learning (written in standard tokens) doesn’t help with translation, and activation vectors stop making sense.
Not sure you need continual learning for that …
Reasoning Models Sometimes Output Illegible Chains of Thought, Oct 2025
https://arxiv.org/pdf/2510.27338
I’m not sure that’s true …
Unsupervised decoding of encoded reasoning using language model interpretability, Dec 2025
https://arxiv.org/abs/2512.01222v1
but I don’t think the experiment has been done for RL?