TurnTrout comments on Training Process Transparency through Gradient Interpretability: Early experiments on toy language models

[TurnTrout]

Focusing on language models, we note that models exhibit “consistent developmental stages,” at first behaving similarly to $n$-gram models and later exhibiting linguistic patterns.

I wrote a shortform comment which seems relevant:

Are there convergently-ordered developmental milestones for AI? I suspect there may be convergent orderings in which AI capabilities emerge. For example, it seems that LMs develop syntax before semantics, but maybe there’s an even more detailed ordering relative to a fixed dataset. And in embodied tasks with spatial navigation and recurrent memory, there may be an order in which enduring spatial awareness emerges (i.e. “object permanence”).