I find the framing in your review is somewhat odd - I think the state of ‘deep learning theory’ is fairly impressive and that its sterility vis-a-vis frontier LLMs is a hint that we are looking in the wrong place. Early-2026 DLT is a major piece of evidence that we need more data-centric theory, precisely because sophisticated theory has had so much trouble connecting to the frontier. If we had worse theory, we would be more uncertain if the relevant complexities were to be located in the data or in the learning process.
Two analogies I have in mind that guide my thinking here:
Understanding the physics of animal neurons is necessary to understanding neuroscience. The complexity of brains exists at a ‘higher level’ than individual neurons, but understanding neurons carves out how much of that complexity can be subcellular vs. being in their larger-scale organization. In the same way, something like LLM behaviour is a product of training process, data, etc., and a good theory of learning lets us ask what part of that complexity belongs where.
Statistical physics as a formalism provides a family of techniques for analyzing physical systems with many degrees-of-freedom. Its great intellectual triumph is the discovery that some things depend on the details how how those degrees-of-freedom interact, while others do not: behaving-like-a-gas is a highly generic property, but material properties like fatigue and fracture-strength can depend quite sensitively on the specifics of the sample in question. The key thing is that we can try to figure out which properties are ‘universal’, and which are not. In DLT, I think it’s much more like a spectrum, as there are many more knobs to tune—data, gross and fine architecture, hyperparameters etc.
Of note to me is that most of the successes of DLT have been at the level of structural depth you’d expect from studying neurons as relate to brain function. E.g., the average neuron firing must activate on average exactly one neuron (lest one die or have a grand mal; comparable to the ‘edge of chaos’ in DLT). These are pretty coarse results more to do with signal processing than with structured computation. It is still illuminating to see that these coarse results hold because they validate our mental models (‘however this neural network is computing stuff, it still has to navigate some kind of signal/noise tradeoff in its activations’).
Insofar as the intelligence of LLMs is the ability to generalize, the no-free-lunch theorems tell us that this generalization has to reflect common structure of the pre-training data and the fine-tuning task (duh!). But our theory of data isn’t yet advanced enough to talk more than proleptically about that structure—e.g. a claim that fine-tuning is ‘conditioning nodes in the common sparse hierarchical latent world model’ is descriptive and substantive, but not enough that it is easily falsifiable. “Write Ruby code” and “Write Python code” are obviously more similar to each other than “design a jet turbine”, but given only a black-box loss-function for each of those 3 tasks, it’s not that clear how we could principledly determine that similarity a priori from the ‘geometry’ of the functions alone.
If by “impressive” in “the state of ‘deep learning theory’ is fairly impressive” you mean “there’s a big mathematical edifice, but not much of it is useful”, then I don’t disagree. Insofar as by “impressive”, you mean has concrete evidence of being “useful” or “at the right level of analysis”, I suspect we would strongly disagree.
(Note that I use the word “impressive” zero times in my post.)
It seems that you think deep learning theory is “impressive” because it contains sophisticated machinery that has done little in practice.
My first response is that deep learning theory in 2016 also had plenty of sophisticated machinery, but all of it turned out to be even more inapposite than current theory. Do you think that, a priori, without looking at the empirical results, you’d find the mathematical machinery of 2026 is sufficiently more sophisticated than in 2016? The additional sophistication of theory itself since 2016 was not the reason people started abandoning deep learning theory in 2016-2018; in what sense does the new work deserve additional credit for revealing that the line of work people didn’t think would work in 2016 also doesn’t work in 2026? In fact, aclassic result from 2016 was that neural networks + SGD can easily fit random labels (which never generalize) -- surely this already conclusively demonstrates that any theory of neural network generalization must make reference to the structure of the specific hypothesis neural networks are asked to learn, if not the data itself (if the earlier no-free-lunch theorems from learning theory don’t count).
My second response is that, was the mathematical machinery really necessary for the insights you point out? The toy models around superposition have arguably been more productive than all of deep learning theory, in that techniques derived from them (SAE variants) are used in production at at least one frontier lab. This is despite the much lower degree of sophistication of such work. (Even math-heavy pieces like the Comp in Sup line of work are far less mathematically tedious than the tensor program calculations, and the productive part of the superposition did not come from the Comp in Sup line of work!) Similarly, you can derive much of the scaling results for muP witha simple toy model of multi-layered linear networks (as Yang does in either the 4th or 5th tensor program papers, iirc), just like you can derive the average neuron firing must activate on average exactly 1 neuron result with hardly any mathematical model of brain activity at all.
My third response is that, if deep learning theory has indeed not been useful in practice (and indeed, might have failed so conclusively that we should look elsewhere), what makes you think a theory of data would be useful in practice?
The implicit argument you seem to give is that some theory must exist that can adequately explain the success of large deep learning models. And it’s clearly not something that a theory of the learning process or network architecture can explain, so it must be a theory of data that can explain this. But why think there’s an adequate explanation in the first place, let alone one that must be a result of the data or the learning process?
In the final paragraph, you give an explicit argument for this.
Insofar as the intelligence of LLMs is the ability to generalize, the no-free-lunch theorems tell us that this generalization has to reflect common structure of the pre-training data and the fine-tuning task (duh!).
No-free-lunch theorems also tell us that this generalization has to reflect the inductive bias of LLMs and their training procedures! After all, a rock can’t generalize from any amount of pretraining or finetuning data to the real environment in any way but the trivial one.
e.g. a claim that fine-tuning is ‘conditioning nodes in the common sparse hierarchical latent world model’ is descriptive and substantive, but not enough that it is easily falsifiable.
This really does seem like you’re including a claim about how fine-tuning works on neural networks, not just what we’re fine-tuning them on!
I do think that, insofar as deep learning theory has failed so completely that it’s ruled out the relevance of the learning mechanism to generalization, then the people who demonstrated this deserve credit. But I don’t think it has failed to nearly that extent. And even if it had, I think I’d still respond critically to the paper, because the piece doesn’t make that case either, as opposed to a case in favor of learning mechanics being a field worth investing into.
I find the framing in your review is somewhat odd - I think the state of ‘deep learning theory’ is fairly impressive and that its sterility vis-a-vis frontier LLMs is a hint that we are looking in the wrong place. Early-2026 DLT is a major piece of evidence that we need more data-centric theory, precisely because sophisticated theory has had so much trouble connecting to the frontier. If we had worse theory, we would be more uncertain if the relevant complexities were to be located in the data or in the learning process.
Two analogies I have in mind that guide my thinking here:
Understanding the physics of animal neurons is necessary to understanding neuroscience. The complexity of brains exists at a ‘higher level’ than individual neurons, but understanding neurons carves out how much of that complexity can be subcellular vs. being in their larger-scale organization. In the same way, something like LLM behaviour is a product of training process, data, etc., and a good theory of learning lets us ask what part of that complexity belongs where.
Statistical physics as a formalism provides a family of techniques for analyzing physical systems with many degrees-of-freedom. Its great intellectual triumph is the discovery that some things depend on the details how how those degrees-of-freedom interact, while others do not: behaving-like-a-gas is a highly generic property, but material properties like fatigue and fracture-strength can depend quite sensitively on the specifics of the sample in question. The key thing is that we can try to figure out which properties are ‘universal’, and which are not. In DLT, I think it’s much more like a spectrum, as there are many more knobs to tune—data, gross and fine architecture, hyperparameters etc.
Of note to me is that most of the successes of DLT have been at the level of structural depth you’d expect from studying neurons as relate to brain function. E.g., the average neuron firing must activate on average exactly one neuron (lest one die or have a grand mal; comparable to the ‘edge of chaos’ in DLT). These are pretty coarse results more to do with signal processing than with structured computation. It is still illuminating to see that these coarse results hold because they validate our mental models (‘however this neural network is computing stuff, it still has to navigate some kind of signal/noise tradeoff in its activations’).
Insofar as the intelligence of LLMs is the ability to generalize, the no-free-lunch theorems tell us that this generalization has to reflect common structure of the pre-training data and the fine-tuning task (duh!). But our theory of data isn’t yet advanced enough to talk more than proleptically about that structure—e.g. a claim that fine-tuning is ‘conditioning nodes in the common sparse hierarchical latent world model’ is descriptive and substantive, but not enough that it is easily falsifiable. “Write Ruby code” and “Write Python code” are obviously more similar to each other than “design a jet turbine”, but given only a black-box loss-function for each of those 3 tasks, it’s not that clear how we could principledly determine that similarity a priori from the ‘geometry’ of the functions alone.
Thanks for the response!
If by “impressive” in “the state of ‘deep learning theory’ is fairly impressive” you mean “there’s a big mathematical edifice, but not much of it is useful”, then I don’t disagree. Insofar as by “impressive”, you mean has concrete evidence of being “useful” or “at the right level of analysis”, I suspect we would strongly disagree.
(Note that I use the word “impressive” zero times in my post.)
It seems that you think deep learning theory is “impressive” because it contains sophisticated machinery that has done little in practice.
My first response is that deep learning theory in 2016 also had plenty of sophisticated machinery, but all of it turned out to be even more inapposite than current theory. Do you think that, a priori, without looking at the empirical results, you’d find the mathematical machinery of 2026 is sufficiently more sophisticated than in 2016? The additional sophistication of theory itself since 2016 was not the reason people started abandoning deep learning theory in 2016-2018; in what sense does the new work deserve additional credit for revealing that the line of work people didn’t think would work in 2016 also doesn’t work in 2026? In fact, a classic result from 2016 was that neural networks + SGD can easily fit random labels (which never generalize) -- surely this already conclusively demonstrates that any theory of neural network generalization must make reference to the structure of the specific hypothesis neural networks are asked to learn, if not the data itself (if the earlier no-free-lunch theorems from learning theory don’t count).
My second response is that, was the mathematical machinery really necessary for the insights you point out? The toy models around superposition have arguably been more productive than all of deep learning theory, in that techniques derived from them (SAE variants) are used in production at at least one frontier lab. This is despite the much lower degree of sophistication of such work. (Even math-heavy pieces like the Comp in Sup line of work are far less mathematically tedious than the tensor program calculations, and the productive part of the superposition did not come from the Comp in Sup line of work!) Similarly, you can derive much of the scaling results for muP with a simple toy model of multi-layered linear networks (as Yang does in either the 4th or 5th tensor program papers, iirc), just like you can derive the average neuron firing must activate on average exactly 1 neuron result with hardly any mathematical model of brain activity at all.
My third response is that, if deep learning theory has indeed not been useful in practice (and indeed, might have failed so conclusively that we should look elsewhere), what makes you think a theory of data would be useful in practice?
The implicit argument you seem to give is that some theory must exist that can adequately explain the success of large deep learning models. And it’s clearly not something that a theory of the learning process or network architecture can explain, so it must be a theory of data that can explain this. But why think there’s an adequate explanation in the first place, let alone one that must be a result of the data or the learning process?
In the final paragraph, you give an explicit argument for this.
No-free-lunch theorems also tell us that this generalization has to reflect the inductive bias of LLMs and their training procedures! After all, a rock can’t generalize from any amount of pretraining or finetuning data to the real environment in any way but the trivial one.
This really does seem like you’re including a claim about how fine-tuning works on neural networks, not just what we’re fine-tuning them on!
I do think that, insofar as deep learning theory has failed so completely that it’s ruled out the relevance of the learning mechanism to generalization, then the people who demonstrated this deserve credit. But I don’t think it has failed to nearly that extent. And even if it had, I think I’d still respond critically to the paper, because the piece doesn’t make that case either, as opposed to a case in favor of learning mechanics being a field worth investing into.