That problem— What must the world be like in order that man may know it?— was not, however, created by this essay. On the contrary, it is as old as science itself, and it remains unanswered. But it need not be answered in this place.
At this point I lose patience. Kuhn is no longer being thought-provoking, he’s being disingenuous. IT’S BECAUSE THERE’S AN OBJECTIVE REALITY, TOM.
I haven’t read Kuhn and I don’t know whether I’m interpreting em correctly, but to me it seems not that simple at all.
Saying there is an objective reality doesn’t explain why this reality is comprehensible. In statistical learning theory there are various analyses of what mathematical conditions must hold for it to be possible to learn a model from observations (i.e. so that you can avoid the no-free-lunch theorems) and how difficult it is to learn it, and when you add computational complexity considerations into the mix it becomes even more complicated. Our understanding of these questions is far from complete.
In particular, our ability to understand physics seems to rely on the hierarchical nature of physical phenomena. You can discover classical mechanics without knowing anything about molecules or quantum physics, you can discover atomic and molecular physics while knowing little about nuclear physics, and you can discover nuclear and particle physics without understanding quantum gravity (i.e. what happens to spacetime on the Planck scale). If the universe was s.t. it is impossible to compute the trajectory of a tennis ball without string theory, we might have never discovered any physics.
From looking at Conway’s Game of Life, my intuition is that if a universe can support non-ontologically-fundamental Turing machines (I’m invoking anthropic reasoning), then it’s likely to have phenomena analyzable at multiple hierarchical levels (beyond the looser requirement of being simple/compressible).
Basically, if a universe allows any reductionistic understanding at all (that’s what I mean by calling the Turing Machine “non-ontologically-fundamental”), then the reductionist structure is probably a multi-layered one. Either zero reduction layers or lots, but not exactly one layer.
Our universe has a simplifying structure: it abstracts well, implying a particular kind of modularity.
Goal-oriented systems in our universe tend to evolve a modular structure which reflects the structure of the universe.
One major corollary of these two ideas is that goal-oriented systems will tend to evolve similar modular structures, reflecting the relevant parts of their environment. Systems to which this applies include organisms, machine learning algorithms, and the learning performed by the human brain. In particular, this suggests that biological systems and trained deep learning systems are likely to have modular, human-interpretable internal structure. (At least, interpretable by humans familiar with the environment in which the organism/ML system evolved.)
This post talks about some of the evidence behind this model: biological systems are indeed quite modular, and simulated evolution experiments find that circuits do indeed evolve modular structure reflecting the modular structure of environmental variations. The companion post reviews the rest of the book, which makes the case that the internals of biological systems are indeed quite interpretable.
Going forward, this view is in need of a more formal and general model, ideally one which would let us empirically test key predictions—e.g. check the extent to which different systems learn similar features, or whether learned features in neural nets satisfy the expected abstraction conditions, as well as tell us how to look for environment-reflecting structures in evolved/trained systems.
If the universe was s.t. it is impossible to compute the trajectory of a tennis ball without string theory, we might have never discovered any physics.
Makes one wonder, what things we have not discovered, because they are in a such way?
I believe the term “chaotic” refers to those things. E.g. for an airplane’s lift, there are higher-level-than-particle-modeling physical principles you can explain it with, but for a 1-month weather forecast you have to go down to particle modeling, or close.
Crossposted from SSC comments section
I haven’t read Kuhn and I don’t know whether I’m interpreting em correctly, but to me it seems not that simple at all.
Saying there is an objective reality doesn’t explain why this reality is comprehensible. In statistical learning theory there are various analyses of what mathematical conditions must hold for it to be possible to learn a model from observations (i.e. so that you can avoid the no-free-lunch theorems) and how difficult it is to learn it, and when you add computational complexity considerations into the mix it becomes even more complicated. Our understanding of these questions is far from complete.
In particular, our ability to understand physics seems to rely on the hierarchical nature of physical phenomena. You can discover classical mechanics without knowing anything about molecules or quantum physics, you can discover atomic and molecular physics while knowing little about nuclear physics, and you can discover nuclear and particle physics without understanding quantum gravity (i.e. what happens to spacetime on the Planck scale). If the universe was s.t. it is impossible to compute the trajectory of a tennis ball without string theory, we might have never discovered any physics.
From looking at Conway’s Game of Life, my intuition is that if a universe can support non-ontologically-fundamental Turing machines (I’m invoking anthropic reasoning), then it’s likely to have phenomena analyzable at multiple hierarchical levels (beyond the looser requirement of being simple/compressible).
Basically, if a universe allows any reductionistic understanding at all (that’s what I mean by calling the Turing Machine “non-ontologically-fundamental”), then the reductionist structure is probably a multi-layered one. Either zero reduction layers or lots, but not exactly one layer.
While re-reading things for the 2019 Review, I noticed this is followed up in johnswentworth’s more recent self-review on the Evolution of Modularity:
Makes one wonder, what things we have not discovered, because they are in a such way?
I believe the term “chaotic” refers to those things. E.g. for an airplane’s lift, there are higher-level-than-particle-modeling physical principles you can explain it with, but for a 1-month weather forecast you have to go down to particle modeling, or close.