This discussion is an excellent instance of a pattern I see often, which I should write a post on at some point.
Person 1: Seems like the only way to <do useful thing> is carve up the problem along low-dimensional interfaces.
Person 2: But in practice, when people try to do that, they pick carvings which don’t work, and then try to shoehorn things into their carvings, and then everything is terrible.
Resolution (which most such discussions don’t actually reach, good job guys): You Don’t Get To Choose The Ontology. The low-dimensional interfaces are determined by the problem domain; if at some point someone “picks” a carving, then they’ve shot themselves in the foot. Also, it takes effort to discover the carvings of a problem domain.
Another mildly-hot-take example: the bitter lesson. The way I view the bitter lesson is:
A bunch of AI researchers tried to hand-code ontologies. They mostly picked how to carve things, and didn’t do the work to discover natural carvings.
That failed. Eventually some folks figured out how to do brute-force optimization in such a way that the optimized systems would “discover” natural ontologies for themselves, but not in a way where the natural ontologies are made externally-visible to humans (alas).
The ideal path would be to figure out how to make the natural ontological divisions visible to humans.
(I think most people today interpret the bitter lesson as something like “brute force scaling beats clever design”, whereas I think the original essay reads closer to my interpretation above, and I think the history of ML also better supports my interpretation above.)
I revisited the Bitter Lesson essay to see if it indeed matches your interpretation. I think it basically does, up to some uncertainty about whether “ontologies” and “approximations” are quite the same kind of thing.
The second general point to be learned from the bitter lesson is that the actual contents of minds are tremendously, irredeemably complex; we should stop trying to find simple ways to think about the contents of minds, such as simple ways to think about space, objects, multiple agents, or symmetries. All these are part of the arbitrary, intrinsically-complex, outside world. They are not what should be built in, as their complexity is endless; instead we should build in only the meta-methods that can find and capture this arbitrary complexity. Essential to these methods is that they can find good approximations, but the search for them should be by our methods, not by us. We want AI agents that can discover like we can, not which contain what we have discovered. Building in our discoveries only makes it harder to see how the discovering process can be done.
This discussion is an excellent instance of a pattern I see often, which I should write a post on at some point.
Person 1: Seems like the only way to <do useful thing> is carve up the problem along low-dimensional interfaces.
Person 2: But in practice, when people try to do that, they pick carvings which don’t work, and then try to shoehorn things into their carvings, and then everything is terrible.
Resolution (which most such discussions don’t actually reach, good job guys): You Don’t Get To Choose The Ontology. The low-dimensional interfaces are determined by the problem domain; if at some point someone “picks” a carving, then they’ve shot themselves in the foot. Also, it takes effort to discover the carvings of a problem domain.
Another mildly-hot-take example: the bitter lesson. The way I view the bitter lesson is:
A bunch of AI researchers tried to hand-code ontologies. They mostly picked how to carve things, and didn’t do the work to discover natural carvings.
That failed. Eventually some folks figured out how to do brute-force optimization in such a way that the optimized systems would “discover” natural ontologies for themselves, but not in a way where the natural ontologies are made externally-visible to humans (alas).
The ideal path would be to figure out how to make the natural ontological divisions visible to humans.
(I think most people today interpret the bitter lesson as something like “brute force scaling beats clever design”, whereas I think the original essay reads closer to my interpretation above, and I think the history of ML also better supports my interpretation above.)
For another class of examples, I’d point to the whole “high modernism” thing.
I revisited the Bitter Lesson essay to see if it indeed matches your interpretation. I think it basically does, up to some uncertainty about whether “ontologies” and “approximations” are quite the same kind of thing.