This idea seems to require (basically) a major revolution in or even a complete solution to program induction
Eh, I think any nontrivial technical project can be made to sound like an incredibly significant and therefore dauntingly impossible achievement, if you pick the right field to view it from. But what matters is the actual approach you’re using, and how challenging the technical problems are from the perspective of the easiest field in which they could be represented.
Some examples:
Consider various geometry problems, e. g., one of those. If you use the tools of analytic geometry, you’d end up having to solve a complicated system of nonlinear equations. If you use synthetic geometry instead, the way to resolve them might consist of applying a well-known theorem and a few simple reasoning steps, so simple you can do it in your head.
Consider the problem of moving fast. Before the invention of the car, the problem of moving at 120 km/h could’ve been cast as “a major revolution in horse-breeding and genetic engineering”. But the actual approach taken did not route through horses or biology at all. It achieved the end result through a different pathway, in which the technical problems were dramatically easier.
Consider AI. Prior to Deep Learning, there was a throve of symbolic approaches to it; and even before that, hand-written GOFAIs. The technical problem of “achieve DL-level performance using symbolic/GOFAI tools” is dramatically harder than “achieve DL-level performance”, unqualified. And yet, the latter can be technically described as a revolution in the relevant fields.
Consider various other modeling problems, e. g., weather prediction, volcano modeling, materials-science modeling, quantitative trading. Any advancement in general modeling techniques would revolutionize all of those. But should that technical problem really be framed in the daunting terms of “come up with a revolutionary stock-trading algorithm”?
To generalize: Suppose there’s some field A which is optimizing for X. Improving on X using the tools of A would necessarily require you to beat a market that is efficient-relative-to-you. Experts in A already know the tools of A in and out, and how to use them to maximize X. Even if you can beat them, it would only be an incremental improvement. A slightly better solver for systems of nonlinear equations, a slightly faster horse, a slightly better trading algorithm.
The way to actually massively improve on X is to ignore the extant tools of A entirely, and try to develop new tools for optimizing X by using some other field B. On the outside view, this is necessarily a high-risk proposition, since B might end up entirely unhelpful; but it’s also high-reward, since it might allow you to actually “beat the market”. And if you succeed, the actual technical problems you’ll end up solving will be massively easier than the problems you’d need to solve to achieve the same performance using A’s tools.
Bringing it back around: This agenda may or may not be viewed as aiming to revolutionize program induction, but I’m not setting out to take the extant program-induction tools and try to cobble together something revolutionary using them. The idea is to use an entirely different line of theory (agent foundations, natural abstractions, information theory, recent DL advances) to achieve that end result.
All of your examples seem to involve applying some radically different approach to overturn a status quo, which is certainly possible. But I am unsure of where you suggest a radically different approach.
Many program induction researchers are cognitive scientists, and have suggested compositional / hierarchical representations for a decades [1] and actively in recent years [2] including even a paper I am on: https://arxiv.org/html/2504.20628v1 So, basically I don’t expect there to be alpha from this observation in itself.
I am, of course, an information theory enthusiast (particularly AIT), but this is an old field and it is not clear to me what specific (recent?) results you hope to leverage? Or why these would results would have been overlooked?
You point to progress on natural abstractions, but to me this only indicates that it should in principle perhaps be possible to come up with some kind of interpretable world model. Has NA research actually produced practical algorithms or methods, or is it in reach of doing so?
Agent foundations as a category is too broad for me to understand what mathematics you are hoping to leverage. And unfortunately, I do not think the AF community has made a large number of significant breakthoughs—reflective oracles, logical induction, and perhaps incomplete models/IB are the main examples left in my mind (you categorized NAH separately), but do not see how they are relevant here.
Basically, you have expressed various hopes, and perhaps some of them are promising (and I will read about them throughout this sequence) but from your comment alone, your alpha over the program induction community (or for instance even the Cyc project) is not legible (to me) - not even as a high-level summary of a technical program. So, I am left hopeful that you will make progress here, but your highly ambitious goal still seems like a distant point on a nearly blank map to me.
[1] Brenden M Lake and Steven T Piantadosi. People infer recursive visual concepts from just a few examples. Computational Brain & Behavior, 3(1):54–65, 2020 [2] Jerry A Fodor. The language of thought, volume 5. Harvard university press, 1975.
Nice, that’s the sort of poking-of-holes I was looking for.
your alpha over the program induction community (or for instance even the Cyc project) is not legible (to me)
That’s a good thinking prompt. What is the full set of reasons I’m optimistic about this, in legible terms?
… Hm, but perhaps outlining what you think your edge is in public is not a great idea. I’ll answer in PMs tomorrow. (To ensure future readers get some information about how convincing my reasons are, though, it’d be neat if you posted your impressions afterwards as a response to this comment.)
Eh, I think any nontrivial technical project can be made to sound like an incredibly significant and therefore dauntingly impossible achievement, if you pick the right field to view it from. But what matters is the actual approach you’re using, and how challenging the technical problems are from the perspective of the easiest field in which they could be represented.
Some examples:
Consider various geometry problems, e. g., one of those. If you use the tools of analytic geometry, you’d end up having to solve a complicated system of nonlinear equations. If you use synthetic geometry instead, the way to resolve them might consist of applying a well-known theorem and a few simple reasoning steps, so simple you can do it in your head.
Consider the problem of moving fast. Before the invention of the car, the problem of moving at 120 km/h could’ve been cast as “a major revolution in horse-breeding and genetic engineering”. But the actual approach taken did not route through horses or biology at all. It achieved the end result through a different pathway, in which the technical problems were dramatically easier.
Consider AI. Prior to Deep Learning, there was a throve of symbolic approaches to it; and even before that, hand-written GOFAIs. The technical problem of “achieve DL-level performance using symbolic/GOFAI tools” is dramatically harder than “achieve DL-level performance”, unqualified. And yet, the latter can be technically described as a revolution in the relevant fields.
Consider various other modeling problems, e. g., weather prediction, volcano modeling, materials-science modeling, quantitative trading. Any advancement in general modeling techniques would revolutionize all of those. But should that technical problem really be framed in the daunting terms of “come up with a revolutionary stock-trading algorithm”?
To generalize: Suppose there’s some field A which is optimizing for X. Improving on X using the tools of A would necessarily require you to beat a market that is efficient-relative-to-you. Experts in A already know the tools of A in and out, and how to use them to maximize X. Even if you can beat them, it would only be an incremental improvement. A slightly better solver for systems of nonlinear equations, a slightly faster horse, a slightly better trading algorithm.
The way to actually massively improve on X is to ignore the extant tools of A entirely, and try to develop new tools for optimizing X by using some other field B. On the outside view, this is necessarily a high-risk proposition, since B might end up entirely unhelpful; but it’s also high-reward, since it might allow you to actually “beat the market”. And if you succeed, the actual technical problems you’ll end up solving will be massively easier than the problems you’d need to solve to achieve the same performance using A’s tools.
Bringing it back around: This agenda may or may not be viewed as aiming to revolutionize program induction, but I’m not setting out to take the extant program-induction tools and try to cobble together something revolutionary using them. The idea is to use an entirely different line of theory (agent foundations, natural abstractions, information theory, recent DL advances) to achieve that end result.
All of your examples seem to involve applying some radically different approach to overturn a status quo, which is certainly possible. But I am unsure of where you suggest a radically different approach.
The program induction community does take advantage of neural methods and has been doing so for a long time, see recent work from Kevin Ellis but I think as far back as his thesis: https://www.cs.cornell.edu/~ellisk/documents/kevin_ellis_thesis.pdf
Many program induction researchers are cognitive scientists, and have suggested compositional / hierarchical representations for a decades [1] and actively in recent years [2] including even a paper I am on: https://arxiv.org/html/2504.20628v1
So, basically I don’t expect there to be alpha from this observation in itself.
I am, of course, an information theory enthusiast (particularly AIT), but this is an old field and it is not clear to me what specific (recent?) results you hope to leverage? Or why these would results would have been overlooked?
You point to progress on natural abstractions, but to me this only indicates that it should in principle perhaps be possible to come up with some kind of interpretable world model. Has NA research actually produced practical algorithms or methods, or is it in reach of doing so?
Agent foundations as a category is too broad for me to understand what mathematics you are hoping to leverage. And unfortunately, I do not think the AF community has made a large number of significant breakthoughs—reflective oracles, logical induction, and perhaps incomplete models/IB are the main examples left in my mind (you categorized NAH separately), but do not see how they are relevant here.
Basically, you have expressed various hopes, and perhaps some of them are promising (and I will read about them throughout this sequence) but from your comment alone, your alpha over the program induction community (or for instance even the Cyc project) is not legible (to me) - not even as a high-level summary of a technical program. So, I am left hopeful that you will make progress here, but your highly ambitious goal still seems like a distant point on a nearly blank map to me.
[1] Brenden M Lake and Steven T Piantadosi. People infer recursive visual concepts from just a few
examples. Computational Brain & Behavior, 3(1):54–65, 2020
[2] Jerry A Fodor. The language of thought, volume 5. Harvard university press, 1975.
Nice, that’s the sort of poking-of-holes I was looking for.
That’s a good thinking prompt. What is the full set of reasons I’m optimistic about this, in legible terms?
… Hm, but perhaps outlining what you think your edge is in public is not a great idea. I’ll answer in PMs tomorrow. (To ensure future readers get some information about how convincing my reasons are, though, it’d be neat if you posted your impressions afterwards as a response to this comment.)