I actually implemented my own private benchmark last year to try to test this with different models. The domain was a toy OOD task where the system had access to three possible tools that performed simple transformations on a configuration of binary values in a particular spatial arrangement. Stage 1 was exploration. The system was given a certain number of steps to probe with the tools (which were chosen randomly from a subset prior to each trial). After the experimentation stage, the system was required to use the tools to perform a transformation on a random arrangement to make it match a target one.
The exercise of building a benchmark was a great learning experience for me. My main takeaway was that differences in performance were nearly all driven by differences in scaffolding, and not so much the base model. This made me fairly disillusioned about benchmarks in general. Made me suspect that gains in benchmarks like ARC-AGI are mostly driven by scaffolding improvements. Maybe someone here has much more insight into that.
But it also made me think that the problem is probably not some far-out radically intractable problem. You mention continual learning and long time horizons. Just generally for OOD tasks, the system needs to be able to log results, generate and revise hypotheses, and carry out Bayesian updates in an iterative manner. Whether that can be cracked reliably for increasingly difficult problems with relatively straightforward scaffolding, or the base models need to be radically improved along with scaffolding, I don’t really know. Maybe for the much more difficult problems (like a Theory of Everything or a cure for the common cold) those advances are very far out. I would think though, that for simple and medium-difficulty problems, the frontier labs are already well on their way.
So with decent scaffolding (search, summarization, etc) and 1m-token context memory, one can do quite a lot even without a robust solution to continual learning? That matches the current situations for quite a lot of agentic tasks.
ARC-AGi is notorious for being insoluble without scaffolding (e.g domain-specific languages), and strongly scaffolding-dependent with it. Scores on it do depend somewhat on model capacity, but are also strongly dependent on the effort and skill put into building scaffolding for it. What would impress me most would be a score where the model built its own scaffolding with only some small amount of human assistance (ideally, zero)
I actually implemented my own private benchmark last year to try to test this with different models. The domain was a toy OOD task where the system had access to three possible tools that performed simple transformations on a configuration of binary values in a particular spatial arrangement. Stage 1 was exploration. The system was given a certain number of steps to probe with the tools (which were chosen randomly from a subset prior to each trial). After the experimentation stage, the system was required to use the tools to perform a transformation on a random arrangement to make it match a target one.
The exercise of building a benchmark was a great learning experience for me. My main takeaway was that differences in performance were nearly all driven by differences in scaffolding, and not so much the base model. This made me fairly disillusioned about benchmarks in general. Made me suspect that gains in benchmarks like ARC-AGI are mostly driven by scaffolding improvements. Maybe someone here has much more insight into that.
But it also made me think that the problem is probably not some far-out radically intractable problem. You mention continual learning and long time horizons. Just generally for OOD tasks, the system needs to be able to log results, generate and revise hypotheses, and carry out Bayesian updates in an iterative manner. Whether that can be cracked reliably for increasingly difficult problems with relatively straightforward scaffolding, or the base models need to be radically improved along with scaffolding, I don’t really know. Maybe for the much more difficult problems (like a Theory of Everything or a cure for the common cold) those advances are very far out. I would think though, that for simple and medium-difficulty problems, the frontier labs are already well on their way.
So with decent scaffolding (search, summarization, etc) and 1m-token context memory, one can do quite a lot even without a robust solution to continual learning? That matches the current situations for quite a lot of agentic tasks.
ARC-AGi is notorious for being insoluble without scaffolding (e.g domain-specific languages), and strongly scaffolding-dependent with it. Scores on it do depend somewhat on model capacity, but are also strongly dependent on the effort and skill put into building scaffolding for it. What would impress me most would be a score where the model built its own scaffolding with only some small amount of human assistance (ideally, zero)