So with decent scaffolding (search, summarization, etc) and 1m-token context memory, one can do quite a lot even without a robust solution to continual learning? That matches the current situations for quite a lot of agentic tasks.
ARC-AGi is notorious for being insoluble without scaffolding (e.g domain-specific languages), and strongly scaffolding-dependent with it. Scores on it do depend somewhat on model capacity, but are also strongly dependent on the effort and skill put into building scaffolding for it. What would impress me most would be a score where the model built its own scaffolding with only some small amount of human assistance (ideally, zero)
So with decent scaffolding (search, summarization, etc) and 1m-token context memory, one can do quite a lot even without a robust solution to continual learning? That matches the current situations for quite a lot of agentic tasks.
ARC-AGi is notorious for being insoluble without scaffolding (e.g domain-specific languages), and strongly scaffolding-dependent with it. Scores on it do depend somewhat on model capacity, but are also strongly dependent on the effort and skill put into building scaffolding for it. What would impress me most would be a score where the model built its own scaffolding with only some small amount of human assistance (ideally, zero)