I understand your point of view and think it is reasonable.
However, I don’t think “don’t build bigger models” and “don’t train models to do complicated things” need to be at odds with each other. I see the argument you are making, but I think success on these asks are likely highly correlated via the underlying causal factor of humanity being concerned enough about AI x-risk and coordinated enough to ensure responsible AI development.
I also think the training procedure matters a lot (and you seem to be suggesting otherwise?), since if you don’t do RL or other training schemes that seem designed to induce agentyness and you don’t do tasks that use an agentic supervision signal, then you probably don’t get agents for a long time (if ever).
if you don’t do RL or other training schemes that seem designed to induce agentyness and you don’t do tasks that use an agentic supervision signal, then you probably don’t get agents for a long time
Is this really the case? If you imagine a perfect Oracle AI, which is certainly not agenty, it seems to me that with some simple scaffolding, one could construct a highly agentic system. It would go something along the lines of
Setup API access to ‘things’ which can interact with the real world.
Ask the oracle ‘What would be the optimal action if you want to do <insert-goal> via <insert-api-functions>?’
Do the actions that are outputted.
Some kind of looping mechanism to gain feedback from the world and account for it.
This is my line of reasoning why AIS matters for language models in general.
I meant “other training schemes” to encompass things like scaffolding that deliberately engineers agents using LLMs as components, although I acknowledge they are not literally “training” and more like “engineering”.
The thing that we care about is how long it takes to get to agents. If we put lots of effort making powerful Oracle systems or other non-agentic systems, we must assume that agentic systems will follow shortly. Someone will make them, even if you do not.
I understand your point of view and think it is reasonable.
However, I don’t think “don’t build bigger models” and “don’t train models to do complicated things” need to be at odds with each other. I see the argument you are making, but I think success on these asks are likely highly correlated via the underlying causal factor of humanity being concerned enough about AI x-risk and coordinated enough to ensure responsible AI development.
I also think the training procedure matters a lot (and you seem to be suggesting otherwise?), since if you don’t do RL or other training schemes that seem designed to induce agentyness and you don’t do tasks that use an agentic supervision signal, then you probably don’t get agents for a long time (if ever).
Is this really the case? If you imagine a perfect Oracle AI, which is certainly not agenty, it seems to me that with some simple scaffolding, one could construct a highly agentic system. It would go something along the lines of
Setup API access to ‘things’ which can interact with the real world.
Ask the oracle ‘What would be the optimal action if you want to do <insert-goal> via <insert-api-functions>?’
Do the actions that are outputted.
Some kind of looping mechanism to gain feedback from the world and account for it.
This is my line of reasoning why AIS matters for language models in general.
I meant “other training schemes” to encompass things like scaffolding that deliberately engineers agents using LLMs as components, although I acknowledge they are not literally “training” and more like “engineering”.
The thing that we care about is how long it takes to get to agents. If we put lots of effort making powerful Oracle systems or other non-agentic systems, we must assume that agentic systems will follow shortly. Someone will make them, even if you do not.
I don’t disagree… in this case you don’t get agents for a long time; someone else does though.