AF has long been focused on AIXI-type cases, and later things like embedded agency. It’s not implausible that there are formalizable concepts for agency/optimization in scaffolded LLM agents (like optimization in in-context learning?), but as of right now, AF seems quite far from it. AF’s wikitag should also give a good idea of where the field is at
I think a scaffolded LLM agent is substantially an embedded agent, no? To the extent that it interfaces with users and tries to satisfy user requests, it needs to have a self-model in its world model.
AF has long been focused on AIXI-type cases, and later things like embedded agency. It’s not implausible that there are formalizable concepts for agency/optimization in scaffolded LLM agents (like optimization in in-context learning?), but as of right now, AF seems quite far from it. AF’s wikitag should also give a good idea of where the field is at
I think a scaffolded LLM agent is substantially an embedded agent, no? To the extent that it interfaces with users and tries to satisfy user requests, it needs to have a self-model in its world model.