Really interesting. Do you think beliefs and values can be good primitives? (But still, even assuming we have beliefs and values as primitives, if the state space is very large or even infinite, we can’t enumerate all states and check their values so we need something more compact, structured, and interpretable.)
We recently tried to work on a simplified version of this problem using a combination of behavioural evals and analysis of model internals. The state space is much more manageable in grid worlds, but we’re excited about (1) formalising goals, goal inference, goal generalisation, etc. and (2) scaling up the approach to complex, realistic tasks.
Really interesting. Do you think beliefs and values can be good primitives? (But still, even assuming we have beliefs and values as primitives, if the state space is very large or even infinite, we can’t enumerate all states and check their values so we need something more compact, structured, and interpretable.)
We recently tried to work on a simplified version of this problem using a combination of behavioural evals and analysis of model internals. The state space is much more manageable in grid worlds, but we’re excited about (1) formalising goals, goal inference, goal generalisation, etc. and (2) scaling up the approach to complex, realistic tasks.