The idea here is that world modelling (working out what the state of the world at the present moment is) and planning (working out what to do given the state of the world at the present moment) might be very tangled up with each other in the source code for some AI agents.
When we think of building agents that act in the world, it’s common to imagine that they will first use the information available to them to create a model of the world, and then, given that, formulate a plan to achieve some kind of goal. That’s one possible way to build agents, but John’s post here actually attempts to say something about the space of all possible agents. While some agents may have nicely separated modelling/planning algorithms, it’s not guaranteed that it will be that way at all, and the point of my comment here was to show that for any nicely separated agent, there is a not-nicely-separated agent that arrives at the same plan in the limit of time.
My argument here goes as follows: suppose that you have some agent that is nicely separated into world modelling and planning submodules/sub-algorithms. Then you can use the three dot points in my comment to construct source code for a new agent that does the same thing, but is not nicely separated. The point of this is to show that it cannot be that the best or most optimal agents are nicely separated, because for every nicely separated agent source code, there is an equally-good not-nice-separated agent source code.
Then you can use the three dot points in my comment to construct source code for a new agent that does the same thing, but is not nicely separated.
This is the step I don’t get (how we make the construction), because I don’t understand SGD well. What does “sample N world models” mean?
My attempt to understand: We have a space of world models (Sm) and a space of plans (Sp). We pick points from Sp (using SGD) and evaluate them on the best points of Sm (we got those best points by trying to predict the world and applying SGD).
My thoughts/questions: To find the best points of Sm, we still need to do modelling independently from planning? While the world model is not stored in memory, some pointer to the best points of Smis stored? We at least have “the best current plan” stored independently from the world models?
The idea here is that world modelling (working out what the state of the world at the present moment is) and planning (working out what to do given the state of the world at the present moment) might be very tangled up with each other in the source code for some AI agents.
When we think of building agents that act in the world, it’s common to imagine that they will first use the information available to them to create a model of the world, and then, given that, formulate a plan to achieve some kind of goal. That’s one possible way to build agents, but John’s post here actually attempts to say something about the space of all possible agents. While some agents may have nicely separated modelling/planning algorithms, it’s not guaranteed that it will be that way at all, and the point of my comment here was to show that for any nicely separated agent, there is a not-nicely-separated agent that arrives at the same plan in the limit of time.
My argument here goes as follows: suppose that you have some agent that is nicely separated into world modelling and planning submodules/sub-algorithms. Then you can use the three dot points in my comment to construct source code for a new agent that does the same thing, but is not nicely separated. The point of this is to show that it cannot be that the best or most optimal agents are nicely separated, because for every nicely separated agent source code, there is an equally-good not-nice-separated agent source code.
This is the step I don’t get (how we make the construction), because I don’t understand SGD well. What does “sample N world models” mean?
My attempt to understand: We have a space of world models (Sm) and a space of plans (Sp). We pick points from Sp (using SGD) and evaluate them on the best points of Sm (we got those best points by trying to predict the world and applying SGD).
My thoughts/questions: To find the best points of Sm, we still need to do modelling independently from planning? While the world model is not stored in memory, some pointer to the best points of Sm is stored? We at least have “the best current plan” stored independently from the world models?