It doesn’t even have any agency in choosing its input; it’s given a big wodge of training data, and has to ingest it. It has less agency than a slug, and therefore can’t really learn to do anything “agenty”.
It’s quite trival to change it in a way where it’s output feeds back into it’s input given that it’s input is text and it’s output is text.
You can make the output console comments and then feed the resulting console answer back into the model. It likely needs a larger attention fields to be practically useful but more compute and clever ways to handle it could lead there.
Our own thinking process is also a lot about having a short term memory into which we put new thoughts and based on which our next action/thought gets generated.
It’s quite trival to change it in a way where it’s output feeds back into it’s input given that it’s input is text and it’s output is text.
You can make the output console comments and then feed the resulting console answer back into the model. It likely needs a larger attention fields to be practically useful but more compute and clever ways to handle it could lead there.
Our own thinking process is also a lot about having a short term memory into which we put new thoughts and based on which our next action/thought gets generated.