In response, Reiichiro Nakano shared this paper: https://arxiv.org/pdf/1901.03729.pdf which kinda shows it’s possible to have agent state/action representations in natural language for Frogger. There are probably glaring/obvious flaws with my OP, but this was what inspired those thoughts.
(I’ve only read the abstract of the linked paper.)
If you did something like this with GPT-3, you’d essentially have GPT-3 try to rationalize the actions of the chess engine the way a human would. This feels more like having two separate agents with a particular mode of interaction, rather than a single agent with a connection between symbolic and subsymbolic representations.
(One intuition pump: notice that there isn’t any point where a gradient affects both the GPT-3 weights and the chess engine weights.)
(I’ve only read the abstract of the linked paper.)
If you did something like this with GPT-3, you’d essentially have GPT-3 try to rationalize the actions of the chess engine the way a human would. This feels more like having two separate agents with a particular mode of interaction, rather than a single agent with a connection between symbolic and subsymbolic representations.
(One intuition pump: notice that there isn’t any point where a gradient affects both the GPT-3 weights and the chess engine weights.)