“The base layer is ultimately made up of models of characters, in a Simulators-ish sense” No it is not, in a similar way as what your brain is running is not ultimately made of characters. It’s ultimately made of approximate bayesian models.
With respect to active inference … Sorry, don’t want to be offensive, but it would actually be helpful for your project to understand active inference at least a bit. Empirically it seems has-repeatedly-read-Scott-Alexander’s-posts-on-it leads people to some weird epistemic state, in which people seem to have a sense of understanding, but are unable to answer even basic questions, make very easy predictions, etc. I suspect what’s going on is a bit like if someone reads some well written science popularization book about quantum mechanics but actually lacks concepts like complex numbers or vector spaces, they may have somewhat superficial sense of understanding. Obviously active inference has a lot to say about how people self-model themselves. For example, when typing these words, I assume it’s me who types them (and not someone else, for example). Why? That’s actually important question for why self. Why not, or to what extent not in LLMs? How stories that people tell themselves about who they are impact what they do is totally something which makes sense to understand from active inference perspective.
it would actually be helpful for your project to understand active inference at least a bit. Empirically it seems has-repeatedly-read-Scott-Alexander’s-posts-on-it leads people to some weird epistemic state
Fair enough — is there a source you’d most recommend for learning more?
“The base layer is ultimately made up of models of characters, in a Simulators-ish sense” No it is not, in a similar way as what your brain is running is not ultimately made of characters. It’s ultimately made of approximate bayesian models.
what distinguishes a context-prompted ephemeral persona from that richer and more persistent character Check Why Simulator AIs want to be Active Inference AIs
With respect to active inference … Sorry, don’t want to be offensive, but it would actually be helpful for your project to understand active inference at least a bit. Empirically it seems has-repeatedly-read-Scott-Alexander’s-posts-on-it leads people to some weird epistemic state, in which people seem to have a sense of understanding, but are unable to answer even basic questions, make very easy predictions, etc. I suspect what’s going on is a bit like if someone reads some well written science popularization book about quantum mechanics but actually lacks concepts like complex numbers or vector spaces, they may have somewhat superficial sense of understanding.
Obviously active inference has a lot to say about how people self-model themselves. For example, when typing these words, I assume it’s me who types them (and not someone else, for example). Why? That’s actually important question for why self. Why not, or to what extent not in LLMs? How stories that people tell themselves about who they are impact what they do is totally something which makes sense to understand from active inference perspective.
Fair enough — is there a source you’d most recommend for learning more?