I don’t know what the “underlying real probability” is (no condescendence in this remark; I’m genuinely confused about the physics and philosophy of probability and haven’t got time to figure it out for myself, and I’m not sure this is a settled question).
Both P and Q are something that is implemented (i. e., encoded in some way) by the agent itself. The agent knows nothing about the “true generative model” of the environment (even if we can discuss it; see below). The only place where “the feedback from the environment” enters this process is in the calculation of P(st+1|ot), so-called “posterior” belief, which is calculated according to the rules of Bayesian inference. This is the place where the agent is “ensured not to detach from the observations”, i. e., the reality of its environment.
I would say, the book doesn’t do a very good job of explaining this point. I recommend this paper, section 1 (“Basic terminology, concepts, and mathematics”), and appending A (“Additional mathematical details”) that make the mathematics of Active Inference really clear, they explain every transition and derivation of the formalism in detail.
Then, even though an agent uses “its own” generative model of the environment, it is expected to track, with some degree of fidelity, the real dynamics of the environment. This is the whole point of Active Inference, of course. I used the phrase “real dynamics” rather than “generative model” because there is philosophical nuance and can make the phrase “generative model of the environment” misleading or confusing to people. There was a paper specifically aimed to clear out to clear this confusion (“A tale of two densities: Active Inference is enactive inference”) But I think that attempt failed, i. e. the paper only added more confusion. Instead of that paper, for physical foundations of Active Inference, that also elucidates this dynamics between the agent and the environment, I’d recommend “A free energy principle for generic quantum systems”.
I don’t know what the “underlying real probability” is (no condescendence in this remark; I’m genuinely confused about the physics and philosophy of probability and haven’t got time to figure it out for myself, and I’m not sure this is a settled question).
Both P and Q are something that is implemented (i. e., encoded in some way) by the agent itself. The agent knows nothing about the “true generative model” of the environment (even if we can discuss it; see below). The only place where “the feedback from the environment” enters this process is in the calculation of P(st+1|ot), so-called “posterior” belief, which is calculated according to the rules of Bayesian inference. This is the place where the agent is “ensured not to detach from the observations”, i. e., the reality of its environment.
I would say, the book doesn’t do a very good job of explaining this point. I recommend this paper, section 1 (“Basic terminology, concepts, and mathematics”), and appending A (“Additional mathematical details”) that make the mathematics of Active Inference really clear, they explain every transition and derivation of the formalism in detail.
Then, even though an agent uses “its own” generative model of the environment, it is expected to track, with some degree of fidelity, the real dynamics of the environment. This is the whole point of Active Inference, of course. I used the phrase “real dynamics” rather than “generative model” because there is philosophical nuance and can make the phrase “generative model of the environment” misleading or confusing to people. There was a paper specifically aimed to clear out to clear this confusion (“A tale of two densities: Active Inference is enactive inference”) But I think that attempt failed, i. e. the paper only added more confusion. Instead of that paper, for physical foundations of Active Inference, that also elucidates this dynamics between the agent and the environment, I’d recommend “A free energy principle for generic quantum systems”.