Good questions. I apparently took too long to answer them, as it sounds like you’ve mostly found the answers yourself (well done!).
“Approximately deterministic functions” and their diagrammatic expression were indeed in Deterministic Natual Latents; I don’t think we’ve talked about them much elsewhere. So you’re not missing any other important source there.
On why we care, a recent central example which you might not have seen is our Illiadpaper. The abstract reads:
Suppose two Bayesian agents each learn a generative model of the same environment. We will assume the two have converged on the predictive distribution (i.e. distribution over some observables in the environment), but may have different generative models containing different latent variables. Under what conditions can one agent guarantee that their latents can be faithfully expressed in terms of the other agent’s latents?
We give simple conditions under which such translation is guaranteed to be possible: the natural latent conditions. We also show that, absent further constraints, these are the most general conditions under which translatability is guaranteed.
(Bold added for emphasis.) That “faithfully expressed in terms of” part is key here. In the paper, we operationalize that as “Under what conditions can one agent guarantee that their latents are (approximately) a stochastic function of the other agent’s latents?”. But that requires dragging in stochastic functions, which are conceptually messier than plain old functions. As a result, there’s a whole section in the paper called “Under what model?”, and basically-all the theorems require an annoying “independent latents” assumption (which basically says that all the interactions between the two agents’ latents are route through the environment).
We could cut all that conceptual mess and all those annoying assumptions out if we instead use (approximately) deterministic natural latents. Then, the core question would be operationalized as “Under what conditions can one agent guarantee that their latents are (approximately) a function of the other agent’s latents?”. Then it’s just plain old functions, we don’t need to worry conceptually about the “under what model?” question (because there’s approximately only one way to put all the latents in one model), and the “independent latents” conditions can all be removed (because different deterministic functions of X are always independent given X). All the messiest and least-compelling parts of the paper could be dropped.
… but the cost would be generality. Stochastic functions are more general than functions, and much more relevant to Bayesians’ world models. So narrowing from stochastic functions to functions would probably mean “assuming our way out of reality”… unless it turns out that (Approximately) Deterministic Natural Latents Are All You Need, which was the subject of our previous bounty post.
The conjecture in this bounty post is even stronger; it would imply that one, and therefore also allow us to simplify the paper a lot.
Good questions. I apparently took too long to answer them, as it sounds like you’ve mostly found the answers yourself (well done!).
“Approximately deterministic functions” and their diagrammatic expression were indeed in Deterministic Natual Latents; I don’t think we’ve talked about them much elsewhere. So you’re not missing any other important source there.
On why we care, a recent central example which you might not have seen is our Illiad paper. The abstract reads:
(Bold added for emphasis.) That “faithfully expressed in terms of” part is key here. In the paper, we operationalize that as “Under what conditions can one agent guarantee that their latents are (approximately) a stochastic function of the other agent’s latents?”. But that requires dragging in stochastic functions, which are conceptually messier than plain old functions. As a result, there’s a whole section in the paper called “Under what model?”, and basically-all the theorems require an annoying “independent latents” assumption (which basically says that all the interactions between the two agents’ latents are route through the environment).
We could cut all that conceptual mess and all those annoying assumptions out if we instead use (approximately) deterministic natural latents. Then, the core question would be operationalized as “Under what conditions can one agent guarantee that their latents are (approximately) a function of the other agent’s latents?”. Then it’s just plain old functions, we don’t need to worry conceptually about the “under what model?” question (because there’s approximately only one way to put all the latents in one model), and the “independent latents” conditions can all be removed (because different deterministic functions of X are always independent given X). All the messiest and least-compelling parts of the paper could be dropped.
… but the cost would be generality. Stochastic functions are more general than functions, and much more relevant to Bayesians’ world models. So narrowing from stochastic functions to functions would probably mean “assuming our way out of reality”… unless it turns out that (Approximately) Deterministic Natural Latents Are All You Need, which was the subject of our previous bounty post.
The conjecture in this bounty post is even stronger; it would imply that one, and therefore also allow us to simplify the paper a lot.