“Even if you know everything about a system, there will still be uncertainty left.”
My guess, prior to reading the rest: aleatoric uncertainty? Like, the type signature of “systems” in question is that they’re inherently stochastic. To “know everything” about a system does not mean eliminating all uncertainty about the system’s state, much like knowing the exact function computing a probability distribution does not mean being able to deterministically predict the next sample from that distribution. The catch is in where you draw the “boundaries” of what defines the relevant “system” you want to model.
Okay, it’s not the above.
Hmm, the actual explanation doesn’t seem like an elegant way to structure this argument, though. It is not a pure probability-theoretic claim about the relevant systems, that you end up with uncertainty after seeing the exact state. It’s a claim about what happens when you use a specific convention for modeling systems, a convention whose choice is downstream of additional practical constraints/desiderata that sit outside pure (compute-unbounded!) probability theory. Nothing in the initial statement hints at that!
“If you’re using a probabilistic model with latent variables in it, your model will still have remaining uncertainty even after you’ve seen the low-level state of the whole world.”
I don’t think this pins it down exactly. It’s not true for every possible model with latent variables; you can have a model with posterior-update rules that do collapse probability distributions to point estimates if they detect that they’ve updated on everything.[1] Rather, it’s a claim about the specific type of probabilistic models that optimal embedded agents are going to use to model the worlds they’re in.
Well, I suppose there’s some messiness around the edges. E. g., if you’re modeling the latent variables as being real-valued, pining their values down exactly would imply updating on an infinite amount of evidence, so you’re forced to either use discrete variables, or “ugly” update rules. Still, generally there’s nothing saying that we can’t do that, per se.
My guess, prior to reading the rest: aleatoric uncertainty? Like, the type signature of “systems” in question is that they’re inherently stochastic. To “know everything” about a system does not mean eliminating all uncertainty about the system’s state, much like knowing the exact function computing a probability distribution does not mean being able to deterministically predict the next sample from that distribution. The catch is in where you draw the “boundaries” of what defines the relevant “system” you want to model.
Okay, it’s not the above.
Hmm, the actual explanation doesn’t seem like an elegant way to structure this argument, though. It is not a pure probability-theoretic claim about the relevant systems, that you end up with uncertainty after seeing the exact state. It’s a claim about what happens when you use a specific convention for modeling systems, a convention whose choice is downstream of additional practical constraints/desiderata that sit outside pure (compute-unbounded!) probability theory. Nothing in the initial statement hints at that!
I don’t think this pins it down exactly. It’s not true for every possible model with latent variables; you can have a model with posterior-update rules that do collapse probability distributions to point estimates if they detect that they’ve updated on everything.[1] Rather, it’s a claim about the specific type of probabilistic models that optimal embedded agents are going to use to model the worlds they’re in.
Well, I suppose there’s some messiness around the edges. E. g., if you’re modeling the latent variables as being real-valued, pining their values down exactly would imply updating on an infinite amount of evidence, so you’re forced to either use discrete variables, or “ugly” update rules. Still, generally there’s nothing saying that we can’t do that, per se.