I wrote this tl;dr for a friend, and thought it worth sharing. I’m not sure it’s accurate. I’ve only read the “Recap”
Here is how I understand it.
Suppose that, depending on the temperature, your mirror might be foggy and you might have goose pimples. As in, the temperature helps you predict those variables. But once you know the temperature, there’s (approximately) nothing you learn about the state of your mirror from your skin, and vice versa. And! Once you know whether your mirror is foggy, there’s basically nothing left to learn about the temperature by observing your skin (and vice versa).
But you still don’t know the temperature once you observe those things.
This is a stochastic (approximate) natural latent. The stochasticity is that you don’t know the temperature once you know the mirror and skin states.
Their theorem, iiuc, says that there does exist a variable where you (approximately) know its exact state after you’ve observed either the mirror or your skin.
(I don’t currently understand exactly what coarse-graining process they’re using to construct the exact natural latent).
The theorem doesn’t actually specify a coarse-graining process. The proof would say:
We can construct a new variable T’ by sampling a temperature given mirror-state. By construction, mirror-state perfectly mediates between T’ and goosebumps.
There exists some pareto-optimal (under the errors of the natural latent conditions) latent which is pareto-as-good-as T’
Any pareto optimal latent which is pareto-as-good-as T’ can be perfectly coarse-grained, by graining together any values of the latent which give exactly the same distribution P[mirror|latent value].
Because the middle bullet is not constructive, we don’t technically specify a process. That said, one could specify a process straightforwardly by just starting from T’ and pareto-improving the latent in a specific direction until one hits an optimum.
In this case, the coarse-graining would probably just be roughly (temperatures at which the mirror fogs) and (temperatures at which it doesn’t), since that’s the only nontrivial coarse-graining allowed by the setup (because the coarse-grained value must be approximately determined by the mirror-state).
Noticing this only works as an example if the two signals are (approximately) the same partition of T, i.e. (temperatures at which the mirror fogs) is approximately the same as (temperatures at which you have goosebumps).
And! Once you know whether your mirror is foggy, there’s basically nothing left to learn about the temperature by observing your skin (and vice versa).
is supposed to be scoped under the “Suppose that” from the beginning of the paragraph
I wrote this tl;dr for a friend, and thought it worth sharing. I’m not sure it’s accurate. I’ve only read the “Recap”
Here is how I understand it.
Suppose that, depending on the temperature, your mirror might be foggy and you might have goose pimples. As in, the temperature helps you predict those variables. But once you know the temperature, there’s (approximately) nothing you learn about the state of your mirror from your skin, and vice versa. And! Once you know whether your mirror is foggy, there’s basically nothing left to learn about the temperature by observing your skin (and vice versa).
But you still don’t know the temperature once you observe those things.
This is a stochastic (approximate) natural latent. The stochasticity is that you don’t know the temperature once you know the mirror and skin states.
Their theorem, iiuc, says that there does exist a variable where you (approximately) know its exact state after you’ve observed either the mirror or your skin.
(I don’t currently understand exactly what coarse-graining process they’re using to construct the exact natural latent).
Yup, good example!
The theorem doesn’t actually specify a coarse-graining process. The proof would say:
We can construct a new variable T’ by sampling a temperature given mirror-state. By construction, mirror-state perfectly mediates between T’ and goosebumps.
There exists some pareto-optimal (under the errors of the natural latent conditions) latent which is pareto-as-good-as T’
Any pareto optimal latent which is pareto-as-good-as T’ can be perfectly coarse-grained, by graining together any values of the latent which give exactly the same distribution P[mirror|latent value].
Because the middle bullet is not constructive, we don’t technically specify a process. That said, one could specify a process straightforwardly by just starting from T’ and pareto-improving the latent in a specific direction until one hits an optimum.
In this case, the coarse-graining would probably just be roughly (temperatures at which the mirror fogs) and (temperatures at which it doesn’t), since that’s the only nontrivial coarse-graining allowed by the setup (because the coarse-grained value must be approximately determined by the mirror-state).
Noticing this only works as an example if the two signals are (approximately) the same partition of T, i.e. (temperatures at which the mirror fogs) is approximately the same as (temperatures at which you have goosebumps).
Yeah I think
is supposed to be scoped under the “Suppose that” from the beginning of the paragraph