johnswentworth comments on $500 Bounty Problem: Are (Approximately) Deterministic Natural Latents All You Need?

[johnswentworth]

Here’s a trick which might be helpful for anybody tackling the problem.

First, note that $f\left(\Lambda \right):=(x\mapsto P[X=x|\Lambda \left]\right)$ is always a sufficient statistic of $\Lambda$ for $X$, i.e.

$\Lambda \to f\left(\Lambda \right)\to X$

Now, we typically expect that the lower-order bits of $f\left(\Lambda \right)$ are less relevant/​useful/​interesting. So, we might hope that we can do some precision cutoff on $f\left(\Lambda \right)$, and end up with an approximate suficient statistic, while potentially reducing the entropy (or some other information content measure) of $f\left(\Lambda \right)$ a bunch. We’d broadcast the cutoff function like this:

$g\left(\Lambda \right):=\text{precison\_cutoff}\left(f\right(\Lambda \left)\right)=(x\mapsto \text{precision\_cutoff}(P[X=x|\Lambda ]\left)\right)$

Now we’ll show a trick for deriving $D_{KL}$ bounds involving $g\left(\Lambda \right)$.

First note that

$E\left[D_{KL}\right(P\left[X|\Lambda \right]||P\left[X|g\right(\Lambda \left)\right]\left)\right]\le E\left[D_{KL}\right(P\left[X|\Lambda \right]||g\left(\Lambda \right)\left)\right]$

This is a tricky expression, so let’s talk it through. On the left, $g\left(\Lambda \right)$ is treated informationally; it’s just a generic random variable constructed as a generic function of $\Lambda$, and we condition on that random variable in the usual way. On the right, the output-value of $g$ is being used as a distribution over $X$.

The reason this inequality holds is because a Bayes update is the “best” update one can make, as measured by expected $D_{KL}$. Specifically, if I’m given the value of any function $g\left(\Lambda \right)$, then the distribution $Q$ (as a function of $g\left(\Lambda \right))$ which minimizes $E\left[D_{KL}\right(P\left[X|\Lambda \right]||Q\left)\right]$ is $P\left[X|g\right(\Lambda \left)\right]$. Since $P\left[X|g\right(\Lambda \left)\right]$ minimizes that expected $D_{KL}$, any other distribution over $X$ (as a function of $g\left(\Lambda \right)$) can only do “worse”—including $g\left(\Lambda \right)$ itself, since that’s a distribution over $X$, and is a function of $g\left(\Lambda \right)$.

Plugging in the definition of $g$, that establishes

$E\left[D_{KL}\right(P\left[X|\Lambda \right]||P\left[X|g\right(\Lambda \left)\right]\left)\right]\le E\left[D_{KL}\right(P\left[X|\Lambda \right]||(x\mapsto \text{precision\_cutoff}(P[X=x|\Lambda ]\left)\right)\left)\right]$

Then the final step is to use the properties of whatever $\text{precision\_cutoff}$ function one chose, to establish that $E\left[D_{KL}\right(P\left[X|\Lambda \right]||(x\mapsto \text{precision\_cutoff}(P[X=x|\Lambda ]\left)\right)\left)\right]$ can’t be too far from $E\left[D_{KL}\right(P\left[X|\Lambda \right]||P\left[X|\Lambda \right]\left)\right]$, i.e. 0. That produces an upper bound on $E\left[D_{KL}\right(P\left[X|\Lambda \right]||P\left[X|g\right(\Lambda \left)\right]\left)\right]$, where the bound is 0 + (whatever terms came from the precision cutoff).

[johnswentworth]

@Alfred Harwood @David Johnston

If anyone else would like to be tagged in comments like this one on this post, please eyeball-react on this comment. Alfred and David, if you would like to not be tagged in the future, please say so.