Adding to reading list: clearly I need to read Jaynes’s writing on determining priors from symmetries.
I initially dismissed this, because I’m not looking for a prior here. But consider the statistical mechanics case: an ideal gas’s distribution is given by the max entropy distribution subject to the constraint that average energy is some fixed value. This is basically the sort of thing Jaynes looks at. What I want to do is essentially go backwards—we start with a probability distribution, and need to figure out the energy, and it shouldn’t require having literally max entropy.
The stuff he did, to my understanding, is to show that if we require that the distribution is invariant under some symmetry group, then we can deduce the probability distribution in many cases. This seems to include doing things like deducing a distribution over a parameter, which should take the role of a latent?
I still find it fairly likely that it ends up not doing what’s needed, but is definitely something to look at.
Currently, I realize there’s a simple question that surely has standard answers: what’s the symmetries of the Boltzmann distribution? From there, maybe we can get that if know that P(X|L) has some symmetries, then we can find P(X|L)… and hopefully maybe also P(L). It’s still too circular, though. But regardless—first, find the symmetries in that case.
Adding to reading list: clearly I need to read Jaynes’s writing on determining priors from symmetries.
I initially dismissed this, because I’m not looking for a prior here. But consider the statistical mechanics case: an ideal gas’s distribution is given by the max entropy distribution subject to the constraint that average energy is some fixed value. This is basically the sort of thing Jaynes looks at. What I want to do is essentially go backwards—we start with a probability distribution, and need to figure out the energy, and it shouldn’t require having literally max entropy.
The stuff he did, to my understanding, is to show that if we require that the distribution is invariant under some symmetry group, then we can deduce the probability distribution in many cases. This seems to include doing things like deducing a distribution over a parameter, which should take the role of a latent?
I still find it fairly likely that it ends up not doing what’s needed, but is definitely something to look at.
Currently, I realize there’s a simple question that surely has standard answers: what’s the symmetries of the Boltzmann distribution? From there, maybe we can get that if know that P(X|L) has some symmetries, then we can find P(X|L)… and hopefully maybe also P(L). It’s still too circular, though. But regardless—first, find the symmetries in that case.