For my interest, for these reallife latents with many different pieces contributing a small amount of information do you reckon Eisenstat’s Condensation / some unpublished work you mentioned at ODYSSEY would be the right framework here?
Sort of. Condensation as-written requires what David and I call “strong redundancy”, i.e. the latent must be determinable from any one observable downstream, which is the opposite of “small amount of information from each individual observable”. But it’s pretty easy to bridge between the two mathematically by glomming together multiple observables into one, which is usually how David and I think about it.
The way you’d use this is:
Use the sort of machinery above to find a latent which is weakly loaded on many different observables.
Check how well that latent satisfies redundancy over some subset of the observables.
If we can find disjoint subsets of observables (any disjoint subsets) such that the latent can be determined reasonably well from any one of the subsets, then the machinery of natural latents/condensation kicks in to give us guarantees about universality of the latent.
For my interest, for these reallife latents with many different pieces contributing a small amount of information do you reckon Eisenstat’s Condensation / some unpublished work you mentioned at ODYSSEY would be the right framework here?
Sort of. Condensation as-written requires what David and I call “strong redundancy”, i.e. the latent must be determinable from any one observable downstream, which is the opposite of “small amount of information from each individual observable”. But it’s pretty easy to bridge between the two mathematically by glomming together multiple observables into one, which is usually how David and I think about it.
The way you’d use this is:
Use the sort of machinery above to find a latent which is weakly loaded on many different observables.
Check how well that latent satisfies redundancy over some subset of the observables.
If we can find disjoint subsets of observables (any disjoint subsets) such that the latent can be determined reasonably well from any one of the subsets, then the machinery of natural latents/condensation kicks in to give us guarantees about universality of the latent.