This post contains an interesting mathematical result: that the machinery of natural latents can be transferred from classical information theory to algorithmic information theory. I find it intriguing for multiple reasons:
It updates me towards natural latents being a useful concept for foundational questions in agent theory, as opposed to being some artifact of overindexing on Bayesian networks as the “right” ontology.
The proof technique involves defining an algorithmic information theory analogue of Bayesian networks, which is something I haven’t seen before and seems quite interesting in itself.
It would be interesting to see whether any of this carries over to the efficiently computable counterparts of Kolmogorov complexity I recently invented[1].
The main thing this post is missing is any rigorous examples or existence proofs of these AIT natural latents. I’m guessing that the following construction should work:
Choose a universal Turing machine T.
Choose Λ to be a T-program for a total recursive function s.t. K(Λ)≫0.
Choose ϕi to be random strings of length n≫0.
Set xi:=T(Λ,ϕi).
Then, with high probability, Λ is a natural latent for the xi. (I think?)
It would be nice to see something like that in the post.
These ideas seem conceptually close to concepts like sophistication in algorithmic statistics, and the connection might be worth investigating.
Now, about the stated motivation: the OP claims that natural latents capture how “reasonable” agents choose to define categories about the world. The argument seems somewhat compelling, although some further justification is required for the claim that
If you’ve been wondering why on Earth we would ever expect to find such simple structures in the complicated real world, conditioning on background knowledge is the main answer.
That said, I think that real-world categorizations are also somewhat value-laden: depending on the agent’s preferences, and on the laws of the universe in which they find themselves, there might be particular features they care about much more than other features. (Since they are more decision-relevant.) The importance of these features will likely influence which categories are useful to define. This fact cannot be captured in a formalism on the level of abstraction in this post. (Although maybe we can get some of the way there by drawing on rate-distortion theory?)
This post contains an interesting mathematical result: that the machinery of natural latents can be transferred from classical information theory to algorithmic information theory. I find it intriguing for multiple reasons:
It updates me towards natural latents being a useful concept for foundational questions in agent theory, as opposed to being some artifact of overindexing on Bayesian networks as the “right” ontology.
The proof technique involves defining an algorithmic information theory analogue of Bayesian networks, which is something I haven’t seen before and seems quite interesting in itself.
It would be interesting to see whether any of this carries over to the efficiently computable counterparts of Kolmogorov complexity I recently invented[1].
The main thing this post is missing is any rigorous examples or existence proofs of these AIT natural latents. I’m guessing that the following construction should work:
Choose a universal Turing machine T.
Choose Λ to be a T-program for a total recursive function s.t. K(Λ)≫0.
Choose ϕi to be random strings of length n≫0.
Set xi:=T(Λ,ϕi).
Then, with high probability, Λ is a natural latent for the xi. (I think?)
It would be nice to see something like that in the post.
These ideas seem conceptually close to concepts like sophistication in algorithmic statistics, and the connection might be worth investigating.
Now, about the stated motivation: the OP claims that natural latents capture how “reasonable” agents choose to define categories about the world. The argument seems somewhat compelling, although some further justification is required for the claim that
That said, I think that real-world categorizations are also somewhat value-laden: depending on the agent’s preferences, and on the laws of the universe in which they find themselves, there might be particular features they care about much more than other features. (Since they are more decision-relevant.) The importance of these features will likely influence which categories are useful to define. This fact cannot be captured in a formalism on the level of abstraction in this post. (Although maybe we can get some of the way there by drawing on rate-distortion theory?)
Still unpublished.