Yep, I know of this result. I haven’t looked into it in depth, but my understanding is that it only says that powerful predictors have to be “complex” in the sense of high Kolmogorov complexity, right? But “high K-complexity” doesn’t mean “is a monolithic, irreducibly complex mess”. In particular, it doesn’t rule out this property:
“Well-structured”: has an organized top-down hierarchical structure, learning which lets you quickly navigate to specific information in it.
Wikipedia has pretty high K-complexity, well beyond the ability of the human mind to hold in its working memory. But it’s still usable, because you’re not trying to cram all of it into your brain at once. Its structure is navigable, and you only retrieve the information you want.
Similarly, the world’s complexity is high, but it seems decomposable, into small modules that could be understood separately and navigated to locate specific knowledge.
Yep, I know of this result. I haven’t looked into it in depth, but my understanding is that it only says that powerful predictors have to be “complex” in the sense of high Kolmogorov complexity, right? But “high K-complexity” doesn’t mean “is a monolithic, irreducibly complex mess”. In particular, it doesn’t rule out this property:
Wikipedia has pretty high K-complexity, well beyond the ability of the human mind to hold in its working memory. But it’s still usable, because you’re not trying to cram all of it into your brain at once. Its structure is navigable, and you only retrieve the information you want.
Similarly, the world’s complexity is high, but it seems decomposable, into small modules that could be understood separately and navigated to locate specific knowledge.