Another excellent post! Building up from heuristics is a particularly novel piece—it’s something other work has done, but usually not at the same time as anything else. You’re integrating things together into a single coherent model.
One thing you kinda skipped over which I’d highlight:
And since any given system starts out not knowing what statistical correlations its selection pressure depends on, it’s convergent to try to derive every statistical correlation it can — build a complete world-model, then search it for the sources of feedback it receives.
This applies mainly when the system doesn’t know, in advance, which information it will need later on. If the system always needs the same bit, and can ignore everything else, then the argument doesn’t work so well. (I see this as a core idea of the Gooder Regulator Theorem.)
Other than that, the main next question I’d suggest is: how can you take this model, and turn it into a world-model-detector/extractor which we could point at some system in practice? What other pieces are missing?
Another excellent post! Building up from heuristics is a particularly novel piece—it’s something other work has done, but usually not at the same time as anything else. You’re integrating things together into a single coherent model.
One thing you kinda skipped over which I’d highlight:
This applies mainly when the system doesn’t know, in advance, which information it will need later on. If the system always needs the same bit, and can ignore everything else, then the argument doesn’t work so well. (I see this as a core idea of the Gooder Regulator Theorem.)
Other than that, the main next question I’d suggest is: how can you take this model, and turn it into a world-model-detector/extractor which we could point at some system in practice? What other pieces are missing?