I suspect language model in-context learning[1] ‘approximates Solomonoff induction’ in the vague sense that it is a pattern matching thingy navigating a search space somewhat similar in character to the space of possible computer programs, consisting of inputs/parameters for some very universal, Turing-complete-ish computational architecture the lm expresses its guesses for patterns in, looking for a pattern that matches the data.
The way they navigate this search space is totally different from SI, which just checks every single point in its search space of UTM programs. But the geometry of the space is similar to the geometry of the space of UTM programs, with properties like simpler hypotheses corresponding to exponentially more points in the space.
So, even if the language models’ in-context learning algorithm was kind of maximally stupid, and literally just guessed random points in the search space until it found a good match to the data, we’d expect its outputs to somewhat match up with the universal distribution, just because they’re both ≈ uniformly random samples from a space of inputs to Turing-complete-ish computational architectures.
So, to the extent that these experimental results actually hold up[2], I think the main thing they’d be telling us is that the ‘architecture’ or ‘language’ the lm expresses its in-context guesses in is highly expressive, with a computational universality similar to that of UTMs and many neural network architectures.
Arguably, the later may be a special case of the former with an appropriate choice of universal Turing machine (UTM), but I find this perspective to be a bit of a stretch. At the very least I expect LLM ICL to be similar to a universal distribution conditioned on some background information.
What’s even the difference between these propositions? Any UTM can be expressed in another UTM as a bit string of prior knowledge to condition on, and I’d intuitively expect the reverse to hold as well, though I don’t actually know that for sure.
There are pretty strong reasons to expect that neither direction (conditioning or switching UTM) perfectly simulates the other. I think one of the two directions is known to be impossible—that conditioning cannot be replaced by switching UTM.
I guess I wouldn’t expect UTM switching to be able to express any conditioning, that wouldn’t make sense since conditioning can exclude TMs and UTMs can all express any TM. But that doesn’t strike me as the sort of conditioning prior knowledge of the internet would impose?
Actually, now that I think about it, I guess it could be.
I suspect language model in-context learning[1] ‘approximates Solomonoff induction’ in the vague sense that it is a pattern matching thingy navigating a search space somewhat similar in character to the space of possible computer programs, consisting of inputs/parameters for some very universal, Turing-complete-ish computational architecture the lm expresses its guesses for patterns in, looking for a pattern that matches the data.
The way they navigate this search space is totally different from SI, which just checks every single point in its search space of UTM programs. But the geometry of the space is similar to the geometry of the space of UTM programs, with properties like simpler hypotheses corresponding to exponentially more points in the space.
So, even if the language models’ in-context learning algorithm was kind of maximally stupid, and literally just guessed random points in the search space until it found a good match to the data, we’d expect its outputs to somewhat match up with the universal distribution, just because they’re both ≈ uniformly random samples from a space of inputs to Turing-complete-ish computational architectures.
So, to the extent that these experimental results actually hold up[2], I think the main thing they’d be telling us is that the ‘architecture’ or ‘language’ the lm expresses its in-context guesses in is highly expressive, with a computational universality similar to that of UTMs and many neural network architectures.
What’s even the difference between these propositions? Any UTM can be expressed in another UTM as a bit string of prior knowledge to condition on, and I’d intuitively expect the reverse to hold as well, though I don’t actually know that for sure.
and human thought too, I’d guess
I have not actually looked at your numerical results closely at all, sorry.
There are pretty strong reasons to expect that neither direction (conditioning or switching UTM) perfectly simulates the other. I think one of the two directions is known to be impossible—that conditioning cannot be replaced by switching UTM.
I guess I wouldn’t expect UTM switching to be able to express any conditioning, that wouldn’t make sense since conditioning can exclude TMs and UTMs can all express any TM. But that doesn’t strike me as the sort of conditioning prior knowledge of the internet would impose?
Actually, now that I think about it, I guess it could be.