In a sense, it is extremely natural and obvious that any system handling sophisticated problems will be doing different things when handling different problems! But there is also a starting point from which this can be somewhat surprising: if you think of a neural net as a circuit (either just manifestly, or under some translation), then maybe you’d expect the same variables to be computed on each forward pass? It could be helpful here to consider how a Turing machine with a runtime bound can always be unrolled into a circuit that simulates [the contents of its tape and the position of its pointer] at all time steps.6 Whether tape cell 13 has a 0 or a 1 written on it at time step 42 is in one sense the same variable on any input, but in another sense it can easily represent very different variables of the program on different inputs.
Two remarks on how the current (March 2026) field trying to understand what AIs are doing relates to the issue of an AI doing different things on different inputs:
The currently prevailing view in interpretability allows for this to some extent: it is common to think of a big transformer language model as doing various different things depending on the context/input. But the prevailing view still takes there to ultimately be some pre-determined finite list of variables (I mean: corresponding to SAE features) that could be getting determined in a model, and I think this is probably a defect of that view, because a system solving an open-ended variety of complicated problems should be able to determine [what auxiliary problems to solve]/[what auxiliary questions to answer] on the fly.7 (I should note: maybe it is not clear that a forward pass of a transformer is sophisticated enough for this to be true of it?)
As one of the “finite list of variables” people[1]: This is because at the moment, I primarily want to find and understand the variables underlying the general mechanisms which AIs use to have many different kinds of productive thoughts in the first place. I am not particularly trying to find and understand variables defined only within the causal structure of these thoughts. I believe the former might indeed be described as a pre-determined finite list of variables. I agree the latter can’t be, at least not usefully.[2]
To use your analogy: I think of myself as trying to understand something like the basic makeup of a UTM, figuring out the tapes, heads, registers, tables and so on. I am not yet trying to say very much about the inner structures of the many different programs that could be run on that UTM.
I agree that some “finite list of variables” people seem to me to not distinguish between these different levels. I think that this is probably a mistake.
As one of the “finite list of variables” people[1]: This is because at the moment, I primarily want to find and understand the variables underlying the general mechanisms which AIs use to have many different kinds of productive thoughts in the first place. I am not particularly trying to find and understand variables defined only within the causal structure of these thoughts. I believe the former might indeed be described as a pre-determined finite list of variables. I agree the latter can’t be, at least not usefully.[2]
To use your analogy: I think of myself as trying to understand something like the basic makeup of a UTM, figuring out the tapes, heads, registers, tables and so on. I am not yet trying to say very much about the inner structures of the many different programs that could be run on that UTM.
I agree that some “finite list of variables” people seem to me to not distinguish between these different levels. I think that this is probably a mistake.
Loosely speaking.
With a finite context window and finite external memory there is technically a ceiling on how many different thoughts an AI is capable of having.