composeable, that is each circuit takes inputs of the same type as outputs of other circuits in the network
I think this is kinda enforced by having the residual stream, or specifically, having it as the main information highway flowing through the entire network?
Experiments where swapping nearby layers of a network has very little impact on performance also suggest this.
(Maybe this explains the some of the “sameness”/”slop”-factor of LLM outputs, the “semantic type” has to match?)
I think this is kinda enforced by having the residual stream, or specifically, having it as the main information highway flowing through the entire network?
I was talking about the kinds of tokens that are output, more in this comment. I mostly think of one forward pass as being one circuit, but there may be some structure in the internal information flow that I’m not privvy to.
Why would you expect different “types” to help?
I think this is mainly a question of whether there’s capacity for circuits in the model to handle different kinds of text (like OCR errors in medieval manuscripts, usenet archive formatting details &c), vs. being mode collapsed. I guess more obscure text formats are less connected (correlated?) to circuits that are capable at solving complex problems, and fewer serial steps have to be performed on translating from one “textual ontology” into another one. (This is all counterbalanced by the need to pack as much information as possible into the next token, but my guess is that over time RLVR will add more structure/details/entropy to the Markdown-in-English chains-of-thought, instead of e.g. repurposing something like a circuit responsible for representing little bits of Yi script, which at least Llama3-405b-base can do.)
On first read, I broadly agree with this model.
I think this is kinda enforced by having the residual stream, or specifically, having it as the main information highway flowing through the entire network?
Experiments where swapping nearby layers of a network has very little impact on performance also suggest this.
Why would you expect different “types” to help?
I was talking about the kinds of tokens that are output, more in this comment. I mostly think of one forward pass as being one circuit, but there may be some structure in the internal information flow that I’m not privvy to.
I think this is mainly a question of whether there’s capacity for circuits in the model to handle different kinds of text (like OCR errors in medieval manuscripts, usenet archive formatting details &c), vs. being mode collapsed. I guess more obscure text formats are less connected (correlated?) to circuits that are capable at solving complex problems, and fewer serial steps have to be performed on translating from one “textual ontology” into another one. (This is all counterbalanced by the need to pack as much information as possible into the next token, but my guess is that over time RLVR will add more structure/details/entropy to the Markdown-in-English chains-of-thought, instead of e.g. repurposing something like a circuit responsible for representing little bits of Yi script, which at least Llama3-405b-base can do.)