I agree this isn’t easy. In this specific case, the construction is useful to me because it is fairly general, applying to all circuits with , has no noise, making it easier to reason about in many ways, and runs on a pretty different principle than all computation in superposition constructions I’d thought about before seeing this.
One construction that occurred to me that’s making me doubt the problem setup a little at least for :
You can abuse the linear circuit embedding associated with circuit to communicate arbitrary binary codes to the bottom stream provided that you assume the sum of the input coordinates is bounded away from (Analogous to the bloom filter scenario). So transmit a binary code encoding the weights of the circuit and you win by just writing a generic circuit emulator and lose the dependence on (Potentially need bits if you wanna encode output in superposition).
I think this is obviously pretty silly because in a real network you’d have to store the bits associated with the circuits somewhere and it’s just a quirk of the precise setup.
Wonder what you think of instead assuming smth like the residual stream takes the form of concatenated blocks of the form (input | dense binary code of length log(T) encoding the circuit). So making the formatting rigid to avoid encoding too much info in the weight matrices themselves.
Good point. I think this is a pretty good argument against using this setup. You can kind of guard against this failure mode by only looking at circuits that are very deep relative to the width of the network, but it’s a hacky fix.
I agree this isn’t easy. In this specific case, the construction is useful to me because it is fairly general, applying to all circuits with
One construction that occurred to me that’s making me doubt the problem setup a little at least for
You can abuse the linear circuit embedding
I think this is obviously pretty silly because in a real network you’d have to store the bits associated with the circuits somewhere and it’s just a quirk of the precise setup.
Wonder what you think of instead assuming smth like the residual stream takes the form of concatenated blocks of the form (input | dense binary code of length log(T) encoding the circuit). So making the formatting rigid to avoid encoding too much info in the weight matrices themselves.
In general transmitting binary codes to the bottom is fairly useful though. In the setup of Compressed computation (https://www.lesswrong.com/posts/ZxFchCFJFcgysYsT9/compressed-computation-is-probably-not-computation-in), feature
Good point. I think this is a pretty good argument against using this setup. You can kind of guard against this failure mode by only looking at circuits that are very deep relative to the width of the network, but it’s a hacky fix.