Good point. I think this is a pretty good argument against using this setup. You can kind of guard against this failure mode by only looking at circuits that are very deep relative to the width of the network, but it’s a hacky fix.
Good point. I think this is a pretty good argument against using this setup. You can kind of guard against this failure mode by only looking at circuits that are very deep relative to the width of the network, but it’s a hacky fix.