I agree “Figure out random messy stuff” is important. As I see it, there are two types of “random messy stuff”
Approximation error of a clean algorithm. The model doesn’t learn the “perfect circuit” since it doesn’t have to/because it is not overtrained.
Unavoidable errors because of (computations in) superposition. Compressions are more efficient but lossy.
I expect the second one to be more prevalent and also more exploitable, both in terms of generating more adversarial phenomena and containing more structure that can be used to efficiently examine errors.
When looking at the error of trained models, I don’t expect it will look Gaussian, but rather be of the second kind. Do you think that it is fine to model it as Gaussian (+ some higher order cumulants) anyway and/or that it is not tractable to do anything else?
I agree “Figure out random messy stuff” is important. As I see it, there are two types of “random messy stuff”
Approximation error of a clean algorithm. The model doesn’t learn the “perfect circuit” since it doesn’t have to/because it is not overtrained.
Unavoidable errors because of (computations in) superposition. Compressions are more efficient but lossy.
I expect the second one to be more prevalent and also more exploitable, both in terms of generating more adversarial phenomena and containing more structure that can be used to efficiently examine errors.
When looking at the error of trained models, I don’t expect it will look Gaussian, but rather be of the second kind. Do you think that it is fine to model it as Gaussian (+ some higher order cumulants) anyway and/or that it is not tractable to do anything else?