We can compare interpreting neural networks to doing physics: you can look at the lowest level description of the system, or you can look for patterns in higher levels of abstraction. In higher levels, we usually look for explanations that are good enough to some approximation. In physics this is pretty successful—Newtonian physics is useful for many practical purposes even though it’s just an approximation. The analogous approach to AI is discovering behavioral patterns, testing that they are predictive for new experiments, and growing a set of heuristics about how NNs behave. Examples of this include much of Owain’s group’s work, or inoculation prompting. I think this approach has a pretty good track record.
We can compare interpreting neural networks to doing physics: you can look at the lowest level description of the system, or you can look for patterns in higher levels of abstraction. In higher levels, we usually look for explanations that are good enough to some approximation. In physics this is pretty successful—Newtonian physics is useful for many practical purposes even though it’s just an approximation. The analogous approach to AI is discovering behavioral patterns, testing that they are predictive for new experiments, and growing a set of heuristics about how NNs behave. Examples of this include much of Owain’s group’s work, or inoculation prompting. I think this approach has a pretty good track record.
And if you want to do literally statistical physics on top of deep learning: https://deeplearningtheory.com