Reading this, it dawned on me that we tend to talk about human-understandability of NNs as if it was magic.[1] I mean, how much is there to human-understandability—in the sense of sufficiently intelligent (but not extremely rare genius-level) humans being able to work with a phenomenon and model it accurately on a sufficient level of granularity—beyond complexity of that phenomenon and computational reducibility of its dynamics?
Sure, we find some ways of representing information and thinking more intuitive and easier to grasp than others but we can learn new ones that would seem pretty alien to a naive hunter-gatherer mind, like linear algebra or category theory.
Reading this, it dawned on me that we tend to talk about human-understandability of NNs as if it was magic.[1] I mean, how much is there to human-understandability—in the sense of sufficiently intelligent (but not extremely rare genius-level) humans being able to work with a phenomenon and model it accurately on a sufficient level of granularity—beyond complexity of that phenomenon and computational reducibility of its dynamics?
Sure, we find some ways of representing information and thinking more intuitive and easier to grasp than others but we can learn new ones that would seem pretty alien to a naive hunter-gatherer mind, like linear algebra or category theory.
Or perhaps it’s just me and this is background knowledge for everybody else, whatever.