I think you and John are talking about two different facets of interpretability.
The first one is the question of “white-boxing:” how do the model’s internal components interrelate to produce its output? On this dimension, the kind of models that you’ve given as examples are much more interpretable than neural networks.
What I think John is talking about, I understand as “grounding.” (Cf. Symbol grounding problem) Although the decision tree (a) above is clear in that one can easily follow how the final decision comes about, the question remains—who or what makes sure that the labels in the boxes correspond to features of the real world that we would also describe by those labels? So I think the claim is that on this dimension of interpretability, neural networks and logical/probabilistic models are more similar.
This is the focus of General Systems, as outlined by Weinberg. That book is very good, by the way—I highly recommend reading it. It’s both very dense and very accessible.
It’s always puzzled me that the rationalist community hasn’t put more emphasis on general systems. It seems like it should fit in perfectly, but I haven’t seen anyone mention it explicitly. General Semantics mentioned in the recent historical post is somewhat related, but not the same thing.
More on topic: One thing you don’t mention is that there are fairly general problem solving techniques, which start before, and are relatively independent of, your level of specific technical knowledge. From what I’ve observed, most people are completely lost when approaching a new problem, because they don’t even know where to start. So as well as your suggestion of focusing on learning the existence of techniques and when they apply, you can also directly focus on learning problem solving approaches.