Deep learning systems are feasible and powerful but not legible (the hope of interpretability is to change that).
Imo “legible” is a bit of an aggregate rather than a single concept. Is my JS legible to me, if I don’t understand microchips? Is a sheep legible, to a sheep farmer, to a vet, or to a random person? Is a tree legible to us right now?
Thanks, this is a good objection, and I don’t have a good distinction. Now I’m actually not sure that Solomonoff induction is legible according to my original definitions, as the shortest program for a given sequence will likely be just as obfuscated and hard to understand as neural networks.
I would like to replace “legible” with something on the lines of: “can be analyzed using formal methods that show what it can or can’t do”; but this is still a messy definition, and it might exclude e.g. SAT Solvers (which are a formal tool, but the question of which SAT instances can be solved efficiently by it still involves heuristics and is not fully understood).
Imo “legible” is a bit of an aggregate rather than a single concept. Is my JS legible to me, if I don’t understand microchips? Is a sheep legible, to a sheep farmer, to a vet, or to a random person? Is a tree legible to us right now?
Thanks, this is a good objection, and I don’t have a good distinction. Now I’m actually not sure that Solomonoff induction is legible according to my original definitions, as the shortest program for a given sequence will likely be just as obfuscated and hard to understand as neural networks.
I would like to replace “legible” with something on the lines of: “can be analyzed using formal methods that show what it can or can’t do”; but this is still a messy definition, and it might exclude e.g. SAT Solvers (which are a formal tool, but the question of which SAT instances can be solved efficiently by it still involves heuristics and is not fully understood).