Huh. This was very different from the role I originally imagined for this post.
My reading of Gears in Understanding was that it was trying to gesture at a concept, and give that concept a handle, without really explaining or “defining” the concept. It gave some heuristics for recognizing gears-level models in the wild (e.g. “could the model be correct but a given variable be different?” or “if the model were falsified, what else could we infer from that?”), but it never really said why those heuristics should all correlate with each other, or why/how they’re all pointing to the same thing, or why that thing would be useful/interesting. It’s just trying to point to a particular cluster in concept-space, without trying to explain why there’s a cluster there, or precisely delineate the cluster.
Gears vs Behavior is trying to (at least partially) explain why there’s a cluster there and draw a box around it. It’s presenting a frame in which the key, defining property which separates gears-level models from purely behavioral/black box models is that gearsy models can make predictions and leverage data from side-channels and out-of-distribution (which is itself a pseudo-side-channel). Heuristics like “if the model were falsified, what else could we infer from that?” follow from the ability to leverage side-channels. I claim that this is pointing to the same thing Val was trying to point to, and can be usefully viewed as the main defining property of gearsy models (as opposed to blackboxes).
This post comes across as simpler because it’s focused on the key defining property of the cluster, rather than trying to gesture at the concept from a bunch of different directions without really figuring out what the “defining” feature is.
At this point, I actually think a better way to “define” gears-level models is the dimensionality and conditional independence framework, although I still see Gears vs Behavior as a basically-correct “dual” to those frames, focusing on the space of queries that models address, rather than the structures of the models themselves. (In some sense, dimensionality and conditional independence give a gears-level model of gears-level models, while Gears vs Behavior gives a more blackboxy definition of gears-level models.)
Huh. This was very different from the role I originally imagined for this post.
My reading of Gears in Understanding was that it was trying to gesture at a concept, and give that concept a handle, without really explaining or “defining” the concept. It gave some heuristics for recognizing gears-level models in the wild (e.g. “could the model be correct but a given variable be different?” or “if the model were falsified, what else could we infer from that?”), but it never really said why those heuristics should all correlate with each other, or why/how they’re all pointing to the same thing, or why that thing would be useful/interesting. It’s just trying to point to a particular cluster in concept-space, without trying to explain why there’s a cluster there, or precisely delineate the cluster.
Gears vs Behavior is trying to (at least partially) explain why there’s a cluster there and draw a box around it. It’s presenting a frame in which the key, defining property which separates gears-level models from purely behavioral/black box models is that gearsy models can make predictions and leverage data from side-channels and out-of-distribution (which is itself a pseudo-side-channel). Heuristics like “if the model were falsified, what else could we infer from that?” follow from the ability to leverage side-channels. I claim that this is pointing to the same thing Val was trying to point to, and can be usefully viewed as the main defining property of gearsy models (as opposed to blackboxes).
This post comes across as simpler because it’s focused on the key defining property of the cluster, rather than trying to gesture at the concept from a bunch of different directions without really figuring out what the “defining” feature is.
At this point, I actually think a better way to “define” gears-level models is the dimensionality and conditional independence framework, although I still see Gears vs Behavior as a basically-correct “dual” to those frames, focusing on the space of queries that models address, rather than the structures of the models themselves. (In some sense, dimensionality and conditional independence give a gears-level model of gears-level models, while Gears vs Behavior gives a more blackboxy definition of gears-level models.)