I am kind of suprised you didn’t reference causal inference here to just gesture at the task in which we “figure out which variables are directly relevant—i.e. which variables mediate the influence of everything else”. Are you pointing to a different sort of idea/do you not feel causal inference is adequate for describing this task?
Also, scenario 1 and 2 seem fairly close to the “linear” and “non-linear” models of innovation Jason Crawford described in his talk “TheNon-Linear Model of Innovation.” To be honest, I prefered his description of the models. Though he didn’t cover how miraculous it is that somehow the model can work. That, to a good approximation, the universe is simple and local.
Causal inference (or more precisely learning causal structure) is exactly the sort of thing I have in mind here. There’s actually a few places in the post where I should distinguish between variables which control an outcome in an information sense (i.e. sufficient to perfectly predict the outcome) vs in a causal sense (i.e. sufficient to cause the outcome under interventions). The main reason I didn’t talk about it directly is because I would have had to explain that distinction, and decided that would be too much of a distraction from the main point.
I think the takeaway of Jason’s talk, as it relates to this post, is that a large chunk of the “science” of achieving consistent outcomes happens in inventors’ workshops rather than scientists’ labs. The problem is still largely similar, regardless of the label applied, but scientists aren’t the only ones doing science.
I am kind of suprised you didn’t reference causal inference here to just gesture at the task in which we “figure out which variables are directly relevant—i.e. which variables mediate the influence of everything else”. Are you pointing to a different sort of idea/do you not feel causal inference is adequate for describing this task?
Also, scenario 1 and 2 seem fairly close to the “linear” and “non-linear” models of innovation Jason Crawford described in his talk “The Non-Linear Model of Innovation.” To be honest, I prefered his description of the models. Though he didn’t cover how miraculous it is that somehow the model can work. That, to a good approximation, the universe is simple and local.
Causal inference (or more precisely learning causal structure) is exactly the sort of thing I have in mind here. There’s actually a few places in the post where I should distinguish between variables which control an outcome in an information sense (i.e. sufficient to perfectly predict the outcome) vs in a causal sense (i.e. sufficient to cause the outcome under interventions). The main reason I didn’t talk about it directly is because I would have had to explain that distinction, and decided that would be too much of a distraction from the main point.
I think the takeaway of Jason’s talk, as it relates to this post, is that a large chunk of the “science” of achieving consistent outcomes happens in inventors’ workshops rather than scientists’ labs. The problem is still largely similar, regardless of the label applied, but scientists aren’t the only ones doing science.