I agree with your point regarding ML’s historical focus on blackbox prediction. That said, there has been some intriguing recent work (example 1, example 2), which I’ve only just started looking at, on trying to learn causal models.
I bring this up because I think the question of how causal model learning happens and how learning systems can do it may potentially be relevant to the work you’ve been writing about. It’s primarily of interest to me for different reasons, related to applying ML systems to scientific discovery. In particular, in domains where coming up with causal hypotheses is harder at scale.
Nice links. I actually stopped following deep learning for a few years, and very recently started paying attention again as the new generation of probabilistic programming languages came along (I’m particularly impressed with pyro). Those tools are a major step forward for learning causal structure.
I’d also recommend this recent paper by Friston (the predictive processing guy). I might write up a review of it soonish; it’s a really nice piece of math/algorithm for learning causal structure, again using the same ML tools.
Good post.
I agree with your point regarding ML’s historical focus on blackbox prediction. That said, there has been some intriguing recent work (example 1, example 2), which I’ve only just started looking at, on trying to learn causal models.
I bring this up because I think the question of how causal model learning happens and how learning systems can do it may potentially be relevant to the work you’ve been writing about. It’s primarily of interest to me for different reasons, related to applying ML systems to scientific discovery. In particular, in domains where coming up with causal hypotheses is harder at scale.
Nice links. I actually stopped following deep learning for a few years, and very recently started paying attention again as the new generation of probabilistic programming languages came along (I’m particularly impressed with pyro). Those tools are a major step forward for learning causal structure.
I’d also recommend this recent paper by Friston (the predictive processing guy). I might write up a review of it soonish; it’s a really nice piece of math/algorithm for learning causal structure, again using the same ML tools.