I mostly focused on the interpretability section as that’s what I’m most familiar with, and I think your criticisms are very valid. I also wrote up some thoughts recently on where post-hoc interpretability fails, and Daniel Filan has some good responses in the comments below.
Also, re: disappointment on tree regularization, something that does seem more promising is Daniel Filan and others at CHAI working on investigating modularity in neural nets. You can probably ask him more, but last time we chatted, he also had some thoughts (unpublished) on how to enforce modularization as a regularizer, which seems to be what you wished the tree reg paper would have done.
Overall, this is great stuff, and I’ll need to spend more time thinking about the design vs search distinction (which makes sense to me at first glance)/
I mostly focused on the interpretability section as that’s what I’m most familiar with, and I think your criticisms are very valid. I also wrote up some thoughts recently on where post-hoc interpretability fails, and Daniel Filan has some good responses in the comments below.
Also, re: disappointment on tree regularization, something that does seem more promising is Daniel Filan and others at CHAI working on investigating modularity in neural nets. You can probably ask him more, but last time we chatted, he also had some thoughts (unpublished) on how to enforce modularization as a regularizer, which seems to be what you wished the tree reg paper would have done.
Overall, this is great stuff, and I’ll need to spend more time thinking about the design vs search distinction (which makes sense to me at first glance)/
Nice write-up. The note about adversarial examples for LIME and SHAP was not something I’ve come across before—very cool.
Thanks for the pointer to Daniel Filan’s work—that is indeed relevant and I hadn’t read the paper before now.