In case you haven’t seen it, it looks like there’s a separate strain of work somewhat along these lines from Zhi Chen and others, under the name “concept whitening.” You may already be aware of it, but I thought I’d pass it along in case not.
[Concept Whitening for Interpretable Image Recognition](https://arxiv.org/pdf/2002.01650.pdf)
In case you haven’t seen it, it looks like there’s a separate strain of work somewhat along these lines from Zhi Chen and others, under the name “concept whitening.” You may already be aware of it, but I thought I’d pass it along in case not.
[Concept Whitening for Interpretable Image Recognition](https://arxiv.org/pdf/2002.01650.pdf)