I feel like this is trying to apply a neural network where the problem specification says “please train a decision tree.” Even when you are fine with part of the NN not being sparse, it seems like you’re just using the gradient descent training as an elaborate img2vec method.
Maybe the idea is that you think a decision tree is too restrictive, and you want to allow more weightings and nonlinearities? Still, it seems like if you can specify from the top down what operations are “interpretable,” this will give you some tree-like structure that can be trained in a specialized way.
I feel like this is trying to apply a neural network where the problem specification says “please train a decision tree.” Even when you are fine with part of the NN not being sparse, it seems like you’re just using the gradient descent training as an elaborate img2vec method.
Maybe the idea is that you think a decision tree is too restrictive, and you want to allow more weightings and nonlinearities? Still, it seems like if you can specify from the top down what operations are “interpretable,” this will give you some tree-like structure that can be trained in a specialized way.