We will need to do more than just make an effort to solve the problem of interpretability with current machine learning models. We need to work more on constructing new AI systems that are inherently more decipherable and mathematical than the AI systems that we have now and which organize the parameters of the model in a way that is inherently more understandable to experts with the right tools. Hopefully this will work much better than just being a tradeoff between performance and decipherability.
For example, a vector-valued word embedding is problematic since one will represent many different meanings of a token with a vector. One should instead have a word embedding where the individual meanings of the tokens are represented as vectors while the token itself represents text as a matrix (since matrices are very good at representing collections of vectors).
Areas of mathematics such as complex analysis and quantum information theory have many very nice theorems (such as the uniformization theorem), and such theorems should be applied to create more mathematical machine learning models which are easier to interpret due to their mathematical behavior.
Added 5/25/2023
It is also a good idea for the random variable XA of models trained using the training set A to have a small amount of entropy (there are several measures of entropy to choose from). If XA has high entropy, then one will need to sort through all of this random information in order to interpret the trained model. And after one is done interpreting the model, there is more work to do because if we train the model again with different initial conditions, then we will need to rework our initial interpretation for our retrained model. It may be too much to ask for the random variable XA to have zero entropy for all cases, but we should find as many use cases as possible when XA has little or no entropy. And if it is not feasible for neural networks to have zero entropy, then we should search for new kinds of machine learning models (I never said that this is going to be easy, but it is something that we need to do).
We will need to do more than just make an effort to solve the problem of interpretability with current machine learning models. We need to work more on constructing new AI systems that are inherently more decipherable and mathematical than the AI systems that we have now and which organize the parameters of the model in a way that is inherently more understandable to experts with the right tools. Hopefully this will work much better than just being a tradeoff between performance and decipherability.
For example, a vector-valued word embedding is problematic since one will represent many different meanings of a token with a vector. One should instead have a word embedding where the individual meanings of the tokens are represented as vectors while the token itself represents text as a matrix (since matrices are very good at representing collections of vectors).
Areas of mathematics such as complex analysis and quantum information theory have many very nice theorems (such as the uniformization theorem), and such theorems should be applied to create more mathematical machine learning models which are easier to interpret due to their mathematical behavior.
Added 5/25/2023
It is also a good idea for the random variable XA of models trained using the training set A to have a small amount of entropy (there are several measures of entropy to choose from). If XA has high entropy, then one will need to sort through all of this random information in order to interpret the trained model. And after one is done interpreting the model, there is more work to do because if we train the model again with different initial conditions, then we will need to rework our initial interpretation for our retrained model. It may be too much to ask for the random variable XA to have zero entropy for all cases, but we should find as many use cases as possible when XA has little or no entropy. And if it is not feasible for neural networks to have zero entropy, then we should search for new kinds of machine learning models (I never said that this is going to be easy, but it is something that we need to do).