As an example, suppose we have a bacterium, which we are treating as a black box, and we are training an ML system to predict various things it does (movement, secretions etc.) from its environment (chemicals around it, temperature). Instead of just measuring inputs and outputs of the cell, we might do one of two things:
1: Mutate a gene in the bacterium to see how its behaviour changes (knockout)
2: Measure the concentration of a certain protein in the bacterium under different conditions (assay)
Once we have done this, we know a little bit of the internal structure: it is no longer a pure black box. Are there any methods for enforcing this structure onto an existing ML model? (In the case of 1 we could add the knockout status of the bacterium as an extra input node but that seems to be missing the point, and there’s no obvious way to do something similar for 2) Without this it seems like ML systems operating in the real world would be throwing away a lot of potential data. (I am mainly thinking of neural networks as these are undoubtedly the most commonly-used ML systems at the moment, but I would be very interested to hear about other systems)
You might be interested in the Circuits work (summarized here)