This sounds very interesting and I’d be very excited to hear the results of your work. I have a lot of random disorganized thoughts on the matter which I’ll lay out here in case some of them are valuable.
I wonder if, for ordinary neural networks, something like bottlenecking the network at one or more of the layers would force abstraction.
This makes me think of autoencoders trying to compress information, which leads to an interesting question: is there a general way to “translate” between autoencoders trained on the same dataset? By this I mean having a simple function (like an individual matrix) between the first half of one autoencoder and the second half of another. If there is this would give evidence that they are using the same abstractions.
This also reminds me of that essay about the tails coming apart, which suggests to me that the abstractions a system will use will depend on the dataset, the outcome being predicted, and also perhaps the size and capabilities of the model (a bigger model might make more accurate predictions by splitting “grip strength” and “arm strength” apart but a smaller model might have to combine them). This seems to be related to the dimensionality points you’ve mentioned, where the specific abstractions used depend on the number of abstractions a model is allowed to use.
This makes me think of Principal Component Analysis in statistics, which has a similar vibe to the natural abstraction hypothesis in that it involves compressing statistical information onto a smaller number of dimensions, exactly how many dimensions depends on the statistical methods you are using.
In the real world, the classic examples of abstractions are stuff like “temperature of a gas” which involves throwing away something like 180 bits of information per individual gas molecule (if my memory of statistical mechanics is correct), while still letting you predict internal energy, pressure, how it will flow. Abstractions for other systems are unlikely to be as clear-cut: we can probably not compress 10^25ish bits of information into a small number of bits of information, for the average system. For exampe I think that about 8 characteristics of different plant species (seed size, height, leaf thickness etc.) can be compressed onto two dimensions which contain about 80% of the variation in the data, but it’s not immediately clear why we ought to stop there, or indeed use a second dimension when one would presumably contain >40% of the variation.
Finally I suspect that the name “abstraction thermometer” is underselling the capabilities of what you describe. Finding all the abstractions of any given system is incredibly powerful. For example one set of abstractions which would predict the progression of a disease would be the set of the pathogens, proteins, and small molecules which can cause that disease. If the natural abstraction hypothesis is true (and in cases like this it would seem to be) then an “abstraction thermometer” is in this case able to find out everything about the biological system in question, and would therefore give us an incredible amount of knowledge.
This sounds very interesting and I’d be very excited to hear the results of your work. I have a lot of random disorganized thoughts on the matter which I’ll lay out here in case some of them are valuable.
I wonder if, for ordinary neural networks, something like bottlenecking the network at one or more of the layers would force abstraction.
This makes me think of autoencoders trying to compress information, which leads to an interesting question: is there a general way to “translate” between autoencoders trained on the same dataset? By this I mean having a simple function (like an individual matrix) between the first half of one autoencoder and the second half of another. If there is this would give evidence that they are using the same abstractions.
This also reminds me of that essay about the tails coming apart, which suggests to me that the abstractions a system will use will depend on the dataset, the outcome being predicted, and also perhaps the size and capabilities of the model (a bigger model might make more accurate predictions by splitting “grip strength” and “arm strength” apart but a smaller model might have to combine them). This seems to be related to the dimensionality points you’ve mentioned, where the specific abstractions used depend on the number of abstractions a model is allowed to use.
This makes me think of Principal Component Analysis in statistics, which has a similar vibe to the natural abstraction hypothesis in that it involves compressing statistical information onto a smaller number of dimensions, exactly how many dimensions depends on the statistical methods you are using.
In the real world, the classic examples of abstractions are stuff like “temperature of a gas” which involves throwing away something like 180 bits of information per individual gas molecule (if my memory of statistical mechanics is correct), while still letting you predict internal energy, pressure, how it will flow. Abstractions for other systems are unlikely to be as clear-cut: we can probably not compress 10^25ish bits of information into a small number of bits of information, for the average system. For exampe I think that about 8 characteristics of different plant species (seed size, height, leaf thickness etc.) can be compressed onto two dimensions which contain about 80% of the variation in the data, but it’s not immediately clear why we ought to stop there, or indeed use a second dimension when one would presumably contain >40% of the variation.
Finally I suspect that the name “abstraction thermometer” is underselling the capabilities of what you describe. Finding all the abstractions of any given system is incredibly powerful. For example one set of abstractions which would predict the progression of a disease would be the set of the pathogens, proteins, and small molecules which can cause that disease. If the natural abstraction hypothesis is true (and in cases like this it would seem to be) then an “abstraction thermometer” is in this case able to find out everything about the biological system in question, and would therefore give us an incredible amount of knowledge.