I think that there are some abstractions that aren’t predictively useful, but are still useful in deciding your actions.
Suppose I and my friend both have the goal of maximising the number of DNA strings whose MD5 hash is prime.
I call sequences with this property “ana” and those without this property “kata”. Saying that “the DNA over there is ana” does tell me something about the world, there is an experiment that I can do to determine if this is true or false, namely sequencing it and taking the hash. The concept of “ana” isn’t useful in a world where no agents care about it and no detectors have been built. If your utility function cares about the difference, it is a useful concept. If someone has connected an ana detector to the trigger of something important, then its a useful concept. If your a crime scene investigator, and all you know about the perpetrators DNA is that its ana, then finding out if Joe Blogs has ana DNA could be important. The concept of ana is useful. If you know the perpitrators entire genome, the concept stops being useful.
A general abstraction is consistent with several, but not all universe states. There are many different universe states in which the gas has a pressure of 37Pa, but also many where it isn’t. So all abstractions are subsets of possible universe states. Usually, we use subsets that are suitable for reasoning about in some way.
Suppose you were literally omniscient, knowing every detail of the universe, but you had to give humans a 1Tb summary. Unable to include all the info you might want, you can only include a summery of the important points, you are now engaged in lossy compression.
Sensor data is also an abstraction, for instance you might have temperature and pressure sensors. Cameras record roughly how many photons hit them without tracking every one. So real world agents are translating one lossy approximation of the world into another without ever being able to express the whole thing explicitly.
How you do lossy compression depends on what you want. Music is compressed in a way that is specific to defects in human ears. Abstractions are much the same.
How you do lossy compression depends on what you want.
I think this is technically true, but less important than it seems at first glance. Natural abstractions are a thing, which means there’s instrumental convergence in abstractions—some compressed information is relevant to a far wider variety of objectives than other compressed information. Representing DNA sequences as strings of four different symbols is a natural abstraction, and it’s useful for a very wide variety of goals; MD5 hashes of those strings are useful only for a relatively narrow set of goals.
Somewhat more formally… any given territory has some Kolmogorov complexity, a maximally-compressed lossless map. That’s a property of the territory alone, independent of any goal. But it’s still relevant to goal-specific lossy compression—it will very often be useful for lossy models to re-use the compression methods relevant to lossless compression.
For instance, maybe we have an ASCII text file which contains only alphanumeric and punctuation characters. We can losslessly compress that file using e.g. Huffman coding, which uses fewer bits for the characters which appear more often. Now we decide to move on to lossy encoding—but we can still use the compressed character representation found by Huffman, assuming the lossy method doesn’t change the distribution of characters too much.
An abstraction like “object permanence” would be useful for a very wide variety of goals, maybe even for any real-world goal. An abstraction like “golgi apparatus” is useful for some goals but not others. “Lossless” is not an option in practice: our world is too rich, you can just keep digging deeper into any phenomenon until you run out of time and memory … I’m sure that a 50,000 page book could theoretically be written about earwax, and it would still leave out details which for some goals would be critical. :-)
I think that there are some abstractions that aren’t predictively useful, but are still useful in deciding your actions.
Suppose I and my friend both have the goal of maximising the number of DNA strings whose MD5 hash is prime.
I call sequences with this property “ana” and those without this property “kata”. Saying that “the DNA over there is ana” does tell me something about the world, there is an experiment that I can do to determine if this is true or false, namely sequencing it and taking the hash. The concept of “ana” isn’t useful in a world where no agents care about it and no detectors have been built. If your utility function cares about the difference, it is a useful concept. If someone has connected an ana detector to the trigger of something important, then its a useful concept. If your a crime scene investigator, and all you know about the perpetrators DNA is that its ana, then finding out if Joe Blogs has ana DNA could be important. The concept of ana is useful. If you know the perpitrators entire genome, the concept stops being useful.
A general abstraction is consistent with several, but not all universe states. There are many different universe states in which the gas has a pressure of 37Pa, but also many where it isn’t. So all abstractions are subsets of possible universe states. Usually, we use subsets that are suitable for reasoning about in some way.
Suppose you were literally omniscient, knowing every detail of the universe, but you had to give humans a 1Tb summary. Unable to include all the info you might want, you can only include a summery of the important points, you are now engaged in lossy compression.
Sensor data is also an abstraction, for instance you might have temperature and pressure sensors. Cameras record roughly how many photons hit them without tracking every one. So real world agents are translating one lossy approximation of the world into another without ever being able to express the whole thing explicitly.
How you do lossy compression depends on what you want. Music is compressed in a way that is specific to defects in human ears. Abstractions are much the same.
I think this is technically true, but less important than it seems at first glance. Natural abstractions are a thing, which means there’s instrumental convergence in abstractions—some compressed information is relevant to a far wider variety of objectives than other compressed information. Representing DNA sequences as strings of four different symbols is a natural abstraction, and it’s useful for a very wide variety of goals; MD5 hashes of those strings are useful only for a relatively narrow set of goals.
Somewhat more formally… any given territory has some Kolmogorov complexity, a maximally-compressed lossless map. That’s a property of the territory alone, independent of any goal. But it’s still relevant to goal-specific lossy compression—it will very often be useful for lossy models to re-use the compression methods relevant to lossless compression.
For instance, maybe we have an ASCII text file which contains only alphanumeric and punctuation characters. We can losslessly compress that file using e.g. Huffman coding, which uses fewer bits for the characters which appear more often. Now we decide to move on to lossy encoding—but we can still use the compressed character representation found by Huffman, assuming the lossy method doesn’t change the distribution of characters too much.
An abstraction like “object permanence” would be useful for a very wide variety of goals, maybe even for any real-world goal. An abstraction like “golgi apparatus” is useful for some goals but not others. “Lossless” is not an option in practice: our world is too rich, you can just keep digging deeper into any phenomenon until you run out of time and memory … I’m sure that a 50,000 page book could theoretically be written about earwax, and it would still leave out details which for some goals would be critical. :-)