Some frames about abstractions & ontology shifts I had while thinking through similar problems (which you may have considered already):
The dual of “abstraction as redundant information across a wide variety of agents in the same environment” is “abstraction as redundant information/computation across a wide variety of hypotheses about the environment in an agent’s world model” (E.g. a strawberry is a useful concept to model for many worlds that I might be in). I think this is a useful frame when thinking about “carving up” the world model into concepts, since a concept needs to remain invariant while the hypothesis keeps being updated
The semantics of a component in a world model is partly defined by its relationship with the rest of the components (e.g. move a neuron to a different location and its activation will have a different meaning), so if you want a component to have stable semantics over time, you want to put the “relational/indexical information” inside the component itself
In particular, this means that when an agent acquires new concepts, the existing concepts should be able to “specify” how it should relate to that new concept (e.g. learning about chemistry then using it to deduce macro-properties of strawberries from molecular composition)
happy to discuss more via PM as some of my ideas seem exfohazardous
The dual of “abstraction as redundant information across a wide variety of agents in the same environment” is “abstraction as redundant information/computation across a wide variety of hypotheses about the environment in an agent’s world model”
Mm, this one’s shaky. Cross-hypothesis abstractions don’t seem to be a good idea, see here.
My guess is that there’s something like a hierarchy of hypotheses, with specific high-level hypotheses corresponding to several lower-level more-detailed hypotheses, and what you’re pointing at by “redundant information across a wide variety of hypotheses” is just an abstraction in a (single) high-level hypothesis which is then copied over into lower-level hypotheses. (E. g., the high-level hypothesis is the concept of a tree, the lower-level hypotheses are about how many trees are in this forest.) But we don’t derive it by generating a bunch of low-level hypotheses and then abstracting over them, that’d lead to broken ontologies.
The semantics of a component in a world model is partly defined by its relationship with the rest of the components
In particular, this means that when an agent acquires new concepts, the existing concepts should be able to “specify” how it should relate to that new concept
Yeah, this is probably handled by something like a system of types… which are themselves just higher-level abstractions. Like, if we discover a new thing, and then “realize” that it’s a fruit, we mentally classify it as an instance of the “fruit” concept, from which it then automatically inherits various properties (such as “taste” and “caloric content”).
“Truesight” likely enters the play here as well: we want to recognize instances of existing concepts, even if they were introduced to us by some new route (such as realizing that something is a strawberry by looking at its molecular description).
Mm, this one’s shaky. Cross-hypothesis abstractions don’t seem to be a good idea, see here.
yea so I think the final theory of abstraction will have a weaker notion of equivalence espeically when we incorporate ontology shifts. E.g. we want to say that water is the same concept before and after we discover water is H2O, but the discovery obviously breaks predictive agreement (Indeed, the solomonoff version of natural latent is more robust to the agreement condition)
Also, you can totally add new information/abstraction that is not shared between your current and new hypothesis, & that seems consistent with the picture you described here (you can have separate ontologies but you try to capture the overlap as much as possible)
My guess is that there’s something like a hierarchy of hypotheses, with specific high-level hypotheses corresponding to several lower-level more-detailed hypotheses, and what you’re pointing at by “redundant information across a wide variety of hypotheses” is just an abstraction in a (single) high-level hypothesis which is then copied over into lower-level hypotheses. (E. g., the high-level hypothesis is the concept of a tree, the lower-level hypotheses are about how many trees are in this forest.)
yes I think that’s the right picture
But we don’t derive it by generating a bunch of low-level hypotheses and then abstracting over them, that’d lead to broken ontologies.
I agree that we don’t do that practically as it’d be slower (instead we simply generate an abstraction & use future feedback to determine whether it’s a robust one), but I think if you did generate a bunch of low-level hypotheses and look for redundant computation among them, then an adequate version of it would just recover the “high-level low-level hypotheses” picture you’ve described?
In particular, with cross-hypothesis abstraction we don’t have to separately define what the variables are, so we can sidestep dataset-assembly entirely & perhaps simplify the shifting structures problem
Nice post!
Some frames about abstractions & ontology shifts I had while thinking through similar problems (which you may have considered already):
The dual of “abstraction as redundant information across a wide variety of agents in the same environment” is “abstraction as redundant information/computation across a wide variety of hypotheses about the environment in an agent’s world model” (E.g. a strawberry is a useful concept to model for many worlds that I might be in). I think this is a useful frame when thinking about “carving up” the world model into concepts, since a concept needs to remain invariant while the hypothesis keeps being updated
The semantics of a component in a world model is partly defined by its relationship with the rest of the components (e.g. move a neuron to a different location and its activation will have a different meaning), so if you want a component to have stable semantics over time, you want to put the “relational/indexical information” inside the component itself
In particular, this means that when an agent acquires new concepts, the existing concepts should be able to “specify” how it should relate to that new concept (e.g. learning about chemistry then using it to deduce macro-properties of strawberries from molecular composition)
happy to discuss more via PM as some of my ideas seem exfohazardous
Mm, this one’s shaky. Cross-hypothesis abstractions don’t seem to be a good idea, see here.
My guess is that there’s something like a hierarchy of hypotheses, with specific high-level hypotheses corresponding to several lower-level more-detailed hypotheses, and what you’re pointing at by “redundant information across a wide variety of hypotheses” is just an abstraction in a (single) high-level hypothesis which is then copied over into lower-level hypotheses. (E. g., the high-level hypothesis is the concept of a tree, the lower-level hypotheses are about how many trees are in this forest.) But we don’t derive it by generating a bunch of low-level hypotheses and then abstracting over them, that’d lead to broken ontologies.
Yup!
Yeah, this is probably handled by something like a system of types… which are themselves just higher-level abstractions. Like, if we discover a new thing, and then “realize” that it’s a fruit, we mentally classify it as an instance of the “fruit” concept, from which it then automatically inherits various properties (such as “taste” and “caloric content”).
“Truesight” likely enters the play here as well: we want to recognize instances of existing concepts, even if they were introduced to us by some new route (such as realizing that something is a strawberry by looking at its molecular description).
yea so I think the final theory of abstraction will have a weaker notion of equivalence espeically when we incorporate ontology shifts. E.g. we want to say that water is the same concept before and after we discover water is H2O, but the discovery obviously breaks predictive agreement (Indeed, the solomonoff version of natural latent is more robust to the agreement condition)
Also, you can totally add new information/abstraction that is not shared between your current and new hypothesis, & that seems consistent with the picture you described here (you can have separate ontologies but you try to capture the overlap as much as possible)
yes I think that’s the right picture
I agree that we don’t do that practically as it’d be slower (instead we simply generate an abstraction & use future feedback to determine whether it’s a robust one), but I think if you did generate a bunch of low-level hypotheses and look for redundant computation among them, then an adequate version of it would just recover the “high-level low-level hypotheses” picture you’ve described?
In particular, with cross-hypothesis abstraction we don’t have to separately define what the variables are, so we can sidestep dataset-assembly entirely & perhaps simplify the shifting structures problem