Good points. But I think that you can get a little logical uncertainty even with just a little bit of the necessary property.
That property being throwing away more information than logically necessary. Like modeling humans using an agent model you know is contradicted by some low-level information.
(From a Shannon perspective calling this throwing away information is weird, since the agent model might produce a sharper probability distribution than the optimal model. But it makes sense to me from a Solomonoff perspective, where you imagine the true sequence as “model + diff,” where diff is something like an imaginary program that fills in for the model and corrects its mistakes. Models that throw away more information will have a longer diff.)
I guess ultimately, what I mean is that there are some benefits of logical uncertainty like for counterfactual reasoning and planning, where using an abstract model automatically gets those benefits. If you never knew the contradictory low-level information in the first place, like your examples, then we just call this “statistical-mechanics-style things.” If you knew the low-level information but threw it away, you could call it logical uncertainty. But it’s still the same model with the same benefits.
The way I think about it, it’s not that we’re using an agent model which is contradicted by some low-level information, it’s that we’re using an agent model which is only valid for some queries. Every abstraction comes with a set of queries over which the high-level model’s predictions match the low-level model’s predictions. Independence of the low-level variables corresponding to high-level components is the main way we track which queries are valid: valid queries are those which don’t ask about variables “close together”.
So at least the way I’m thinking about it, there’s never any contradictory information to throw away in the first place.
Good points. But I think that you can get a little logical uncertainty even with just a little bit of the necessary property.
That property being throwing away more information than logically necessary. Like modeling humans using an agent model you know is contradicted by some low-level information.
(From a Shannon perspective calling this throwing away information is weird, since the agent model might produce a sharper probability distribution than the optimal model. But it makes sense to me from a Solomonoff perspective, where you imagine the true sequence as “model + diff,” where diff is something like an imaginary program that fills in for the model and corrects its mistakes. Models that throw away more information will have a longer diff.)
I guess ultimately, what I mean is that there are some benefits of logical uncertainty like for counterfactual reasoning and planning, where using an abstract model automatically gets those benefits. If you never knew the contradictory low-level information in the first place, like your examples, then we just call this “statistical-mechanics-style things.” If you knew the low-level information but threw it away, you could call it logical uncertainty. But it’s still the same model with the same benefits.
The way I think about it, it’s not that we’re using an agent model which is contradicted by some low-level information, it’s that we’re using an agent model which is only valid for some queries. Every abstraction comes with a set of queries over which the high-level model’s predictions match the low-level model’s predictions. Independence of the low-level variables corresponding to high-level components is the main way we track which queries are valid: valid queries are those which don’t ask about variables “close together”.
So at least the way I’m thinking about it, there’s never any contradictory information to throw away in the first place.