For the street map example, ML is the physical city streets—which means it’s the molecules/atoms/fields which comprise the streets. When we represent the streets as lines on paper, those lines are summary statistics of molecule positions, just like the ideal gas example. The only difference is that in the ideal gas example, it’s a lot easier to express the relevant distribution which the statistics summarize.
That said, I do think you’re pointing to something interesting. There is a sense in which a high-level model adds something in.
Look at the factorizations in the “Systems View” section. They are factorizations of a joint distribution over both the high-level and low-level variables. We have a single model which includes both sets of variables. The high-level variables are quite literally added into the low-level model as new variables computed from the old. The high-level model then keeps those new variables, and throws away all the original low-level variables.
Ah, I think I see what you mean. That makes sense that the high level model of the street map is also a summary statistic, not just the low level model with stuff thrown away. Let my try to refine my comment.
For the ideal gas example, I think of the low level model as looking something like this:
class LowLevelGas {
Particle[] particles;
}
class Particle {
String compound;
int speed;
int direction;
int mass;
// whatever else
}
And I think of the high level model as looking like this:
class HighLevelGas {
int pressure;
int volume;
int temperature;
}
LowLevelGas and HighLevelGas just look like there’s a big difference between the two. On the other hand, LowLevelStreetMap and HighLevelStreetMap wouldn’t look as different. It’d be analogous to a sketch vs a photograph, where the difference is sort of a matter of resolution. But with LowLevelGas and HighLevelGas, it seems like they are different in a more fundamental way. They have different properties, not the same properties at different resolutions.
I wonder if this “resolution” idea can be made more formal. Something along the lines of looking at the high level variables and low level variables and seeing how… similar?… they are.
Elizer’s idea of Thingspace comes to mind. In theory maybe you could look at how close they are in Thingspace, but in practice that seems really difficult.
I’ll explain what I think you’re pointing to here, then let me know if it sounds like what you’re imagining.
In the street map example, everything is spatially delineated, at both the high and low level. The abstraction can be interpreted as blurring the spatial resolution. The spatial resolution is a conserved structure between the two. But in the gas example, everything is blurred “all the way”, so there’s just a single high-level object which doesn’t retain any structure which is obviously-similar to the low-level.
(An intermediate between these two would be Navier Stokes: it applies basically the same abstraction as the ideal gas, but only within small spatially-delineated cells.)
So there’s potentially a difference between abstractions which throw away basically all the structure vs abstractions which retain some.
This points toward a more general class of questions: when, and to what extent, does it all add up to normality? We learned the high-level ideal gas laws long before we learned the low-level molecular theory, but we knew the low-level had to at least be consistent with that high-level structure. What low-level structures did that constraint exclude? More generally: to what extent does our knowledge of the high-level model structure constrain the possible low-level structures?
One good class of structure for these sorts of questions is causal structure: to what extent does high-level causal structure constrain the possible low-level causal structures? I’ll probably have a post on that soon-ish.
This points toward a more general class of questions: when, and to what extent, does it all add up to normality? We learned the high-level ideal gas laws long before we learned the low-level molecular theory, but we knew the low-level had to at least be consistent with that high-level structure. What low-level structures did that constraint exclude? More generally: to what extent does our knowledge of the high-level model structure constrain the possible low-level structures?
One good class of structure for these sorts of questions is causal structure: to what extent does high-level causal structure constrain the possible low-level causal structures? I’ll probably have a post on that soon-ish.
Doesn’t high-level structure entail statistical averages and not necessarily Boltzmann brains in the low-level structure? Like—what of the nonequilibrium statistical mechanics?
Problem is, we didn’t know beforehand (i.e. in 1800) that the high-level things we saw (like temperatures, heat flow, etc) had anything to do with statistical averages. One could imagine an alternative universe running on different physics, where heat really is a fluid and yet macroscopically it behaves a lot like heat in our universe. If we imagine all the difference ways things could have turned out to work, given only what we knew in 1800, where does that leave us? What low-level structure is implied by the high-level structure?
For the street map example, ML is the physical city streets—which means it’s the molecules/atoms/fields which comprise the streets. When we represent the streets as lines on paper, those lines are summary statistics of molecule positions, just like the ideal gas example. The only difference is that in the ideal gas example, it’s a lot easier to express the relevant distribution which the statistics summarize.
That said, I do think you’re pointing to something interesting. There is a sense in which a high-level model adds something in.
Look at the factorizations in the “Systems View” section. They are factorizations of a joint distribution over both the high-level and low-level variables. We have a single model which includes both sets of variables. The high-level variables are quite literally added into the low-level model as new variables computed from the old. The high-level model then keeps those new variables, and throws away all the original low-level variables.
Ah, I think I see what you mean. That makes sense that the high level model of the street map is also a summary statistic, not just the low level model with stuff thrown away. Let my try to refine my comment.
For the ideal gas example, I think of the low level model as looking something like this:
And I think of the high level model as looking like this:
LowLevelGas
andHighLevelGas
just look like there’s a big difference between the two. On the other hand,LowLevelStreetMap
andHighLevelStreetMap
wouldn’t look as different. It’d be analogous to a sketch vs a photograph, where the difference is sort of a matter of resolution. But withLowLevelGas
andHighLevelGas
, it seems like they are different in a more fundamental way. They have different properties, not the same properties at different resolutions.I wonder if this “resolution” idea can be made more formal. Something along the lines of looking at the high level variables and low level variables and seeing how… similar?… they are.
Elizer’s idea of Thingspace comes to mind. In theory maybe you could look at how close they are in Thingspace, but in practice that seems really difficult.
I’ll explain what I think you’re pointing to here, then let me know if it sounds like what you’re imagining.
In the street map example, everything is spatially delineated, at both the high and low level. The abstraction can be interpreted as blurring the spatial resolution. The spatial resolution is a conserved structure between the two. But in the gas example, everything is blurred “all the way”, so there’s just a single high-level object which doesn’t retain any structure which is obviously-similar to the low-level.
(An intermediate between these two would be Navier Stokes: it applies basically the same abstraction as the ideal gas, but only within small spatially-delineated cells.)
So there’s potentially a difference between abstractions which throw away basically all the structure vs abstractions which retain some.
This points toward a more general class of questions: when, and to what extent, does it all add up to normality? We learned the high-level ideal gas laws long before we learned the low-level molecular theory, but we knew the low-level had to at least be consistent with that high-level structure. What low-level structures did that constraint exclude? More generally: to what extent does our knowledge of the high-level model structure constrain the possible low-level structures?
One good class of structure for these sorts of questions is causal structure: to what extent does high-level causal structure constrain the possible low-level causal structures? I’ll probably have a post on that soon-ish.
Yeah, that’s what I’m getting at.
Doesn’t high-level structure entail statistical averages and not necessarily Boltzmann brains in the low-level structure? Like—what of the nonequilibrium statistical mechanics?
Problem is, we didn’t know beforehand (i.e. in 1800) that the high-level things we saw (like temperatures, heat flow, etc) had anything to do with statistical averages. One could imagine an alternative universe running on different physics, where heat really is a fluid and yet macroscopically it behaves a lot like heat in our universe. If we imagine all the difference ways things could have turned out to work, given only what we knew in 1800, where does that leave us? What low-level structure is implied by the high-level structure?