Gunnar- yes I think this is true, but it’s really surprisingly difficult to operationalize this. Here is how I think this plays out:
Suppose that we are recording videos of some meerkats running around in a certain area. One might think that the raw video data is not very predictive of the future, but that if we used the video data to infer the position and velocity of each meerkat, then we could predict the future position of the meerkats, which would indicate an increase in knowledge compared to just storing the raw data. And I do think that this is what knowledge means, but if we try to operationalize this “predictive” quality in terms of a correspondence between the present configuration of our computer and the future configuration of the meerkats then the raw data will actually have higher mutual information with future configurations than the position-and-velocity representation will.
It is difficult to formalize. But I think your example is off. I didn’t mean that we could infer from the video the positions of meerkats now. I meant that the representation encodes or correlates with the future positions (at least more than with the current positions). It is as if there was a video that showed the future meerkats’ movements. That would be surprising to find in nature.
Gunnar- yes I think this is true, but it’s really surprisingly difficult to operationalize this. Here is how I think this plays out:
Suppose that we are recording videos of some meerkats running around in a certain area. One might think that the raw video data is not very predictive of the future, but that if we used the video data to infer the position and velocity of each meerkat, then we could predict the future position of the meerkats, which would indicate an increase in knowledge compared to just storing the raw data. And I do think that this is what knowledge means, but if we try to operationalize this “predictive” quality in terms of a correspondence between the present configuration of our computer and the future configuration of the meerkats then the raw data will actually have higher mutual information with future configurations than the position-and-velocity representation will.
It is difficult to formalize. But I think your example is off. I didn’t mean that we could infer from the video the positions of meerkats now. I meant that the representation encodes or correlates with the future positions (at least more than with the current positions). It is as if there was a video that showed the future meerkats’ movements. That would be surprising to find in nature.