Daniel’s CRM is about lossless compression, thus my response. I can’t tell what you agree or disagree with.
Minimum description length and information theory are both part of the standard AI curriculum. I’d agree that most AI work suggests (and uses in its implementation, e.g. Bloom filters) lossy compression.
Daniel’s CRM is about lossless compression, thus my response.
I don’t think the distinction between lossy and lossless is as important in AI as you imply here:
I doubt you can even figure out how to use a human-level explanation of the contents of an image to improve lossless image compression rate, so I’d suggest a different route.
As you will note from my comment here, any lossy method can be trivially converted to a corresponding lossless one by adding on a list of “exceptions” to the “theory”. So long as the exceptions are few, the compressed string can still be shorter than the source.
So if you notice a major regularity in the data, using it can still result in a shortening of the message length, even if not all of the data adheres to it. Thus, the constraint that the method be lossless is not significant in this context.
I don’t disagree with your breaking a lossless compression into those three parts, but converting a lossy compression scheme to a lossless one seem a poor way to judge it: you condemn jpeg as doing no compression. Yes, you probably could use the same ideas to improve lossless compression, but the 90% compression rate of jpeg means something.
Lossless compression has the advantage of objective judgement, but for any particular purposes we want a lossy compression that throws out the data irrelevant to that purpose. Of course, that depends on the purpose; if you’re debugging a camera, you don’t want jpeg.
In other words, I endorse lossy compression over lossless compression as a measure of theory. (I think Jonathan Graehl and Phil Goetz agree, I’m not sure.)
But jpeg is being judged by a slightly different figure of merit, which changes the relevant considerations.
Like other lossy image compression schemes, jpeg is judged based on how much of the source picture it is that a human notices as being lost. So, for that problem, the relevant “source” to be encoded is really the human perception of the image, not the bitstream of the image file.
And really, since all phenomena pass through someone’s senses before they’re observed, this is the relevant metric for any MML formalism: whether you can come up with a way to briefly describe what you have experienced. It’s just that in typical lossless tests, they’ve eliminated the impact of human cognition by making it possible to compare directly to a specific string and determine if that string can be completely reproduced.
So I don’t condemn jpeg or regard it as meaningless; I’m saying it can be directly compared to lossless methods if you either a) penalize it by the amount of data needed to go from the jpeg to the source image, or b) penalize it by the amount of data needed, on average, to restore the image to something a human would not remember differently (which in many cases is very little if any data).
Daniel’s CRM is about lossless compression, thus my response. I can’t tell what you agree or disagree with.
Minimum description length and information theory are both part of the standard AI curriculum. I’d agree that most AI work suggests (and uses in its implementation, e.g. Bloom filters) lossy compression.
I don’t think the distinction between lossy and lossless is as important in AI as you imply here:
As you will note from my comment here, any lossy method can be trivially converted to a corresponding lossless one by adding on a list of “exceptions” to the “theory”. So long as the exceptions are few, the compressed string can still be shorter than the source.
So if you notice a major regularity in the data, using it can still result in a shortening of the message length, even if not all of the data adheres to it. Thus, the constraint that the method be lossless is not significant in this context.
I don’t disagree with your breaking a lossless compression into those three parts, but converting a lossy compression scheme to a lossless one seem a poor way to judge it: you condemn jpeg as doing no compression. Yes, you probably could use the same ideas to improve lossless compression, but the 90% compression rate of jpeg means something.
Lossless compression has the advantage of objective judgement, but for any particular purposes we want a lossy compression that throws out the data irrelevant to that purpose. Of course, that depends on the purpose; if you’re debugging a camera, you don’t want jpeg.
In other words, I endorse lossy compression over lossless compression as a measure of theory. (I think Jonathan Graehl and Phil Goetz agree, I’m not sure.)
But jpeg is being judged by a slightly different figure of merit, which changes the relevant considerations.
Like other lossy image compression schemes, jpeg is judged based on how much of the source picture it is that a human notices as being lost. So, for that problem, the relevant “source” to be encoded is really the human perception of the image, not the bitstream of the image file.
And really, since all phenomena pass through someone’s senses before they’re observed, this is the relevant metric for any MML formalism: whether you can come up with a way to briefly describe what you have experienced. It’s just that in typical lossless tests, they’ve eliminated the impact of human cognition by making it possible to compare directly to a specific string and determine if that string can be completely reproduced.
So I don’t condemn jpeg or regard it as meaningless; I’m saying it can be directly compared to lossless methods if you either a) penalize it by the amount of data needed to go from the jpeg to the source image, or b) penalize it by the amount of data needed, on average, to restore the image to something a human would not remember differently (which in many cases is very little if any data).