It seems you are claiming that information theory implies there is an objective measure of clusterhood
Not exactly. Rather, I am saying that clusterhood is not the right way to think about these things at all if we want to be free of an arbitrary choice of basis. Rather, we can study the information theoretic properties of the complete partition lattice over an input space and its corresponding probability distribution.
It seems that without clusters we don’t have concepts, and without concepts we don’t have abstractions, and without abstractions we don’t have natural abstractions. So getting rid of clusterhood looks like throwing out the baby with the bathwater.
These dots were sampled from a 2-component Gaussian mixture and then put through a smooth invertible warp. There aren’t any clusters, but the concept is still present and recoverable from the data (altho too hard for our visual cortex to recover in this particular case).
The shortest rule to describe this scatter is that each dot is an independent draw from a mixture of two modes. You have to specify where the two modes are, and you can guess decently well which mode each dot is from. Thru this you’ve rediscovered color without needing clustering.
It looks like x and y are the dimensions of your thingspace here. If you had a different thingspace, would you still be able to recover the same concept?
Yup! This is a very weird space to call a ‘thingspace’ but most transformations of it (anything that’s at least approximately injective, for example) will preserve the same concept.
Here’s what that same distribution I used above looks like if you plot the closed-form pushforward density analytically. In this picture it’s easier for the visual cortex to pick up on the patterns (although it would still be nontrivial for a human to figure out what should be colored red and what should be colored blue if you erased the colors).
Not exactly. Rather, I am saying that clusterhood is not the right way to think about these things at all if we want to be free of an arbitrary choice of basis. Rather, we can study the information theoretic properties of the complete partition lattice over an input space and its corresponding probability distribution.
It seems that without clusters we don’t have concepts, and without concepts we don’t have abstractions, and without abstractions we don’t have natural abstractions. So getting rid of clusterhood looks like throwing out the baby with the bathwater.
Nope! Here’s an example:
These dots were sampled from a 2-component Gaussian mixture and then put through a smooth invertible warp. There aren’t any clusters, but the concept is still present and recoverable from the data (altho too hard for our visual cortex to recover in this particular case).
The shortest rule to describe this scatter is that each dot is an independent draw from a mixture of two modes. You have to specify where the two modes are, and you can guess decently well which mode each dot is from. Thru this you’ve rediscovered color without needing clustering.
It looks like x and y are the dimensions of your thingspace here. If you had a different thingspace, would you still be able to recover the same concept?
Yup! This is a very weird space to call a ‘thingspace’ but most transformations of it (anything that’s at least approximately injective, for example) will preserve the same concept.
Here’s what that same distribution I used above looks like if you plot the closed-form pushforward density analytically. In this picture it’s easier for the visual cortex to pick up on the patterns (although it would still be nontrivial for a human to figure out what should be colored red and what should be colored blue if you erased the colors).