I think it’s not a coincidence that the high-order bits are the ones that are preserved by more physical processes. Like, if you take two photos of the same thing, the high order bits are more likely to be the same than the low order bits. Or if you take a photo of a picture on a screen or printed out. Or if you dye two pieces of fabric in the same vat.
I’m not saying you couldn’t get an agent that cared about the low-order bits and not the high-order bits, and if you did have such an agent maybe it would find abstractions that we wouldn’t. But I don’t think I’m being parochial when I say that would be a really weird agent.
I think it’s not a coincidence that the high-order bits are the ones that are preserved by more physical processes. Like, if you take two photos of the same thing, the high order bits are more likely to be the same than the low order bits. Or if you take a photo of a picture on a screen or printed out. Or if you dye two pieces of fabric in the same vat.
I’m not saying you couldn’t get an agent that cared about the low-order bits and not the high-order bits, and if you did have such an agent maybe it would find abstractions that we wouldn’t. But I don’t think I’m being parochial when I say that would be a really weird agent.