First, an agent that is included in the environment space (we call an agent like this an embedded agent), cannot have a hypothesis that includes itself. Since the agent is part of the environment, modeling each bit of the environment requires the agent to model itself in every detail, which would require the agent’s self-model to be at least as big as the whole agent. An agent can’t fit inside its own head, as this leads to infinite regress.
Stated informally like this, the argument is more naturally false than true. Because quines and compression, and more generally not writing out any specific state in full (if that even makes sense) when reasoning about states in a space that contains them. This is fractionally worse when we get to a hypothesis that only admits a single state and so must be specified with the same information as that state, but again quines and other methods of compression. Compression that only has to work well enough in practice for the actual environment and hypotheses leading up to it, not for every possible environment in the space. Nothing here “leads to infinite regress” except in much more specific circumstances that are not obviously relevant.
The real world is also non-realizable because an agent living in the world cannot fit the entire state of the world inside of its own head.
Could in principle be false about the real world, if it turns out to be a simulation with a finite definition, and an agent is large enough to know it (plus enough information to know its own place in there, to better reflect the spirit of the claim).
A formal version of which argument? That quined states of knowledge are possible? Or that there are circumstances where embedded perfect knowledge can’t work after all? The latter is similar to impossibility of compressing every possible file, there are more states in longer files, so there is no injection to the set of states of shorter files. So if the embedded agent is smaller than half of environment, it can’t encode the environment that exists outside of it, even if it’s free to set its own state.
But if you don’t have to compress every possible file, only likely files, then compression works fine and can be used to construct quines. It’s sufficient to be able to compress “files with holes”. Then all you need is put these holes over locations where representation of the world needs to go, compress the rest of the world, and finally put the compressed data where the holes used to be.
Stated informally like this, the argument is more naturally false than true. Because quines and compression, and more generally not writing out any specific state in full (if that even makes sense) when reasoning about states in a space that contains them. This is fractionally worse when we get to a hypothesis that only admits a single state and so must be specified with the same information as that state, but again quines and other methods of compression. Compression that only has to work well enough in practice for the actual environment and hypotheses leading up to it, not for every possible environment in the space. Nothing here “leads to infinite regress” except in much more specific circumstances that are not obviously relevant.
Could in principle be false about the real world, if it turns out to be a simulation with a finite definition, and an agent is large enough to know it (plus enough information to know its own place in there, to better reflect the spirit of the claim).
Good point. Anyone knows if there is a formal version of this argument written down somewhere?
A formal version of which argument? That quined states of knowledge are possible? Or that there are circumstances where embedded perfect knowledge can’t work after all? The latter is similar to impossibility of compressing every possible file, there are more states in longer files, so there is no injection to the set of states of shorter files. So if the embedded agent is smaller than half of environment, it can’t encode the environment that exists outside of it, even if it’s free to set its own state.
But if you don’t have to compress every possible file, only likely files, then compression works fine and can be used to construct quines. It’s sufficient to be able to compress “files with holes”. Then all you need is put these holes over locations where representation of the world needs to go, compress the rest of the world, and finally put the compressed data where the holes used to be.