I think this is related to the idea that intelligence is compression. But when we think of compression we immediately run into a conundrum: If something is compressible, it means the language used to express the piece of information is not optimal. An optimal description of a thing must be expressed in the most economical way possible. This can only be done if the right frame is used to express the thing in mind. In the right frame you can just “see” the answer, because any translation between your frame and the optimal frame represents a suboptimal routine that can be compressed away. Therefore there is no upper limit on intelligence in the sense that there is no computable way to get the shortest description of a thing. There is always the possibility of compressing an idea further and reaching greater levels of understanding.
In this sense, understanding a physical theory such as classical or quantum mechanics is something akin to being able to just “see” the answer without having to grind through the physical equations used to represent a phenomenon. In some sense this would be akin to developing an intuition behind the equations. But wait one moment. How is it that one can develop an intuition for such physical models? Why is it that it may be hard for some to just “see” the outcome of the equations that are used to express physical phenomena? My hunch is that the physical equations themselves are suboptimal ways for expressing the underlying physical reality in the way that our minds can quickly comprehend. What is really going on inside my mind is that I am building up a world model that tracks the phenomenon of reality and placing “checkpoints” that represent known points corresponding to the outputs of the physical equations. But the physical equations themselves are not a complete description of reality, but only represent these “checkpoints” that correspond to the physical quantities that we can measure at well defined locations. What is really going on I suspect then is just building up of a neural network that is able to predict such phenomena in detail and works surprisingly well for almost all domains of interest.
I reject the first step.
Most posts on this site just seem to posit that there is some “stuff” called information which just exists “somewhere” that is independent of any reference frames. That you can reference a “state” whatever that means.
Consider the set of all possible states in which an observer is reading a page out of a book from the library of babel. Now take one of these states that corresponds to a mind within the library of babel. From its subjective point of view, it has information about its environment corresponding to the page that it is reading, yet the total information contained in the system is actually less than that of the single observer, which can be specified as the set of all possible observers reading a page from the library of babel.
So where is the information coming from? It is the self location of the observer that contains the information. All the information is contained in the reference frame, and this is the primary concern of all condundrums about consciousness. Fundamentally consciousness is about reference frames and the semantics of language.
I could continue making objections at every step, but to keep things brief I will make only an objection which may bear some useful fruit, which occurs at step 5, that is the possibility of digital consciousness. I object to this step in the sense that a digital consciousness means an ability to copy or clone a system “perfectly” with digital accuracy. Again, the problem here is the relationship between reference frames and the ability to copy information. I posit that copying information is forbidden in the sense that you cannot copy reference frames. You run into the usual paradoxes around teleportation and sleeping beauty problems.
In physical reality the only way for an object to be copied is to be destroyed and instantiated somewhere else, and I posit this is how objects actually move through space. Motion is possible because it is impossible to make a digital copy of that object. If you allow for a digital copy of reference frames, suddenly you could perceive all sorts of physical law violations from a subjective point of view, and you may argue this is possible because this happens all the time in dreams. But we enter a slippery slope here, as now we must question our very foundations of physical reality and how to make sense of it.