I don’t buy 4.1. I believe the laws of the universe should be simple, but simplicity of specifying the initial state is not an argument for me (unless there are additional factors). Don’t buy 4.2 either, because I don’t trust anthropic reasoning that much. However, I’m still interested in ideas like this, even if I don’t believe in them.
4.3. I don’t know enough math, but I don’t believe that a) we have the math to prove that an agent can’t survive in an environment with incompressible low-level dynamics or b) it means anything non-trivial enough to prove the kind of points you need to prove. It would be a magically good result.
In a chaotic environment there could be some high-level patterns which the agent might exploit, but those high-level patterns could be barely similar to humanlike abstractions. The Gooder Regulator theorem could be false, maybe, if we don’t need to be prepared for “any possible game”.
I don’t buy 4.1. I believe the laws of the universe should be simple, but simplicity of specifying the initial state is not an argument for me (unless there are additional factors). Don’t buy 4.2 either, because I don’t trust anthropic reasoning that much. However, I’m still interested in ideas like this, even if I don’t believe in them.
4.3. I don’t know enough math, but I don’t believe that a) we have the math to prove that an agent can’t survive in an environment with incompressible low-level dynamics or b) it means anything non-trivial enough to prove the kind of points you need to prove. It would be a magically good result.
In a chaotic environment there could be some high-level patterns which the agent might exploit, but those high-level patterns could be barely similar to humanlike abstractions. The Gooder Regulator theorem could be false, maybe, if we don’t need to be prepared for “any possible game”.