Similarly, it is not clear to me at all what is even meant by saying that a tiny neural network can perfectly predict the “world” of Go
By world model I specifically meant a model of the world physics. For chess/go this is just a tiny amount of memory to store the board state, and a simple set of rules that are very fast to evaluate. I agree that evaluating the rules of go is a bit more complex than chess, especially in edge cases, but still enormously simpler than evaluating the physics of the real world.
I think we probably agree about grokking in NNs but I am doubting that EY would describe that as foom.
By world model I specifically meant a model of the world physics. For chess/go this is just a tiny amount of memory to store the board state, and a simple set of rules that are very fast to evaluate. I agree that evaluating the rules of go is a bit more complex than chess, especially in edge cases, but still enormously simpler than evaluating the physics of the real world.
I think we probably agree about grokking in NNs but I am doubting that EY would describe that as foom.