I think your main counterpoint to what I said is that people are doing an optimization process where they look at the data while simultaneously doing a search for a better theory. In fact, you cannot even disentangle their brain from the reality that created and runs it, so even a best attempt at theory first, observation second is doomed to fail.
I think the second, stronger sentence is mostly wrong. You do not need a universe similar enough to our universe to produce reasoning similar to ours, just one that can produce similar reasoning and has an incentive to. That incentive can be as little as, “I wonder what physics looks like in 3+1 dimensions?” just like our physicists wonder what it looks like in more or less dimensions, with different fundamental constants, with different laws of motion, with positive spacetime curvature, and so on. Or, we can just shove a bunch of data from our universe into theirs, and reward them for figuring it out (i.e. training LLMs).
As for the first, weaker sentence, yes this is true. Pretty much everyone has tight feedback loops, probably because the search space is too large to first categorize its entirety and then match the single branch you end up observing. I think the role of observation here is closer to moving attention to certain areas of the search space, rather than moving the search tree forward (see Richard Ngo’s shortform on chess). The thing is, this process is unnecessary for simple things. You probably learned to solve TicTacToe by playing a bunch of games, but you could have just solved it. I think the concept of trees are relatively simple, though of course if you want a refined concept like its protein composition or DNA sequencing, yeah that space is too big and you probably have to just go out and observe it.
I don’t really understand your point about unsupervised learning. With unsupervised learning, you can just run a bunch of data through your model until it learns something. That’s the observation → theory pipeline and it’s astoundingly inefficient and bad at generalization. Humans could do the same with 100x fewer examples, which is the gap models need to clear to solve ARC-AGI. Humans are probably doing something closer to theory → observation.
I think your main counterpoint to what I said is that people are doing an optimization process where they look at the data while simultaneously doing a search for a better theory. In fact, you cannot even disentangle their brain from the reality that created and runs it, so even a best attempt at theory first, observation second is doomed to fail.
I think the second, stronger sentence is mostly wrong. You do not need a universe similar enough to our universe to produce reasoning similar to ours, just one that can produce similar reasoning and has an incentive to. That incentive can be as little as, “I wonder what physics looks like in 3+1 dimensions?” just like our physicists wonder what it looks like in more or less dimensions, with different fundamental constants, with different laws of motion, with positive spacetime curvature, and so on. Or, we can just shove a bunch of data from our universe into theirs, and reward them for figuring it out (i.e. training LLMs).
As for the first, weaker sentence, yes this is true. Pretty much everyone has tight feedback loops, probably because the search space is too large to first categorize its entirety and then match the single branch you end up observing. I think the role of observation here is closer to moving attention to certain areas of the search space, rather than moving the search tree forward (see Richard Ngo’s shortform on chess). The thing is, this process is unnecessary for simple things. You probably learned to solve TicTacToe by playing a bunch of games, but you could have just solved it. I think the concept of trees are relatively simple, though of course if you want a refined concept like its protein composition or DNA sequencing, yeah that space is too big and you probably have to just go out and observe it.
I don’t really understand your point about unsupervised learning. With unsupervised learning, you can just run a bunch of data through your model until it learns something. That’s the observation → theory pipeline and it’s astoundingly inefficient and bad at generalization. Humans could do the same with 100x fewer examples, which is the gap models need to clear to solve ARC-AGI. Humans are probably doing something closer to theory → observation.