The best frame for this is that better world models and better thinking is not free. It does require paying actual costs, of only in energy. Usually this cost can be very cheap, but things can get expensive in certain problems. Thus, costs are imposed for better reasoning by default.
Also, I think that babbling/brainstorming is pretty useless due to the high amount of dimensions for a lot of problems. Babbling and brainstorming scales as 2^n, with n being the number of dimensions, and for high input values of N, babbling and brainstorming is way too expensive. It’s similar to 2 of John Wentworth’s posts that I’ll link below, but in most real problems, babbling and brainstorming will make progress way too slowly to be of any relevance.
This is also why random exploration is so bad compared to focused exploration.
Links below for why I believe in the idea that babbling/brainstorming is usually not worth it:
The best frame for this is that better world models and better thinking is not free. It does require paying actual costs, of only in energy. Usually this cost can be very cheap, but things can get expensive in certain problems. Thus, costs are imposed for better reasoning by default.
Also, I think that babbling/brainstorming is pretty useless due to the high amount of dimensions for a lot of problems. Babbling and brainstorming scales as 2^n, with n being the number of dimensions, and for high input values of N, babbling and brainstorming is way too expensive. It’s similar to 2 of John Wentworth’s posts that I’ll link below, but in most real problems, babbling and brainstorming will make progress way too slowly to be of any relevance.
This is also why random exploration is so bad compared to focused exploration.
Links below for why I believe in the idea that babbling/brainstorming is usually not worth it:
https://www.lesswrong.com/posts/4XRjPocTprL4L8tmB/science-in-a-high-dimensional-world#comments
https://www.lesswrong.com/posts/pT48swb8LoPowiAzR/everyday-lessons-from-high-dimensional-optimization