I think it’s an issue of “inside the box” vs “open-ended” fields, that we don’t have really good vocabulary to talk about. ‘Katas’ work great for sports that are very much inside the box. You can innovate new strategies, but the rules of the sport set up an unchanging microworld that you must stay inside of. Coincidentally, these are also areas where even current-day AIs often dominate. Established scientific disciplines with research programs are sort of half and half. You can train people in them, but they can also benefit from serious paradigm shifts and there aren’t any a priori hard and fast rules about things that absolutely can’t be done, like the rules of chess for the chess-playing domain.
Then there’s proto-science when things haven’t coalesced into a discipline yet, philosophy when it hasn’t been professionalized to death, Kegan’s stage 5. This is raw pattern matching, flashes of insight, original seeing, very open ended exploration of an unknown landscape. I don’t think anyone has had much of an idea for how to systematically train people for this. This place is also where a lot of the actually efficient rationality practice lives.
7 votes
Overall karma indicates overall quality.
0 votes
Agreement karma indicates agreement, separate from overall quality.
I feel like GEB has been diminished a bit by its own success. People reading it nowadays might go “what’s the big deal?” A big theme is how the mind can be a machine and still do stupid stuff, which had to be spelled out in the 70s but has pretty much permeated the relevant subcultures these days. And of course Hofstadter didn’t know a clear recipe for an actual AGI, so the speculative parts on that were left at the level of intriguing handwaving.