The Power (and limits?) of Chunking
Working memory is important. How can we increase it?
Writing is like an external brain… if you go back and read it. Dual-N-Back works… maybe. Working memory is important enough for me to incessantly ask about improving it.
What about chunking? The one where you remember ideas better by grouping them into a new bigger idea.
Chunking basically works as an abstraction layer applied to an idea itself. Instead of “lots of (insert sigmoid function)s connected using these thingies”, you go “a few layers of sigmoid-activated neurons”, or even just “this neural network”.
So here are my questions for discussion:
What are the human capacities for chunking ideas?
Can we abstract thoughts
endlesslyendlessly-for-current-practical-purposes, to get around our working memory limits?Does that work for the kinds of super-valuable maths research that could be needed to solve AI alignment? Why or why not? How much of this is “giant galaxy-brained unwriteable uncodifiable concepts that you must hold 100% of in your head all the time and cannot be taught to anyone else” VS “an average-IQ person, given 20 years to just write stuff down and reference it, could get it”.
How often people need to be able to switch between abstraction levels, ways this cognitive work can (or reasons it can’t) be chunked/written/automated/offloaded. (E.g., in the
(This post doesn’t have the answers; I just want to get the ball rolling on people thinking thoroughly about this.)
I think with the right texts, a lot of material normally considered implausibly complicated for a given person to learn, could be learned. Not quickly, but eventually. The problem is that with more complicated or obscure topics, it’s necessary to keep developing concepts that are not reflected in the learning material, so that the only practical way of going about learning them is for the reader to reconstruct them with non-original research (by doing exercises, inventing them first when they are not available).
For example, how long would it take to learn undergraduate physics without doing any exercises, or having/exercising ability to do them, instead reading endless descriptions of worked exercises; compared with doing/inventing exercises yourself? The latter puts the training data in your head in a more natural form, so learning happens faster. The former doesn’t require ability to solve/invent exercises, but would probably take a lot more data to distill the same patterns. So I think in theory this should work, but might be orders of magnitude slower, and requires enormous amounts of training data being prepared by someone else. Probably secondary education teachers at ordinary schools have a better intuition for this.
At least for this the answer is certainly no. There is a fundamental upper limit to the possible information storage density of a unit of space-time