I’m not interested in the prize, but as long as we’re spot-checking, this paragraph bothered me:
It turns out that spreading out the communication flow rate budget over a huge memory store with a slow clock rate is fundamentally more powerful than a fast clock rate over a small memory store. One obvious reason: learning machines have a need to at least store their observational history. A human experiences a sensory input stream at a bitrate of about 10^6 bps (assuming maximal near-lossless compression) for about 10^9 seconds over typical historical lifespan, for a total of about 10^15 bits. The brain has about 2∗10^14 synapses that store roughly 5 bits each, for about 10^15 bits of storage. This is probably not a coincidence.
The idea of bitrate * lifespan = storage capacity makes sense for a VHS or DVD but human memory is completely different. Only a tiny percentage of all our sensory input stream makes it through memory consolidation into long-term memory. Sensory memory is also only one type of memory alongside others like semantic, episodic, autobiographical, and procedural (this list is neither mutually exclusive nor collectively exhaustive). Brains are very good at filtering the sensory stream to focus on salient information and forgetting the vast majority of the rest. This results in memory that is highly compressed (and highly lossy).
Because brain memory is so different than simple storage, this napkin math is roughly analogous to computing body mass from daily intake of air, water, and food mass and neglecting how much actually gets stored versus ultimately discarded. You can do the multiplication but it’s not very meaningful without additional information like retention ratios.
I’m not interested in the prize, but as long as we’re spot-checking, this paragraph bothered me:
The idea of bitrate * lifespan = storage capacity makes sense for a VHS or DVD but human memory is completely different. Only a tiny percentage of all our sensory input stream makes it through memory consolidation into long-term memory. Sensory memory is also only one type of memory alongside others like semantic, episodic, autobiographical, and procedural (this list is neither mutually exclusive nor collectively exhaustive). Brains are very good at filtering the sensory stream to focus on salient information and forgetting the vast majority of the rest. This results in memory that is highly compressed (and highly lossy).
Because brain memory is so different than simple storage, this napkin math is roughly analogous to computing body mass from daily intake of air, water, and food mass and neglecting how much actually gets stored versus ultimately discarded. You can do the multiplication but it’s not very meaningful without additional information like retention ratios.