Davidad responds with a brief argument for 1000 FLOP-equivalent per synapse-second (3 OOM more than my guess) on X as follows:
Ok, so assuming we agree on 1e14 synapses and 3e8 seconds, then where we disagree is on average FLOP(-equivalent) per synapse-second: you think it’s about 1, I think it’s about 1000. This is similar to the disagreement you flagged with Joe Carlsmith.
Note: at some point Joe interviewed me about this so there might be some double-counting of “independent” estimates here, but iirc he also interviewed many other neuroscientists.
My estimate would be a lot lower if we were just talking about “inference” rather than learning and memory. STDP seems to have complex temporal dynamics at the 10ms scale.
There also seem to be complex intracellular dynamics at play, possibly including regulatory networks, obviously regarding synaptic weight but also other tunable properties of individual compartments.
The standard arguments for the causal irrelevance of these to cognition (they’re too slow to affect the “forward pass”) don’t apply to learning. I’m estimating there’s like a 10-dimensional dynamical system in each compartment evolving at ~100Hz in importantly nonlinear ways.
Davidad responds with a brief argument for 1000 FLOP-equivalent per synapse-second (3 OOM more than my guess) on X as follows: