Imprecisely multiplying two analog numbers should not require 10^5 times the minimum bit energy in a well-designed computer.
A well-designed computer would also use, say, optical interconnects that worked by pushing one or two photons around at the speed of light. So if neurons are in some sense being relatively efficient at the given task of pumping thousands upon thousands of ions in and out of a depolarizing membrane in order to transmit signals at 100m/sec—every ion of which necessarily uses at least the Landauer minimum energy—they are being vastly far from optimally efficient.
The moment you see ions going in and out of a depolarizing membrane, and contrast that to the possibility of firing a photon down a fiber, you ought to be done asking whether or not biology has built an optimally efficient computer. It actually isn’t any more complicated than that. You are driving yourself further from sanity if you then try to do very complicated reasoning about how it must be close to the limit of efficiency to pump thousands of ions in and out of a membrane instead.
Imprecisely multiplying two analog numbers should not require 10^5 times the minimum bit energy in a well-designed computer.
Much depends on your exact definition of ‘imprecisely’. But if we assume exactly 8-bit equivalent SNR, as I was using above, then you can lookup this question in the research literature and/or ask an LLM and the standard answer is in fact close to ~1e5 eV.
This multiplication op is a masking operation and not inherently reversible so it erases/destroys about 1⁄2 of the energy of the photonic input signal (100% if you multiply by 0, etc). So the min energy boils down to that required to represent an 8-bit number reliably as an analog signal (so for example you could convert a digital 8-bit signal to analog and back to digital losslessly all at the same standard sufficient 1eV reliability).
Analog signals effectively represent numbers as the 1st moment of a binomial distribution over carrier particles, and the information content is basically the entropy of a binomial over 1eV carriers which is ~0.5 log2(N) and thus N ~ 2^(2b) quanta for b bits of precision.
The energy to represent an analog signal doesn’t depend much on the medium—whether you are using photons or electrons/ions. The advantage of the electronic medium is the much smaller practical device dimensions possible when using much heavier/denser particles as bit carriers: 1eV photons are micrometer scale, much larger than the smallest synapses/transistors/biodevices. The obvious advantage of photons is their much higher transmission speed: thus they are used for longer range interconnect (but mostly only for distances larger than the brain radius).
Sorry, explain again why floods of neurotransmitter molecules bopping around are ideally thermodynamically efficient? You’re assuming that they’re trying to do multiplication out to 8-bit precision using analog quantities? Why suppose the 8-bit precision? Even if that part was actually important, why not perhaps ding biology a few engineering points for trying to represent it using analog quantities requiring 2^16 particles bopping around? Optimally doing something incredibly inefficient is incredibly inefficient.
Sorry, explain again why floods of neurotransmitter molecules bopping around are ideally thermodynamically efficient? You’re assuming that they’re trying to do multiplication out to 8-bit precision using analog quantities? Why suppose the 8-bit precision?
I’m not assuming that, but its nonetheless useful as a benchmark for comparison. It helps illustrate that 1e5 eV is really not much—it just allows a single 8-bit analog mult for example.
Earlier in the thread I said:
Now most synapses are probably smaller/cheaper than 8-bit equiv, but most of the energy cost involved is in pushing data down irreversible dissipative wires (just as true in the brain as it is in a GPU). Now add in the additional costs of synaptic adjustment machinery for learning, cell maintenance tax, dendritic computation, etc
The synapse is clearly doing something somewhat more complex than just analog multiplication.
And in terms of communication costs (which are paid at the synaptic junction for the synapse → dendrite → soma path), that 1e5 eV is only enough to carry a reliable 1 bit signal only about ~100mm (1e5 nm) distance through irreversible nano/micro scale wires (the wire bit energy for axons/dendrites and modern cmos is about the same).
Reversible interconnect is much more complex—requires communicating through fully isolated particles over the wire distance, which is obviously much more practical for photons for various reasons, but they are very large etc. Many complex tradeoffs.
And in terms of communication costs (which are paid at the synaptic junction for the synapse → dendrite → soma path), that 1e5 eV is only enough to carry a reliable 1 bit signal only about ~100mm (1e5 nm) distance through irreversible wires (the wire bit energy for axons/dendrites and modern cmos is about the same).
If it applies in the specific cases of axons and cmos there should be justification of why it does, though given the amount of prior discussion I don’t think this would be fruitful.
No—Coax cables are enormous in radius (EM wavelengths), and do not achieve much better than 1 eV / nm in practice. In the same waveguide radius you can you just remove the copper filler and go pure optical and then get significantly below 1 eV/nm anyway—so why even mention coax?
The only thing that was ‘debunked’ was in a tangent conversation that had no bearing on the main point (about nanoscale wire interconnect smaller than EM wavelength—which is irreversible and consumes close to 1 eV/nm in both brains and computers), and it was just my initial conception that coax cables could be modeled in simplification as relays like RC interconnect.
There are many complex tradeoffs between size, speed, energy, etc. Reversible and irreversible comms occupy different regions of that pareto surface. Reversible communication is isomorphic to transmitting particles—in practice always photons—and requires complex/large transmitter/receivers and photon sized waveguides etc. Irreversible communication is isomorphic to domino-based computing, and has the advantage—and cost—of full error correction/erasure at every cycle, and easier to guide down narrow and complex paths.
Imprecisely multiplying two analog numbers should not require 10^5 times the minimum bit energy in a well-designed computer.
A well-designed computer would also use, say, optical interconnects that worked by pushing one or two photons around at the speed of light. So if neurons are in some sense being relatively efficient at the given task of pumping thousands upon thousands of ions in and out of a depolarizing membrane in order to transmit signals at 100m/sec—every ion of which necessarily uses at least the Landauer minimum energy—they are being vastly far from optimally efficient.
The moment you see ions going in and out of a depolarizing membrane, and contrast that to the possibility of firing a photon down a fiber, you ought to be done asking whether or not biology has built an optimally efficient computer. It actually isn’t any more complicated than that. You are driving yourself further from sanity if you then try to do very complicated reasoning about how it must be close to the limit of efficiency to pump thousands of ions in and out of a membrane instead.
Much depends on your exact definition of ‘imprecisely’. But if we assume exactly 8-bit equivalent SNR, as I was using above, then you can lookup this question in the research literature and/or ask an LLM and the standard answer is in fact close to ~1e5 eV.
This multiplication op is a masking operation and not inherently reversible so it erases/destroys about 1⁄2 of the energy of the photonic input signal (100% if you multiply by 0, etc). So the min energy boils down to that required to represent an 8-bit number reliably as an analog signal (so for example you could convert a digital 8-bit signal to analog and back to digital losslessly all at the same standard sufficient 1eV reliability).
Analog signals effectively represent numbers as the 1st moment of a binomial distribution over carrier particles, and the information content is basically the entropy of a binomial over 1eV carriers which is ~0.5 log2(N) and thus N ~ 2^(2b) quanta for b bits of precision.
The energy to represent an analog signal doesn’t depend much on the medium—whether you are using photons or electrons/ions. The advantage of the electronic medium is the much smaller practical device dimensions possible when using much heavier/denser particles as bit carriers: 1eV photons are micrometer scale, much larger than the smallest synapses/transistors/biodevices. The obvious advantage of photons is their much higher transmission speed: thus they are used for longer range interconnect (but mostly only for distances larger than the brain radius).
Sorry, explain again why floods of neurotransmitter molecules bopping around are ideally thermodynamically efficient? You’re assuming that they’re trying to do multiplication out to 8-bit precision using analog quantities? Why suppose the 8-bit precision? Even if that part was actually important, why not perhaps ding biology a few engineering points for trying to represent it using analog quantities requiring 2^16 particles bopping around? Optimally doing something incredibly inefficient is incredibly inefficient.
I’m not assuming that, but its nonetheless useful as a benchmark for comparison. It helps illustrate that 1e5 eV is really not much—it just allows a single 8-bit analog mult for example.
Earlier in the thread I said:
The synapse is clearly doing something somewhat more complex than just analog multiplication.
And in terms of communication costs (which are paid at the synaptic junction for the synapse → dendrite → soma path), that 1e5 eV is only enough to carry a reliable 1 bit signal only about ~100mm (1e5 nm) distance through irreversible nano/micro scale wires (the wire bit energy for axons/dendrites and modern cmos is about the same).
Reversible interconnect is much more complex—requires communicating through fully isolated particles over the wire distance, which is obviously much more practical for photons for various reasons, but they are very large etc. Many complex tradeoffs.
This model of interconnect energy has been thoroughly debunked here, as coax cables violate it by a factor of 200: https://www.lesswrong.com/posts/fm88c8SvXvemk3BhW/brain-efficiency-cannell-prize-contest-award-ceremony
If it applies in the specific cases of axons and cmos there should be justification of why it does, though given the amount of prior discussion I don’t think this would be fruitful.
No—Coax cables are enormous in radius (EM wavelengths), and do not achieve much better than 1 eV / nm in practice. In the same waveguide radius you can you just remove the copper filler and go pure optical and then get significantly below 1 eV/nm anyway—so why even mention coax?
The only thing that was ‘debunked’ was in a tangent conversation that had no bearing on the main point (about nanoscale wire interconnect smaller than EM wavelength—which is irreversible and consumes close to 1 eV/nm in both brains and computers), and it was just my initial conception that coax cables could be modeled in simplification as relays like RC interconnect.
There are many complex tradeoffs between size, speed, energy, etc. Reversible and irreversible comms occupy different regions of that pareto surface. Reversible communication is isomorphic to transmitting particles—in practice always photons—and requires complex/large transmitter/receivers and photon sized waveguides etc. Irreversible communication is isomorphic to domino-based computing, and has the advantage—and cost—of full error correction/erasure at every cycle, and easier to guide down narrow and complex paths.