IABI says: “Transistors, a basic building block of all computers, can switch on and off billions of times per second; unusually fast neurons, by contrast, spike only a hundred times per second. Even if it took 1,000 transistor operations to do the work of a single neural spike, and even if artificial intelligence was limited to modern hardware, that implies human-quality thinking could be emulated 10,000 times faster on a machine— to say nothing of what an AI could do with improved algorithms and improved hardware.
I am confused what the issue is, and it would be awesome if someone can explain it to me.
Where I’m coming from, for context:
We don’t know exactly what the relevant logical operations in the human brain are. The model of the brain that says there are binary spiking neurons that have direct connections from synapse->dendrite and that those connections are akin to floating-point numerical weights is clearly a simplification, albeit a powerful one. (IIUC “neural nets” in computers discard the binary-spikes and suggest another model where the spike-rate is akin to a numerical value, which is the basic story behind “neuron activation” in a modern system. This simplification also seems powerful, though it is surely an oversimplification in some ways.)
My main issue with the source text is that it ignores what is possibly the greater bottleneck in processing speed, which is the time it takes to move information from one area to another. (If my model is right, one of the big advantages of a MoE architecture is to reduce the degree of thrashing weights across the bus to and from the GPU as much, which can be a major bottleneck.) However, on this front I think nerves are still clearly inferior to wires? Even mylenated neurons have a typical speed of only about 100 m/s, while information flows across wires at >50% the speed of light.
My read of the critics is that they aren’t objecting to the notion that clock speeds are significantly faster than neurons, but rather that comparing the two is a bad way of thinking about things. @Eigengender says “I don’t think serial computation is a sane metric here. I expect that the total computation represented in a neuron spike is much much higher than the footnote would indicate.” The “Processor clock speeds are not how fast AIs think” post says “In general, I think it’s more sensible for discussion of cognitive capabilities to focus on throughput metrics such as training compute (units of FLOP) and inference compute.”
I certainly agree that if we’re trying to evaluate power we need to consider throughput and total computation. Suppose that a synapse is not a simple numerical weight, and instead we needed to consider each dendritic neurotransmitter gate as a computational unit. This would force us to use many more FLOPs to model a synapse. But would it change the maximum speed? I agree that on a machine of a given size, if you have twice as many floating point operations to do, it will take twice as much time to get through them all. But if we consider the limit where we are not forced to do parallelizable computations in serial, I expect most of the arguments about computational richness are irrelevant?
Perhaps the critics are saying that it takes more serial computations to capture the logic of a serial firing? But the source text admits that this might be the case, and suggests that it might even require 1,000 clock cycles to handle the computations. Is it really so obvious that it takes more than 1000x serial operations to capture a single neuron?
More context: I do think that the human brain is way more powerful (and WAY more efficient) than any current AI system. The extremely crude BOTEC of comparing weights and neocortex synapses says there’s something like a 100x difference, and my guess is that the brain is doing significantly fancier things than a modern transformer, algorithmically.
And of course, training/learning speed may be much more relevant than processing speed, and AFAIK humans are just wildly more data efficient.
And of course the speed at which the AI can approximate the logical action of a single neuron being higher doesn’t imply that the AI will take less time to have each thought. It seems straightforward that machine systems will make decisions in high-speed contexts using quick pathways and they will use any extra thinking speed to think more deeply in contexts where speed isn’t important (much like humans do!).
Anyway, like I said, I’m confused. I respect IABI’s critics and am hoping to learn where my model is wrong.
The passage seems fine to me; I commented on Erdil’s post and other brain efficiency discussions at the time, and I still think that power consumption is a more objective way of comparing performance characteristics of the brain vs. silicon, and that various kinds of FLOP/s comparisons favored by critics of the clock speed argument in the IAB passage are much more fraught ([1], [2]).
It’s true that clock speed (and neuron firing speed) aren’t straightforwardly / directly translatable to “speed of thought”, but both of them are direct proxies for energy consumption and power density. And a very rough BOTEC shows that ~10,000x is a reasonable estimate for the difference in power density between the brain and silicon.
Essentially, the brain is massively underclocked because of design-space restrictions imposed by biology and evolution, whereas silicon-based processing has been running up against fundamental physical limits on component size, clock speed, and power density for a while now. So once AIs can run whatever cognitive algorithms that the brain implements (or algorithms that match the brain in terms of high-level quality of the actual thoughts) at any speed, the already-existing power density difference implies they’ll immediately have a much higher performance ceiling in terms of the throughput and latency that they can run those algorithms at. It’s not a coincidence that making this argument via clock speeds leads to basically the same conclusion as making the same argument via power density.
Essentially, the brain is massively underclocked because of design-space restrictions imposed by biology and evolution
The main restriction is power efficiency: the brain provides a great deal of intelligence for a budget of only ~20 watts. Spreading out that power budget over a very wide memory operating at very slow speed just turns out to be the most power efficient design (vs a very small memory running at very high speed), because memory > time.
My main issue with the source text is that it ignores what is possibly the greater bottleneck in processing speed, which is the time it takes to move information from one area to another. (If my model is right, one of the big advantages of a MoE architecture is to reduce the degree of thrashing weights across the bus to and from the GPU as much, which can be a major bottleneck.) However, on this front I think nerves are still clearly inferior to wires? Even mylenated neurons have a typical speed of only about 100 m/s, while information flows across wires at >50% the speed of light.
Good point actually, and yeah the ability to move information from one area to the other much faster than brains do is arguably why NNs make different tradeoffs than human brains.
I certainly agree that if we’re trying to evaluate power we need to consider throughput and total computation. Suppose that a synapse is not a simple numerical weight, and instead we needed to consider each dendritic neurotransmitter gate as a computational unit. This would force us to use many more FLOPs to model a synapse. But would it change the maximum speed? I agree that on a machine of a given size, if you have twice as many floating point operations to do, it will take twice as much time to get through them all. But if we consider the limit where we are not forced to do parallelizable computations in serial, I expect most of the arguments about computational richness are irrelevant?
For what it’s worth, I wasn’t depending on the premise that a synapse is computationally more powerful than an artificial neuron.
More context: I do think that the human brain is way more powerful (and WAY more efficient) than any current AI system. The extremely crude BOTEC of comparing weights and neocortex synapses says there’s something like a 100x difference, and my guess is that the brain is doing significantly fancier things than a modern transformer, algorithmically.
I actually agree with this take, but critically not in the domain of allowing AIs to think faster, which was my original objection.
@Alexander Gietelink Oldenziel and @S. Alex Bradt and @Max Harms: The thing I was talking about when I agreed with the claim that the brain is more powerful and doing fancier things is basically the fact that the brain always learns and thinks, called continual learning or continual thinking (there is no knowledge cutoff for brains the way current LLMs have), as well as better long-term memory/keeping things in context.
I do agree that in general, human brains aren’t too special algorithmically.
And of course, training/learning speed may be much more relevant than processing speed, and AFAIK humans are just wildly more data efficient.
Do we actually have a source for this, or is this just a commonly believed fact about AIs? I’m getting worried that this claim isn’t actually supported by much evidence and is instead a social belief around AIs due to our previous prediction errors.
I do think AIs can run quite a bit faster than humans, I’m just making the claim that the transistor argument is locally invalid.
Edit: @Max Harms I no longer endorse this objection, and now think my comment on it being utterly false that AI thinking speeds would increase drastically was not correct, and Max H explains why.
Sweet. Thanks for the thoughtful reply! Seems like we mostly agree.
I don’t have a good source on data efficiency, and it’s tagged in my brain as a combination of “a commonly believed thing” and “somewhat apparent in how many epochs of training on a statement it takes to internalize it combined with how weak LLMs are at in-context learning for things like novel board games” but neither of those is very solid and I would not be that surprised to learn that humans are not more data efficient than large transformers that can do similar levels of transfer learning or something. idk.
So it sounds like your issue is not any of the facts (transistor speeds, neuron speeds, AIs faster that humans) but rather the notion that comparing clock speeds and how many times a neuron can spike in a second is not a valid way to reason about whether AI will think faster than humans?
I’m curious what sort of argument you would make to a general audience to convey the idea that AIs will be able to think much faster than humans. Like, what do you think the valid version of the argument looks like?
I actually now think the direct argument given in IABIED was just directionally correct, and I was being confused in my objection, which Max H explains.
I disagree with your dis-endorsement. “Even if it took 1,000 transistor operations to do the work of a single neural spike” seems quite off. Transistor operations are very low level. it probably takes well over 1000, and I wouldn’t rule out that it’s billions.
like a low estimate of the computation happening in the brain is 1 FLOP per *synapse* (1000 synapses per neuron) per neuron fire (~1/second per neuron). that gets you an average of 1000 FLOP/s per neuron, vs “up to 100 fires”. and transistor ops are much lower level than FLOP.
@Max H may have a different take than mine, and I’m curious for his input, but I find myself still thinking about serial operations versus parallel operations. Like, I don’t think it’s particularly important to the question of whether AIs will think faster to ask how many transistors operating in parallel will be needed to capture the equivalent information processing of a single neuron, but rather how many serial computations are needed. I see no reason it would take that many serial operations to capture a single spike, especially in the limit of e.g. specialized chips.
I think the linked tweet is possibly just misinterpreting what the authors meant by “transistor operations”? My reading is that “1000″ binds to “operations”; the actual number of transistors in each operation is unspecified. That’s how they get the 10,000x number—if a CPU runs at 1 GHz, neurons run at 100 Hz, then even if it takes 1000 clock cycles to do the work of neuron, the CPU can still do it 10,000x faster.
(IDK what the rationale was in the editorial process for using “transistor operations” instead of a more standard term like “clock cycles”, but a priori it seems defensible. Speculating, “transistors” was already introduced in the sentence immediately prior, so maybe the thinking was that the meaning and binding of “transistor operations” would be self-evident in context. Whereas if you use “clock cycles” you have to spend a sentence explaining what that means. So using “transistor operations” reduces the total number of new jargon-y / technical terms in the paragraph by one, and also saves a sentence of explanation.)
Anyway, depending on the architecture, precision, etc. a single floating point multiplication can take around 8 clock cycles. So even if a single neuron spike is doing something complicated that requires several high-precision multiply + accumulate operations in serial to replicate, that can easily fit into 1000 clock cycles on a normal CPU, and much fewer if you use specialized hardware.
As for the actual number of transistors themselves needed to do the work of a neuron spike, it again depends on exactly what the neuron spike is doing and how much precision etc. you need to capture the actual work, but “billions” seems too high by a few OOM at least. Some reference points: a single NAND gate is 4 transistors, and a general-purpose 16-bit floating point multiplier unit is ~5k NAND gates.
I think the linked tweet is possibly just misinterpreting what the authors meant by “transistor operations”? My reading is that “1000″ binds to “operations”; the actual number of transistors in each operation is unspecified. That’s how they get the 10,000x number—if a CPU runs at 1 GHz, neurons run at 100 Hz, then even if it takes 1000 clock cycles to do the work of neuron, the CPU can still do it 10,000x faster.
Hmm I see it. I thought it was making a distinct argument from the one Ege was responding to here, but if you’re right it’s the same one.
Then the claim is that an AI run on some (potentially large) cluster of GPUs can think far faster than any human in serial speed. You do lose the rough equivalency between transistors and neurons: a GPU, which is roughly equal to a person in resource costs, happens to have about the same number of transistors as a human brain has neurons. It’s potentially a big deal that AI has a much faster maximum serial speed than humans, but it’s far from clear that such an AI can outwit human society.
If you have a lump of 7,000 neurons, they can each connect to each other neuron, and you can spherical-cow approximate that as a 7000x7000 matrix multiplication. That matrix multiplication will all happen within O(1) spikes, 1⁄100 of a second. That’s ~700GFlop. An H100 GPU takes ~1 millisecond to do that operation, or 1M cycles, to approximate one brain spike cycle! And the gpu has 70B or whatever transistors, so it’s more like 10M transistors per neuron!
I’m guessing the intended meaning of “modern hardware” in the quote is easy to misread (it’s actually just gesturing at the transistors), and for many unintended senses of “modern hardware” (actual AI chips and systems, actual LLMs with KV cache) the claims in the quote are obviously wrong.
The MoE architecture doesn’t just avoid thrashing weights around. It also reduces the amount of calculations per token. For instance, DeepSeek v3.1 has 671B parameters, out of which 37B are activated per token and used in matrices. A model like GPT-3 would use all the 175B parameters it has.
IIRC the human brain makes 1E14 -- 1E15 FLOP/second. The authors of the AI-2027 forecast imply that a human brain creates ~10 tokens/sec, or uses 1E13 -- 1E14 computations per token while having 1E14 synapses.
A more detailed analysis of Yudkowsky’s case for FOOM
If the brain was magically accelerated a million times so that signals reached the speed of 100 million m/s, then the brain would do 1E20 -- 1E21 FLOP/second while doing 1E17 transitions/sec. Cannell’s case for brain efficiency claims that the fundamental baseline irreversible (nano) wire energy is: ~1 Eb/bit/nm, with Eb in the range of 0.1eV (low reliability) to 1eV (high reliability). If reliability is low and each transition is 1E7 nanometers or 1 centimeter, then we need 1E23 EV/second or 1E4 joules/second. IMO this implies that Yudkowsky’s case for a human brain accelerated a million times is as unreliable as Cotra’s case against AI arriving quickly. However, proving that AI is an existential threat is far easier since it requires us to construct an architecture, not to prove that there’s none.
Returning to the human brain being far more powerful or efficient, we notice that it can’t, say, be copied infinitely many times. If it could, one could, say, upload a genius physicist and have an army of its copies work on different projects and exchange insights.
As for the humans being “wildly more data efficient”, Cannell’s post implies that AlphaGo disproves this conjecture with regards to narrow domains like games. What the humans are wildly more efficient is their ability to handle big contexts and to keep the information in mind for more than a single forward pass, as I discussed here and in the collapsible section here.
Yeah, sorry. I should’ve been more clear. I totally agree that there are ways in which brains are super inefficient and weak. I also agree that on restricted domains it’s possible for current AIs to sometimes reach comparable data efficiency.
IABI says: “Transistors, a basic building block of all computers, can switch on and off billions of times per second; unusually fast neurons, by contrast, spike only a hundred times per second. Even if it took 1,000 transistor operations to do the work of a single neural spike, and even if artificial intelligence was limited to modern hardware, that implies human-quality thinking could be emulated 10,000 times faster on a machine— to say nothing of what an AI could do with improved algorithms and improved hardware.
@EigenGender says “aahhhhh this is not how any of this works” and calls it an “egregious error”. Another poster says it’s “utterly false.”
(Relevant online resources text.)
(Potentially relevant LessWrong post.)
I am confused what the issue is, and it would be awesome if someone can explain it to me.
Where I’m coming from, for context:
We don’t know exactly what the relevant logical operations in the human brain are. The model of the brain that says there are binary spiking neurons that have direct connections from synapse->dendrite and that those connections are akin to floating-point numerical weights is clearly a simplification, albeit a powerful one. (IIUC “neural nets” in computers discard the binary-spikes and suggest another model where the spike-rate is akin to a numerical value, which is the basic story behind “neuron activation” in a modern system. This simplification also seems powerful, though it is surely an oversimplification in some ways.)
My main issue with the source text is that it ignores what is possibly the greater bottleneck in processing speed, which is the time it takes to move information from one area to another. (If my model is right, one of the big advantages of a MoE architecture is to reduce the degree of thrashing weights across the bus to and from the GPU as much, which can be a major bottleneck.) However, on this front I think nerves are still clearly inferior to wires? Even mylenated neurons have a typical speed of only about 100 m/s, while information flows across wires at >50% the speed of light.
My read of the critics is that they aren’t objecting to the notion that clock speeds are significantly faster than neurons, but rather that comparing the two is a bad way of thinking about things. @Eigengender says “I don’t think serial computation is a sane metric here. I expect that the total computation represented in a neuron spike is much much higher than the footnote would indicate.” The “Processor clock speeds are not how fast AIs think” post says “In general, I think it’s more sensible for discussion of cognitive capabilities to focus on throughput metrics such as training compute (units of FLOP) and inference compute.”
I certainly agree that if we’re trying to evaluate power we need to consider throughput and total computation. Suppose that a synapse is not a simple numerical weight, and instead we needed to consider each dendritic neurotransmitter gate as a computational unit. This would force us to use many more FLOPs to model a synapse. But would it change the maximum speed? I agree that on a machine of a given size, if you have twice as many floating point operations to do, it will take twice as much time to get through them all. But if we consider the limit where we are not forced to do parallelizable computations in serial, I expect most of the arguments about computational richness are irrelevant?
Perhaps the critics are saying that it takes more serial computations to capture the logic of a serial firing? But the source text admits that this might be the case, and suggests that it might even require 1,000 clock cycles to handle the computations. Is it really so obvious that it takes more than 1000x serial operations to capture a single neuron?
More context: I do think that the human brain is way more powerful (and WAY more efficient) than any current AI system. The extremely crude BOTEC of comparing weights and neocortex synapses says there’s something like a 100x difference, and my guess is that the brain is doing significantly fancier things than a modern transformer, algorithmically.
And of course, training/learning speed may be much more relevant than processing speed, and AFAIK humans are just wildly more data efficient.
And of course the speed at which the AI can approximate the logical action of a single neuron being higher doesn’t imply that the AI will take less time to have each thought. It seems straightforward that machine systems will make decisions in high-speed contexts using quick pathways and they will use any extra thinking speed to think more deeply in contexts where speed isn’t important (much like humans do!).
Anyway, like I said, I’m confused. I respect IABI’s critics and am hoping to learn where my model is wrong.
The passage seems fine to me; I commented on Erdil’s post and other brain efficiency discussions at the time, and I still think that power consumption is a more objective way of comparing performance characteristics of the brain vs. silicon, and that various kinds of FLOP/s comparisons favored by critics of the clock speed argument in the IAB passage are much more fraught ([1], [2]).
It’s true that clock speed (and neuron firing speed) aren’t straightforwardly / directly translatable to “speed of thought”, but both of them are direct proxies for energy consumption and power density. And a very rough BOTEC shows that ~10,000x is a reasonable estimate for the difference in power density between the brain and silicon.
Essentially, the brain is massively underclocked because of design-space restrictions imposed by biology and evolution, whereas silicon-based processing has been running up against fundamental physical limits on component size, clock speed, and power density for a while now. So once AIs can run whatever cognitive algorithms that the brain implements (or algorithms that match the brain in terms of high-level quality of the actual thoughts) at any speed, the already-existing power density difference implies they’ll immediately have a much higher performance ceiling in terms of the throughput and latency that they can run those algorithms at. It’s not a coincidence that making this argument via clock speeds leads to basically the same conclusion as making the same argument via power density.
This is the answer, but
The main restriction is power efficiency: the brain provides a great deal of intelligence for a budget of only ~20 watts. Spreading out that power budget over a very wide memory operating at very slow speed just turns out to be the most power efficient design (vs a very small memory running at very high speed), because memory > time.
My response.
Good point actually, and yeah the ability to move information from one area to the other much faster than brains do is arguably why NNs make different tradeoffs than human brains.
For what it’s worth, I wasn’t depending on the premise that a synapse is computationally more powerful than an artificial neuron.
I actually agree with this take, but critically not in the domain of allowing AIs to think faster, which was my original objection.
@Alexander Gietelink Oldenziel and @S. Alex Bradt and @Max Harms: The thing I was talking about when I agreed with the claim that the brain is more powerful and doing fancier things is basically the fact that the brain always learns and thinks, called continual learning or continual thinking (there is no knowledge cutoff for brains the way current LLMs have), as well as better long-term memory/keeping things in context.
I do agree that in general, human brains aren’t too special algorithmically.
Here are some links as to why below:
lc on why the current lack of long-term memory creates problems, especially for benchmarking
Dwarkesh Patel and Gwern on continual learning/thinking.
Do we actually have a source for this, or is this just a commonly believed fact about AIs? I’m getting worried that this claim isn’t actually supported by much evidence and is instead a social belief around AIs due to our previous prediction errors.
I do think AIs can run quite a bit faster than humans, I’m just making the claim that the transistor argument is locally invalid.Edit: @Max Harms I no longer endorse this objection, and now think my comment on it being utterly false that AI thinking speeds would increase drastically was not correct, and Max H explains why.
Sweet. Thanks for the thoughtful reply! Seems like we mostly agree.
I don’t have a good source on data efficiency, and it’s tagged in my brain as a combination of “a commonly believed thing” and “somewhat apparent in how many epochs of training on a statement it takes to internalize it combined with how weak LLMs are at in-context learning for things like novel board games” but neither of those is very solid and I would not be that surprised to learn that humans are not more data efficient than large transformers that can do similar levels of transfer learning or something. idk.
So it sounds like your issue is not any of the facts (transistor speeds, neuron speeds, AIs faster that humans) but rather the notion that comparing clock speeds and how many times a neuron can spike in a second is not a valid way to reason about whether AI will think faster than humans?
I’m curious what sort of argument you would make to a general audience to convey the idea that AIs will be able to think much faster than humans. Like, what do you think the valid version of the argument looks like?
I actually now think the direct argument given in IABIED was just directionally correct, and I was being confused in my objection, which Max H explains.
It’s fine to use the argument now.
Josh You has some interesting points on this subject. Quoting him for any counterarguments, as well as responses:
@Max H may have a different take than mine, and I’m curious for his input, but I find myself still thinking about serial operations versus parallel operations. Like, I don’t think it’s particularly important to the question of whether AIs will think faster to ask how many transistors operating in parallel will be needed to capture the equivalent information processing of a single neuron, but rather how many serial computations are needed. I see no reason it would take that many serial operations to capture a single spike, especially in the limit of e.g. specialized chips.
I think the linked tweet is possibly just misinterpreting what the authors meant by “transistor operations”? My reading is that “1000″ binds to “operations”; the actual number of transistors in each operation is unspecified. That’s how they get the 10,000x number—if a CPU runs at 1 GHz, neurons run at 100 Hz, then even if it takes 1000 clock cycles to do the work of neuron, the CPU can still do it 10,000x faster.
(IDK what the rationale was in the editorial process for using “transistor operations” instead of a more standard term like “clock cycles”, but a priori it seems defensible. Speculating, “transistors” was already introduced in the sentence immediately prior, so maybe the thinking was that the meaning and binding of “transistor operations” would be self-evident in context. Whereas if you use “clock cycles” you have to spend a sentence explaining what that means. So using “transistor operations” reduces the total number of new jargon-y / technical terms in the paragraph by one, and also saves a sentence of explanation.)
Anyway, depending on the architecture, precision, etc. a single floating point multiplication can take around 8 clock cycles. So even if a single neuron spike is doing something complicated that requires several high-precision multiply + accumulate operations in serial to replicate, that can easily fit into 1000 clock cycles on a normal CPU, and much fewer if you use specialized hardware.
As for the actual number of transistors themselves needed to do the work of a neuron spike, it again depends on exactly what the neuron spike is doing and how much precision etc. you need to capture the actual work, but “billions” seems too high by a few OOM at least. Some reference points: a single NAND gate is 4 transistors, and a general-purpose 16-bit floating point multiplier unit is ~5k NAND gates.
Hmm I see it. I thought it was making a distinct argument from the one Ege was responding to here, but if you’re right it’s the same one.
Then the claim is that an AI run on some (potentially large) cluster of GPUs can think far faster than any human in serial speed. You do lose the rough equivalency between transistors and neurons: a GPU, which is roughly equal to a person in resource costs, happens to have about the same number of transistors as a human brain has neurons. It’s potentially a big deal that AI has a much faster maximum serial speed than humans, but it’s far from clear that such an AI can outwit human society.
If you have a lump of 7,000 neurons, they can each connect to each other neuron, and you can spherical-cow approximate that as a 7000x7000 matrix multiplication. That matrix multiplication will all happen within O(1) spikes, 1⁄100 of a second. That’s ~700GFlop. An H100 GPU takes ~1 millisecond to do that operation, or 1M cycles, to approximate one brain spike cycle! And the gpu has 70B or whatever transistors, so it’s more like 10M transistors per neuron!
Not really—its vector matrix multiplication, not matrix matrix mult.
I’m guessing the intended meaning of “modern hardware” in the quote is easy to misread (it’s actually just gesturing at the transistors), and for many unintended senses of “modern hardware” (actual AI chips and systems, actual LLMs with KV cache) the claims in the quote are obviously wrong.
Ah, I hadn’t thought about that misreading being a source of confusion. Thanks!
The MoE architecture doesn’t just avoid thrashing weights around. It also reduces the amount of calculations per token. For instance, DeepSeek v3.1 has 671B parameters, out of which 37B are activated per token and used in matrices. A model like GPT-3 would use all the 175B parameters it has.
IIRC the human brain makes 1E14 -- 1E15 FLOP/second. The authors of the AI-2027 forecast imply that a human brain creates ~10 tokens/sec, or uses 1E13 -- 1E14 computations per token while having 1E14 synapses.
A more detailed analysis of Yudkowsky’s case for FOOM
If the brain was magically accelerated a million times so that signals reached the speed of 100 million m/s, then the brain would do 1E20 -- 1E21 FLOP/second while doing 1E17 transitions/sec. Cannell’s case for brain efficiency claims that the
fundamentalbaseline irreversible (nano) wire energy is: ~1 Eb/bit/nm, with Eb in the range of 0.1eV (low reliability) to 1eV (high reliability). If reliability is low and each transition is 1E7 nanometers or 1 centimeter, then we need 1E23 EV/second or 1E4 joules/second. IMO this implies that Yudkowsky’s case for a human brain accelerated a million times is as unreliable as Cotra’s case against AI arriving quickly. However, proving that AI is an existential threat is far easier since it requires us to construct an architecture, not to prove that there’s none.Returning to the human brain being far more powerful or efficient, we notice that it can’t, say, be copied infinitely many times. If it could, one could, say, upload a genius physicist and have an army of its copies work on different projects and exchange insights.
As for the humans being “wildly more data efficient”, Cannell’s post implies that AlphaGo disproves this conjecture with regards to narrow domains like games. What the humans are wildly more efficient is their ability to handle big contexts and to keep the information in mind for more than a single forward pass, as I discussed here and in the collapsible section here.
Yeah, sorry. I should’ve been more clear. I totally agree that there are ways in which brains are super inefficient and weak. I also agree that on restricted domains it’s possible for current AIs to sometimes reach comparable data efficiency.
on the bio side, I’d imagine the transistor equivalent is not a neuron spike, but rather a protein binding