Biology is incredibly efficient, and generally seems to be near pareto-optimal.
This seems really implausible. I’d like to see a debate about this. E.g. why can’t I improve on heat by having super-cooled fluid pumped throughout my artificial brain; doesn’t having no skull-size limit help a lot; doesn’t metal help; doesn’t it help to not have to worry about immune system stuff; doesn’t it help to be able to maintain full neuroplasticity; etc.
Biology is incredibly efficient at certain things that happen at the cell level. To me, it seems like OP is extrapolating this observation rather too broadly. Human brains are quite inefficient at things they haven’t faced selective pressure to be good at, like matrix multiplication.
Claiming that human brains are near Pareto-optimal efficiency for general intelligence seems like a huge stretch to me. Even assuming that’s true, I’m much more worried about absolute levels of general intelligence rather than intelligence per Watt. Conventional nuclear bombs are dangerous even though they aren’t anywhere near the efficiency of a theoretical antimatter bomb. AI “brains” need not be constrained by the size and energy constraints of a human brain.
Human brains are quite inefficient at things they haven’t faced selective pressure to be good at, like matrix multiplication.
The human brain hardware is essentially a giant analog/digital hybrid vector matrix multiplication engine if you squint the right way, and later neuromorphic hardware for AGI will look similar.
But GPT4 isn’t good at explicit matrix multiplication either.
Your instinct is right. The Landauer limit says that it takes at least kbTln2 energy to erase 1 bit of information, which is necessary to run a function which outputs 1 bit (to erase the output bit). The important thing to note is that it scales with temperature T (measured in an absolute scale). Human brains operate at 310 Kelvin. Ordinary chips can already operate down to around ~230 Kelvin, and there is even a recently developed chip which operates at ~0.02 Kelvin.
So human brains being near the thermodynamic limit in this case means very little about what sort of efficiencies are possible in practice.
Your point about skull-sizes [being bounded by childbirth death risk] seems very strong for evolutionary reasons, and to which I would also add the fact that bird brains seem to do similar amounts of cognition (to smallish mammals) in a much more compact volume without having substantially higher body temperatures (~315 Kelvin).
Cooling the computer doesn’t let you get around the Landauer limit! The savings in energy you get by erasing bits at low temperature are offset by the energy you need to dissipate to keep your computer cold. (Erasing a bit at low temperature still generates some heat, and when you work out how much energy your refrigerator has to use to get rid of that heat, it turns out that you must dissipate the same amount as the Landauer limit says you’d have to if you just erased the bit at ambient temperatures.) To get real savings, you have to actually put your computer in an environment that is naturally colder. For example, if you could put a computer in deep space, that would work.
On the other hand, there might also be other good reasons to keep a computer cold, for example if you want to lower the voltage needed to represent a bit, then keeping your computer cold would plausibly help with that. It just won’t reduce your Landauer-limit-imposed power bill.
None of this is to say that I agree with the rest of Jacob’s analysis of thermodynamic efficiency, I believe he’s made a couple of shaky assumptions and one actual mistake. Since this is getting a lot of attention, I might write a post on it.
Deep space is a poor medium as the only energy dissipation there is radiation, which is slower than convection in Earth. Vacuums are typically used to insulate things (thermos).
That is true, and I concede that that weakens my point.
It still seems to be the case that you could get a ~35% efficiency increase by operating in e.g. Antarctica. I also have this intuition I’ll need to think more about that there are trade-offs with the Landauer limit that could get substantial gains by separating things that are biologically constrained to be close… similar to how a human with an air conditioner can thrive in much hotter environments (using more energy overall, but not energy that has to be in thermal contact with the brain via e.g. the same circulatory system).
Norway/sweden do happen to be currently popular datacenter building locations, but more for cheap power than cooling from what I understand. The problem with Antarctica would be terrible solar production for much of the year.
You can play the same game in the other direction. Given a cold source, you can run your chips hot, and use a steam engine to recapture some of the heat.
I don’t think heat dissipation is actually a limiting factor for humans as things stand right now. Looking at the heat dissipation capabilities of a human brain from three perspectives (maximum possible heat dissipation by sweat glands across the whole body, maximum actual amount of sustained power output by a human in practice, maximum heat transfer from the brain to arterial blood with current-human levels of arterial bloodflow), none of them look to me to be close to the 20w the human brain consumes.
Based on sweat production of athletic people reaching 2L per hour, that gives an estimate of ~1kW of sustained cooling capacity for an entire human
5 watts per kg seems to be pretty close to the maximum power output well-trained humans can actually output in practice for a full hour, so that suggests that a 70 kg human has at least 350 watts of sustained cooling capacity (and probably more, because the limiting factor does not seem to be overheating).
Bloodflow to the brain is about 45L / h, and brains tolerate temperature ranges of 3-4ºC, so working backwards from that we get that a 160W brain would reach temperatures of about 3ºC higher than arterial blood assuming that arterial bloodflow was the primary heat remover. Probably add in 20-100 watts to account for sweat dissipation on the head. And also the carotid artery is less than a cm in diameter, so bloodflow to the brain could probably be substantially increased if there were evolutionary pressure in that direction.
Brains in practice produce about 20W of heat, so it seems likely to me that energy consumption could probably increase by at least one order of magnitude without causing the brain to cook itself, if there was strong enough selection pressure to use that much energy (probably not two orders of magnitude though).
Getting rid of the energy constraint would help though. Proof of concept: ten humans take more energy to run than one human does, and can do more thinking than one human.
I do also find it quite likely that skull size is probably the most tightly binding constraint for humans—we have smaller and very differently tuned neurons compared to other mammals, and I expect that the drive for smaller neurons in particular is downstream of space being very much at a premium, even more so than energy.
Further evidence for the “space, rather than energy expenditure or cooling, is the main binding constraint” hypothesis is the existence of Fontanelles—human brains continue to grow after birth and the skull is not entirely solid in order to allow for that—a skull that does not fully protect the brain seems like a very expensive adaptation, so it’s probably buying something quite valuable.
I note in passing that the elephant brain is not only much larger, but also has many more neurons than any human brain. Since I’ve no reason to believe the elephant brain is maximally efficient, making the same claim for our brains should require much more evidence than I’m seeing.
That’s if you’re counting the cerebellum, which doesn’t seem to contribute much to intelligence, but is important for controlling the complicated musculature of a trunk and large body.
By cortical neuron count, humans have about 18 billion, while elephants have less than 6, comparable to a chimpanzee. (source)
Elephants are undeniably intelligent as animals go, but not at human level.
Even blue whales barely approach human level by cortical neuron count, although some cetaceans (notably orcas) exceed it.
The brain is perhaps 1 to 2 OOM larger than the physical limits for a computer of equivalent power, but is constrained to its somewhat larger than minimal size due in part to thermodynamic cooling considerations.
I conclude something more like “the brain consumes perhaps 1 to 2 OOM less energy than the biological limits of energy density for something of its size, but is constrained to its somewhat lower than maximal energy density due in part to energy availability considerations” but I suspect that this is more of a figure/ground type of disagreement about which things are salient to look at vs a factual disagreement.
That said @jacob_cannell is likely to be much more informed in this space than I am—if the thermodynamic cooling considerations actually bind much more tightly than I thought, I’d be interested to know that (although not necessarily immediately, I expect that he’s dealing with rather a lot of demands on his time that are downstream of kicking the hornet’s nest here).
efficient for the temperature it runs at. Jake is correct about the fundamental comparison, but he’s leaving off the part where he expects reversible computing to fundamentally change the efficiency tradeoffs for intelligence eventually, which is essentially “the best way to make use of near perfect cooling” as a research field; I don’t have a link to where he’s said this before, since I’m remembering conversations we had out loud.
it’s relevant in that there’s a lot of room to improve, it’s just not at the same energy budget and temperature. I’m not trying to imply a big hidden iceberg in addition to that claim; what it implies is up to your analysis.
Then how is that relevant to the argument in your OP?
I thought you were arguing:
Yudkowsky argues that there’s lots of room at the top for hardware that provides much more compute than human brains, and therefore supports much greater intelligence than humans. However, actually biology is efficient. Therefore there’s not much room at the top for hardware that provides much more compute.
That’s what I responded to in my top-level comment. Is that not what you’re arguing? If it is what you’re arguing, then I’m confused because it seems like here in this comment you’re talking about something irrelevant and not responding to my comment (though I could be confused about that as well!).
The specific line where I said “biology is incredibly efficient, and generally seems to be near pareto-optimal”, occurs immediately after and is mainly referring to the EY claim that “biology is not that efficient”, and his more specific claim about thermodynamic efficiency—which I already spent a whole long post refuting.
None of your suggestions:
E.g. why can’t I improve on heat by having super-cooled fluid pumped throughout my artificial brain; doesn’t having no skull-size limit help a lot; doesn’t metal help; doesn’t it help to not have to worry about immune system stuff; doesn’t it help to be able to maintain full neuroplasticity;
Improve thermodynamic efficiency, nor do they matter much in terms of OOM. EY’s argument is essentially that AGI will quickly find many OOM software improvement, and then many more OOM improvement via new nanotech hardware.
This seems really implausible. I’d like to see a debate about this. E.g. why can’t I improve on heat by having super-cooled fluid pumped throughout my artificial brain; doesn’t having no skull-size limit help a lot; doesn’t metal help; doesn’t it help to not have to worry about immune system stuff; doesn’t it help to be able to maintain full neuroplasticity; etc.
Biology is incredibly efficient at certain things that happen at the cell level. To me, it seems like OP is extrapolating this observation rather too broadly. Human brains are quite inefficient at things they haven’t faced selective pressure to be good at, like matrix multiplication.
Claiming that human brains are near Pareto-optimal efficiency for general intelligence seems like a huge stretch to me. Even assuming that’s true, I’m much more worried about absolute levels of general intelligence rather than intelligence per Watt. Conventional nuclear bombs are dangerous even though they aren’t anywhere near the efficiency of a theoretical antimatter bomb. AI “brains” need not be constrained by the size and energy constraints of a human brain.
The human brain hardware is essentially a giant analog/digital hybrid vector matrix multiplication engine if you squint the right way, and later neuromorphic hardware for AGI will look similar.
But GPT4 isn’t good at explicit matrix multiplication either.
>But GPT4 isn’t good at explicit matrix multiplication either.
So it is also very inefficient.
Probably a software problem.
Your instinct is right. The Landauer limit says that it takes at least kbTln2 energy to erase 1 bit of information, which is necessary to run a function which outputs 1 bit (to erase the output bit). The important thing to note is that it scales with temperature T (measured in an absolute scale). Human brains operate at 310 Kelvin. Ordinary chips can already operate down to around ~230 Kelvin, and there is even a recently developed chip which operates at ~0.02 Kelvin.
So human brains being near the thermodynamic limit in this case means very little about what sort of efficiencies are possible in practice.
Your point about skull-sizes [being bounded by childbirth death risk] seems very strong for evolutionary reasons, and to which I would also add the fact that bird brains seem to do similar amounts of cognition (to smallish mammals) in a much more compact volume without having substantially higher body temperatures (~315 Kelvin).
Cooling the computer doesn’t let you get around the Landauer limit! The savings in energy you get by erasing bits at low temperature are offset by the energy you need to dissipate to keep your computer cold. (Erasing a bit at low temperature still generates some heat, and when you work out how much energy your refrigerator has to use to get rid of that heat, it turns out that you must dissipate the same amount as the Landauer limit says you’d have to if you just erased the bit at ambient temperatures.) To get real savings, you have to actually put your computer in an environment that is naturally colder. For example, if you could put a computer in deep space, that would work.
On the other hand, there might also be other good reasons to keep a computer cold, for example if you want to lower the voltage needed to represent a bit, then keeping your computer cold would plausibly help with that. It just won’t reduce your Landauer-limit-imposed power bill.
None of this is to say that I agree with the rest of Jacob’s analysis of thermodynamic efficiency, I believe he’s made a couple of shaky assumptions and one actual mistake. Since this is getting a lot of attention, I might write a post on it.
Deep space is a poor medium as the only energy dissipation there is radiation, which is slower than convection in Earth. Vacuums are typically used to insulate things (thermos).
In a room temp bath this always costs more energy—there is no free lunch in cooling. However in the depths of outer space this may become relevant.
That is true, and I concede that that weakens my point.
It still seems to be the case that you could get a ~35% efficiency increase by operating in e.g. Antarctica. I also have this intuition I’ll need to think more about that there are trade-offs with the Landauer limit that could get substantial gains by separating things that are biologically constrained to be close… similar to how a human with an air conditioner can thrive in much hotter environments (using more energy overall, but not energy that has to be in thermal contact with the brain via e.g. the same circulatory system).
Norway/sweden do happen to be currently popular datacenter building locations, but more for cheap power than cooling from what I understand. The problem with Antarctica would be terrible solar production for much of the year.
You can play the same game in the other direction. Given a cold source, you can run your chips hot, and use a steam engine to recapture some of the heat.
The Landauer limit still applies.
I don’t think heat dissipation is actually a limiting factor for humans as things stand right now. Looking at the heat dissipation capabilities of a human brain from three perspectives (maximum possible heat dissipation by sweat glands across the whole body, maximum actual amount of sustained power output by a human in practice, maximum heat transfer from the brain to arterial blood with current-human levels of arterial bloodflow), none of them look to me to be close to the 20w the human brain consumes.
Based on sweat production of athletic people reaching 2L per hour, that gives an estimate of ~1kW of sustained cooling capacity for an entire human
5 watts per kg seems to be pretty close to the maximum power output well-trained humans can actually output in practice for a full hour, so that suggests that a 70 kg human has at least 350 watts of sustained cooling capacity (and probably more, because the limiting factor does not seem to be overheating).
Bloodflow to the brain is about 45L / h, and brains tolerate temperature ranges of 3-4ºC, so working backwards from that we get that a 160W brain would reach temperatures of about 3ºC higher than arterial blood assuming that arterial bloodflow was the primary heat remover. Probably add in 20-100 watts to account for sweat dissipation on the head. And also the carotid artery is less than a cm in diameter, so bloodflow to the brain could probably be substantially increased if there were evolutionary pressure in that direction.
Brains in practice produce about 20W of heat, so it seems likely to me that energy consumption could probably increase by at least one order of magnitude without causing the brain to cook itself, if there was strong enough selection pressure to use that much energy (probably not two orders of magnitude though).
Getting rid of the energy constraint would help though. Proof of concept: ten humans take more energy to run than one human does, and can do more thinking than one human.
I do also find it quite likely that skull size is probably the most tightly binding constraint for humans—we have smaller and very differently tuned neurons compared to other mammals, and I expect that the drive for smaller neurons in particular is downstream of space being very much at a premium, even more so than energy.
Further evidence for the “space, rather than energy expenditure or cooling, is the main binding constraint” hypothesis is the existence of Fontanelles—human brains continue to grow after birth and the skull is not entirely solid in order to allow for that—a skull that does not fully protect the brain seems like a very expensive adaptation, so it’s probably buying something quite valuable.
I note in passing that the elephant brain is not only much larger, but also has many more neurons than any human brain. Since I’ve no reason to believe the elephant brain is maximally efficient, making the same claim for our brains should require much more evidence than I’m seeing.
That’s if you’re counting the cerebellum, which doesn’t seem to contribute much to intelligence, but is important for controlling the complicated musculature of a trunk and large body.
By cortical neuron count, humans have about 18 billion, while elephants have less than 6, comparable to a chimpanzee. (source)
Elephants are undeniably intelligent as animals go, but not at human level.
Even blue whales barely approach human level by cortical neuron count, although some cetaceans (notably orcas) exceed it.
jacob_cannell’s post here https://www.lesswrong.com/posts/xwBuoE9p8GE7RAuhd/brain-efficiency-much-more-than-you-wanted-to-know#Space argues that:
Does that seem about right to you?
I conclude something more like “the brain consumes perhaps 1 to 2 OOM less energy than the biological limits of energy density for something of its size, but is constrained to its somewhat lower than maximal energy density due in part to energy availability considerations” but I suspect that this is more of a figure/ground type of disagreement about which things are salient to look at vs a factual disagreement.
That said @jacob_cannell is likely to be much more informed in this space than I am—if the thermodynamic cooling considerations actually bind much more tightly than I thought, I’d be interested to know that (although not necessarily immediately, I expect that he’s dealing with rather a lot of demands on his time that are downstream of kicking the hornet’s nest here).
efficient for the temperature it runs at. Jake is correct about the fundamental comparison, but he’s leaving off the part where he expects reversible computing to fundamentally change the efficiency tradeoffs for intelligence eventually, which is essentially “the best way to make use of near perfect cooling” as a research field; I don’t have a link to where he’s said this before, since I’m remembering conversations we had out loud.
But how is “efficient for the temperature it runs at” relevant to whether there’s much room to improve on how much compute biology provides?
it’s relevant in that there’s a lot of room to improve, it’s just not at the same energy budget and temperature. I’m not trying to imply a big hidden iceberg in addition to that claim; what it implies is up to your analysis.
Near pareto-optimal in terms of thermodynamic efficiency as replicators and nanobots, see the discussions and links here and here.
Then how is that relevant to the argument in your OP?
I thought you were arguing:
That’s what I responded to in my top-level comment. Is that not what you’re arguing? If it is what you’re arguing, then I’m confused because it seems like here in this comment you’re talking about something irrelevant and not responding to my comment (though I could be confused about that as well!).
The specific line where I said “biology is incredibly efficient, and generally seems to be near pareto-optimal”, occurs immediately after and is mainly referring to the EY claim that “biology is not that efficient”, and his more specific claim about thermodynamic efficiency—which I already spent a whole long post refuting.
None of your suggestions:
Improve thermodynamic efficiency, nor do they matter much in terms of OOM. EY’s argument is essentially that AGI will quickly find many OOM software improvement, and then many more OOM improvement via new nanotech hardware.