Occasionally I hear arguments that ‘scaling will lead to AGI’ or that ‘compute & data limits will prevent the step to AGI’. Or ‘if LLMs knows so much why aren’t they automating every job already’.
This seems straightforwardly false to me and predicated on a conceptual confusion.
A human can be a general intelligence. The compute power of the human brain is probably roughly equivalent to 0.1-10 H100s.
The step to general intelligence is algorithmic not scaling resources. Yes, more compute will yield greater capabilities but fundamentally a dangerous and superintelligent AGI could probably run on your laptop.
As a helpful analogy, think about the difference between a Universal Turing machine and a finite state machine. Larger finite state machine can recognize more complex languages, so scaling a finite state machine will make it more powerful- however there will always be computational tasks that a universal Turing machine can do that a large finite state machine can’t.
The compute power of the human brain is probably roughly equivalent to 0.1-10 H100s.
I’d like to know how this number was estimated.
I think you ignore possibility that scaled brute-force regime can discover general algorithm and then general intelligence refines itself into more efficient form.
fundamentally a dangerous and superintelligent AGI could probably run on your laptop
I’m skeptical that this is true, or at least that it could be confidently predicted to be true based on our current understanding of intelligence. My understanding is that the human brain is much larger in “effective parameter count” than even the largest LLMs (although there’s no 1:1 comparison of neurons to parameters), such that even if my laptop has enough electricity coursing through it to emulate a human brain, it hasn’t got anywhere near enough VRAM. It could be that both AIs and humans are systematically inefficient in some way that, if we understood it, could allow us to produce a more capable general intelligence at <1% the scale of either. But I’m not sure why one would expect this, or what principles imply it.
I think people are frequently mixing in assumptions/claims of how fast we get to ASI if compute and data limits play a role so more dramatic algorithmic improvements are necessary.
And there’s a good reason for doing so. If it takes another ten years, while we’ve got roughly human-level LLM agents, that could be a really good thing for alignment risk (even if it’s pretty bad in other ways and relative to other scenarios).
I agree that based on this we should assume large jumps in capability to be possible (if and when we get said algorithmic progress). I think this doesn’t directly address the ‘scaling will lead to AGI’ claim though: It’s at least plausible that large enough LLMs can be “generally intelligent enough” to outperform humans across the board on general reasoning tasks.
Occasionally I hear arguments that ‘scaling will lead to AGI’ or that ‘compute & data limits will prevent the step to AGI’. Or ‘if LLMs knows so much why aren’t they automating every job already’.
This seems straightforwardly false to me and predicated on a conceptual confusion.
A human can be a general intelligence. The compute power of the human brain is probably roughly equivalent to 0.1-10 H100s.
The step to general intelligence is algorithmic not scaling resources. Yes, more compute will yield greater capabilities but fundamentally a dangerous and superintelligent AGI could probably run on your laptop.
As a helpful analogy, think about the difference between a Universal Turing machine and a finite state machine. Larger finite state machine can recognize more complex languages, so scaling a finite state machine will make it more powerful- however there will always be computational tasks that a universal Turing machine can do that a large finite state machine can’t.
I’d like to know how this number was estimated.
I think you ignore possibility that scaled brute-force regime can discover general algorithm and then general intelligence refines itself into more efficient form.
I’m skeptical that this is true, or at least that it could be confidently predicted to be true based on our current understanding of intelligence. My understanding is that the human brain is much larger in “effective parameter count” than even the largest LLMs (although there’s no 1:1 comparison of neurons to parameters), such that even if my laptop has enough electricity coursing through it to emulate a human brain, it hasn’t got anywhere near enough VRAM. It could be that both AIs and humans are systematically inefficient in some way that, if we understood it, could allow us to produce a more capable general intelligence at <1% the scale of either. But I’m not sure why one would expect this, or what principles imply it.
I think people are frequently mixing in assumptions/claims of how fast we get to ASI if compute and data limits play a role so more dramatic algorithmic improvements are necessary.
And there’s a good reason for doing so. If it takes another ten years, while we’ve got roughly human-level LLM agents, that could be a really good thing for alignment risk (even if it’s pretty bad in other ways and relative to other scenarios).
I agree that based on this we should assume large jumps in capability to be possible (if and when we get said algorithmic progress). I think this doesn’t directly address the ‘scaling will lead to AGI’ claim though: It’s at least plausible that large enough LLMs can be “generally intelligent enough” to outperform humans across the board on general reasoning tasks.