Fantastic. I’m listening to it now, because this feels important. Dwarkesh is a huge voice and his skepticism about RSI and fast timelines may make a difference for attitudes in general, and therefore caution.
At 14:29 he says he gives his main crux: skepticism that intelligence is that intelligence is that important for AI progress. He’s arguing that compute is dominant over researcher intelligence.
This seems pretty strange to me. Porque no los dos? Which is pretty much your answer. Obviously compute is super important. And it seems equally important that AI research depends on intelligence. The idea that there are no more good ideas and it’s all just scaling from here seems so wildly counterintuitive to me that I have trouble understanding the mindset.
This is probably at root based on my perspective on human brain function. Scale is important. To a large extent, human intelligence is just a result of scaling up primate intelligence. But even primate intelligence involves a lot more moving parts than the general learning mechanisms of LLMs. At the very least there are a bunch of motivational tricks that direct our continual learning very efficiently. And it’s pretty inarguable that episodic memory and continual learning play a big role in human intelligence, on top of core learning mechanisms that are (much more arguably) fairly similar to the combination of predictive and RL learning used to train LLMs.
Anyway that’s just one intuition pump for the intuition that we haven’t likely discovered nearly all of the tricks that will make scaffolded LLMs effectively smarter/more competent.
Actually, it seems contradictory if I’m right that Dwarkesh believes ASI will need continual learning (or mountains of new RLVR data). Why doesn’t he think intelligence won’t help create better CL mechanisms? I guess he’s assuming the RLVR method beats the CL method. I assume the opposite. I think well-selected fine-tuning will work just fine for CL and it’s the likely breakthrough.
The idea that there are no more good ideas and it’s all just scaling from here
Fwiw, this is not how I understand his take. I think he’s saying AI progress will be bottlenecked by compute (and human expert data), which I interpret to mean that the elasticity of substitution between compute and intelligence isn’t high enough. In fact, I think this his view includes compute to run experiments to do AI research.
Right but it seems that increased intelligence will reduce that bottleneck by making the limited amount of compute for experiments go farther. How much farther is an open question.
Fantastic. I’m listening to it now, because this feels important. Dwarkesh is a huge voice and his skepticism about RSI and fast timelines may make a difference for attitudes in general, and therefore caution.
At 14:29 he says he gives his main crux: skepticism that intelligence is that intelligence is that important for AI progress. He’s arguing that compute is dominant over researcher intelligence.
This seems pretty strange to me. Porque no los dos? Which is pretty much your answer. Obviously compute is super important. And it seems equally important that AI research depends on intelligence. The idea that there are no more good ideas and it’s all just scaling from here seems so wildly counterintuitive to me that I have trouble understanding the mindset.
This is probably at root based on my perspective on human brain function. Scale is important. To a large extent, human intelligence is just a result of scaling up primate intelligence. But even primate intelligence involves a lot more moving parts than the general learning mechanisms of LLMs. At the very least there are a bunch of motivational tricks that direct our continual learning very efficiently. And it’s pretty inarguable that episodic memory and continual learning play a big role in human intelligence, on top of core learning mechanisms that are (much more arguably) fairly similar to the combination of predictive and RL learning used to train LLMs.
Anyway that’s just one intuition pump for the intuition that we haven’t likely discovered nearly all of the tricks that will make scaffolded LLMs effectively smarter/more competent.
Actually, it seems contradictory if I’m right that Dwarkesh believes ASI will need continual learning (or mountains of new RLVR data). Why doesn’t he think intelligence won’t help create better CL mechanisms? I guess he’s assuming the RLVR method beats the CL method. I assume the opposite. I think well-selected fine-tuning will work just fine for CL and it’s the likely breakthrough.
Fwiw, this is not how I understand his take. I think he’s saying AI progress will be bottlenecked by compute (and human expert data), which I interpret to mean that the elasticity of substitution between compute and intelligence isn’t high enough. In fact, I think this his view includes compute to run experiments to do AI research.
Right but it seems that increased intelligence will reduce that bottleneck by making the limited amount of compute for experiments go farther. How much farther is an open question.
I agree with that!