I’m not sure the situation in which LLMs are not good at crystallised intelligence is good for AI Safety. Or at least, it is good in the sense that timelines are delayed by a few years but makes our odds of surviving an AGI worse.
A core component of modern technical AI safety is studying LLMs. Major figures in the field seem to be under the impression that their experiments will extrapolate to future models, at least to some extent. As a concrete example, the team at Google DeepMind endorses a safety approach in which “there will not be large discontinuous jumps in general AI capabilities”(pg 3). Authors on this paper include Rohin Shah, Neel Nanda and Victoria Krakovna.*
My concern is that, should LLMs be terrible at fluid intelligence and are fundamentally inefficient learners, then we’ll see a temporary slow in progress. But this leads to a strong economic incentive to explore alternate architectures and at some point one is found which learns faster with less data and energy. (This is Yann LeCun’s current agenda.)
The end result is we will see a sudden jump in AI capabilities that we are unprepared for.
*This isn’t to single out DeepMind’s team, their paper just came to mind.
I wonder if a similar error is why Ants seem so confident in a very fast takeoff—they assume the models are better at fluid intelligence than they actually are, because their capabilities are strongest in the domain Ants are best at evaluating.
I’m not sure the situation in which LLMs are not good at crystallised intelligence is good for AI Safety. Or at least, it is good in the sense that timelines are delayed by a few years but makes our odds of surviving an AGI worse.
A core component of modern technical AI safety is studying LLMs. Major figures in the field seem to be under the impression that their experiments will extrapolate to future models, at least to some extent. As a concrete example, the team at Google DeepMind endorses a safety approach in which “there will not be large discontinuous jumps in general AI capabilities”(pg 3). Authors on this paper include Rohin Shah, Neel Nanda and Victoria Krakovna.*
My concern is that, should LLMs be terrible at fluid intelligence and are fundamentally inefficient learners, then we’ll see a temporary slow in progress. But this leads to a strong economic incentive to explore alternate architectures and at some point one is found which learns faster with less data and energy. (This is Yann LeCun’s current agenda.)
The end result is we will see a sudden jump in AI capabilities that we are unprepared for.
*This isn’t to single out DeepMind’s team, their paper just came to mind.
Oof, yeah, seems overconfident.
I wonder if a similar error is why Ants seem so confident in a very fast takeoff—they assume the models are better at fluid intelligence than they actually are, because their capabilities are strongest in the domain Ants are best at evaluating.
(Does Ants mean Anthropic here?)
yep—sorry, I thought that was common slang. I think it’s what they call themselves too