It also lines up with a speculation I’ve always had. In theory LLMs are predictors, but in practice, are they pretty much imitators?
Yep, I think that’s basically the case.
@nostalgebraist makes an excellent point that eliciting any latent superhuman capabilities which bigger models might have is an art of its own, and that “just train chatbots” doesn’t exactly cut it, for this task. Maybe that’s where some additional capabilities progress might still come from.
But the AI industry (both AGI labs and the menagerie of startups and open-source enthusiasts) has so far been either unwilling or unable to move past the chatbot paradigm.
(Also, I would tentatively guess that this type of progress is not existentially threatening. It’d yield us a bunch of nice software tools, not a lightcone-eating monstrosity.)
I agree that chatbot progress is probably not existentially threatening. But it’s all too short a leap to making chatbots power general agents. The labs have claimed to be willing and enthusiastic about moving to an agent paradigm. And I’m afraid that a proliferation of even weakly superhuman or even roughly parahuman agents could be existentially threatening.
I spell out my logic for how short the leap might be from current chatbots to takeover-capable AGI agents in my argument for short timelines being quite possible. I do think we’ve still got a good shot of aligning that type of LLM agent AGI since it’s a nearly best-case scenario. RL even in o1 is really mostly used for making it accurately follow instructions, which is at least roughly the ideal alignment goal of Corrigibility as Singular Target. Even if we lose faithful chain of thought and orgs don’t take alignment that seriously, I think those advantages of not really being a maximizer and having corrigibility might win out.
That in combination with the slower takeoff make me tempted to believe its actually a good thing if we forge forward, even though I’m not at all confident that this will actually get us aligned AGI or good outcomes. I just don’t see a better realistic path.
Yep, I think that’s basically the case.
@nostalgebraist makes an excellent point that eliciting any latent superhuman capabilities which bigger models might have is an art of its own, and that “just train chatbots” doesn’t exactly cut it, for this task. Maybe that’s where some additional capabilities progress might still come from.
But the AI industry (both AGI labs and the menagerie of startups and open-source enthusiasts) has so far been either unwilling or unable to move past the chatbot paradigm.
(Also, I would tentatively guess that this type of progress is not existentially threatening. It’d yield us a bunch of nice software tools, not a lightcone-eating monstrosity.)
I agree that chatbot progress is probably not existentially threatening. But it’s all too short a leap to making chatbots power general agents. The labs have claimed to be willing and enthusiastic about moving to an agent paradigm. And I’m afraid that a proliferation of even weakly superhuman or even roughly parahuman agents could be existentially threatening.
I spell out my logic for how short the leap might be from current chatbots to takeover-capable AGI agents in my argument for short timelines being quite possible. I do think we’ve still got a good shot of aligning that type of LLM agent AGI since it’s a nearly best-case scenario. RL even in o1 is really mostly used for making it accurately follow instructions, which is at least roughly the ideal alignment goal of Corrigibility as Singular Target. Even if we lose faithful chain of thought and orgs don’t take alignment that seriously, I think those advantages of not really being a maximizer and having corrigibility might win out.
That in combination with the slower takeoff make me tempted to believe its actually a good thing if we forge forward, even though I’m not at all confident that this will actually get us aligned AGI or good outcomes. I just don’t see a better realistic path.