I think that if we retain the architecture of current LLMs, we will be in world one. I have two reasons. First, the architecture of current LLMs place a limit on how much information they can retain about the task at hand. They have memory of a prompt (both the system prompt and your task-specific prompt) plus the memory of everything they’ve said so far. When what they’ve said so far gets long enough, they attend mostly to what they’ve already said, rather than attending to the prompt. Then they wander off into La-La land. Second, the problem may also be inherent in their training methods. In the first (and largest) part of their training, they’re trained to predict the next word from a snippet of English text. A few years ago, these snippets were a sentence or a paragraph. They’ve gotten longer recently, but I don’t think they amount to entire books yet (readers, please tell us if you know). So it’s never seen a text that’s coherent over longer than its snippet length. It seems unsurprising that it doesn’t know how to remain coherent indefinitely. People have tried preventing these phenomena by various schemes, such as telling the LLM to prepare summaries for later expansion, or periodically reminding it of the task at hand. So far these haven’t been enough to make indefinitely long tasks feasible. Of course, there are lots of smart people working on this, and we could transition from world one to world two at any moment.
I think that if we retain the architecture of current LLMs, we will be in world one. I have two reasons.
First, the architecture of current LLMs place a limit on how much information they can retain about the task at hand. They have memory of a prompt (both the system prompt and your task-specific prompt) plus the memory of everything they’ve said so far. When what they’ve said so far gets long enough, they attend mostly to what they’ve already said, rather than attending to the prompt. Then they wander off into La-La land.
Second, the problem may also be inherent in their training methods. In the first (and largest) part of their training, they’re trained to predict the next word from a snippet of English text. A few years ago, these snippets were a sentence or a paragraph. They’ve gotten longer recently, but I don’t think they amount to entire books yet (readers, please tell us if you know). So it’s never seen a text that’s coherent over longer than its snippet length. It seems unsurprising that it doesn’t know how to remain coherent indefinitely.
People have tried preventing these phenomena by various schemes, such as telling the LLM to prepare summaries for later expansion, or periodically reminding it of the task at hand. So far these haven’t been enough to make indefinitely long tasks feasible. Of course, there are lots of smart people working on this, and we could transition from world one to world two at any moment.