I agree on 1 and 2. My comment was to suggest that I think 3 has lots of the interesting detail in it, and is under-discussed in the original.
Why does 10-100M feel sufficient to you, where 1M does not?
My weakly held intuition is that, as a general intelligence myself, if I had my long-term memory reset every day, with the ability to store ‘only’ 1 million words of notes between days, I’d still be able to make a lot of progress on large, ambiguous tasks, like my current software engineering job.
Imagine that you forgot everything about your current company completely between days, not just current task. Every day is like your first day on the job (coming as an experienced dev from different companies). But you can store 1 million words of notes. The notes aren’t carefully curated—they’re just the last million words you happened to write in your notes, and you take notes on everything. Do you still think you’d make decent progress?
LLMs are able to curate their notes, though? Compaction of conversations and multi-agent hierarchies already work. They might not work well enough according to some standard of performance you have in mind, but it’s an incorrect comparison to say that we (the LLM and I) can’t curate notes.
I agree on 1 and 2. My comment was to suggest that I think 3 has lots of the interesting detail in it, and is under-discussed in the original.
Why does 10-100M feel sufficient to you, where 1M does not?
My weakly held intuition is that, as a general intelligence myself, if I had my long-term memory reset every day, with the ability to store ‘only’ 1 million words of notes between days, I’d still be able to make a lot of progress on large, ambiguous tasks, like my current software engineering job.
Imagine that you forgot everything about your current company completely between days, not just current task. Every day is like your first day on the job (coming as an experienced dev from different companies). But you can store 1 million words of notes. The notes aren’t carefully curated—they’re just the last million words you happened to write in your notes, and you take notes on everything. Do you still think you’d make decent progress?
LLMs are able to curate their notes, though? Compaction of conversations and multi-agent hierarchies already work. They might not work well enough according to some standard of performance you have in mind, but it’s an incorrect comparison to say that we (the LLM and I) can’t curate notes.
They can. But yes, it doesn’t work great. Adjust the metaphor as necessary