I feel very very confused as to why these timelines are so long. And why just scaling up pretraining more won’t just work and why the primary bottleneck isn’t mainly the physical buildout and gpu availability.
My basic case, available in the linked posts in some more detail and breadth, is:
By default, we have “very long” timelines because we don’t have a strong reason to think AGI gets invented at some specific time.
To update strongly on that, we’d have to get strong evidence that we’ve gotten most or all of the ideas involved in making AGI.
People adduce the rise in capabilities as the main reason that they updated to confident short timelines.
But the rise seems largely explicable as ~imitation + a cherry on top. Or in other words, the inference to “we’ve found most of all of the ideas” is not supported by our observations
People adduce the utility of AI coding in AI R&D as a reason to expect fast future progress.
But this doesn’t say much about how far is left; and I’m pretty skeptical of huge (like >10x, say) speedups on the research tasks that most matter for AGI capabilities progress.
So we should somewhat still be in the prior, though with some substantial update towards short timelines; but not way over to like “90% in 10 years” and similar. That’s unjustified.
You write:
And why just scaling up pretraining more won’t just work
I feel very very confused as to why these timelines are so long. And why just scaling up pretraining more won’t just work and why the primary bottleneck isn’t mainly the physical buildout and gpu availability.
My basic case, available in the linked posts in some more detail and breadth, is:
By default, we have “very long” timelines because we don’t have a strong reason to think AGI gets invented at some specific time.
To update strongly on that, we’d have to get strong evidence that we’ve gotten most or all of the ideas involved in making AGI.
People adduce the rise in capabilities as the main reason that they updated to confident short timelines.
But the rise seems largely explicable as ~imitation + a cherry on top. Or in other words, the inference to “we’ve found most of all of the ideas” is not supported by our observations
People adduce the utility of AI coding in AI R&D as a reason to expect fast future progress.
But this doesn’t say much about how far is left; and I’m pretty skeptical of huge (like >10x, say) speedups on the research tasks that most matter for AGI capabilities progress.
So we should somewhat still be in the prior, though with some substantial update towards short timelines; but not way over to like “90% in 10 years” and similar. That’s unjustified.
You write:
Would you please read https://www.lesswrong.com/posts/sTDfraZab47KiRMmT/views-on-when-agi-comes-and-on-strategy-to-reduce#The__no_blockers__intuition and maybe some surrounding sections?