I notice I am confused. I generally think that after full automation of AI R&D, AI progress will be approximately linear in compute, so e.g. 10xing compute would make progress go like 8x faster overall rather than e.g. simply adding 0.5 years of progress per year to the many years of progress per year already being produced. It seems like you disagree? Or maybe I’m just mathing wrong. Exponentials are unintuitive.
So 3X more training compute would be a 2X speed-up. Could bump to 3X speed-up due to the additional runtime compute. So overall this would make the slower tail-end of the IE happen 2-3X faster.
I.e. roughly linear.
So this does mean the IE happens faster. I.e. 10 years in 6 months rather than in 12 months.
But i was then commenting on how long it goes on for. Where i think the extra compute makes less difference between once r<1 things slow down fairly quickly. So you maybe still only get ~11 years in 12 months.
Could use the online tool to figure this out. Just do two runs, and in one of them double the ‘initial speed’. That has a similar effect to doubling compute.
I notice I am confused. I generally think that after full automation of AI R&D, AI progress will be approximately linear in compute, so e.g. 10xing compute would make progress go like 8x faster overall rather than e.g. simply adding 0.5 years of progress per year to the many years of progress per year already being produced. It seems like you disagree? Or maybe I’m just mathing wrong. Exponentials are unintuitive.
Think we agree on that.
My last comment says:
I.e. roughly linear.
So this does mean the IE happens faster. I.e. 10 years in 6 months rather than in 12 months.
But i was then commenting on how long it goes on for. Where i think the extra compute makes less difference between once r<1 things slow down fairly quickly. So you maybe still only get ~11 years in 12 months.
Could use the online tool to figure this out. Just do two runs, and in one of them double the ‘initial speed’. That has a similar effect to doubling compute.