I think the algorithmic progress isn’t as fast as you say, at least not in the sense relevant for progress towards TAI/AGI.
Algorithmic improvements can be divided into two types, I think: Doing what we already know how to do more efficiently, and exploring new stuff more efficiently. I think lots of the improvements we’ve seen so far are of the first category, but the second category is what’s relevant for TAI/AGI. I’m not sure what the ratio is but my guess is it’s 50⁄50 or so. I’d love to see someone tackle this question and come up with a number.
That said, once we get close to TAI/AGI we should expect the first category to kick in and make it much cheaper very quickly.
My algorithmic estimates essentially only quantify your “first category” type of improvements, I wouldn’t know where to begin making estimates for qualitative “second category” AGI algorithmic progress.
My comparisons to human level NLP (which I don’t think would necessarily yield AGI) assumed scaling would hold for current (or near future) techniques, so do you think that current techniques won’t scale, or/and that the actual 100-1000x figure I gave was too high?
I’m not sure what the ratio is but my guess is it’s 50⁄50 or so. I’d love to see someone tackle this question and come up with a number.
I think the algorithmic progress isn’t as fast as you say, at least not in the sense relevant for progress towards TAI/AGI.
Algorithmic improvements can be divided into two types, I think: Doing what we already know how to do more efficiently, and exploring new stuff more efficiently. I think lots of the improvements we’ve seen so far are of the first category, but the second category is what’s relevant for TAI/AGI. I’m not sure what the ratio is but my guess is it’s 50⁄50 or so. I’d love to see someone tackle this question and come up with a number.
That said, once we get close to TAI/AGI we should expect the first category to kick in and make it much cheaper very quickly.
My algorithmic estimates essentially only quantify your “first category” type of improvements, I wouldn’t know where to begin making estimates for qualitative “second category” AGI algorithmic progress.
My comparisons to human level NLP (which I don’t think would necessarily yield AGI) assumed scaling would hold for current (or near future) techniques, so do you think that current techniques won’t scale, or/and that the actual 100-1000x figure I gave was too high?
Yeah that would be great if someone did that.