the pace of conceptual work on AI algorithms is like >100x faster
In such a case I expect these AI researchers to pick all the low- and medium-hanging fruit at the then-current compute level/hardware technology, and then the algorithmic progress gets saturated until new-gen chips are produced in quality. Check this: https://www.lesswrong.com/posts/sGNFtWbXiLJg2hLzK
Since my Claim 1 is about the conceptual work input being 100x sped up, not some final output being 100x sped up, I’ll take you to be disagreeing with Claim 2. So the question is: is 10 years of thinking about AI algorithms followed by 1 month of retraining sufficient [to get from AI that causes of people to be permanently unemployable to crazy smart AI]? In other words, if one is only going to be able to pick low-hanging and medium-hanging fruit in 10 years, is picking those sufficient to get to crazy smart AI from that point? I claim that the answer is yes; some quick points:
I think we should imagine the fruits at the beginning of this to not have been well-picked (supposing a crazy smart AI does not already exist).
Trusting Byrnes’s decomposition of the 7 year 600x nanochat cost improvement, that’s 6x from hardware and 100x from non-hardware. That would give some sort of baseline guess of for 10 years. Ok, but maybe we should apply some adjustments to the factors. In particular, what about data? On the one hand, it will be tough to collect a lot of data from humans quickly in our scenario. On the other hand, it will be very easy to collect [a lot more data [than we have from humans]] from AIs in this scenario, and by that point this will probably be overall better. On the first hand again, maybe we should imagine data not mattering so much at that point. On the second hand again, all things considered that’s actually conceptually correlated with fooming far past human level quickly. We should also apply some global adjustment down for having less time for experiments to run.
Byrnes explicitly does not include algo ideas that “are not about doing the same thing more efficiently, but rather about trying to do something different instead”. See Section 1.5 of his post. But these clearly should be included in our context here, and are majorly important imo. E.g., curating curricula, creating problems for oneself in a different way, coming up with good ways to reward problem-creation, creating more nested levels of problem-solving with their own rewards, coming up with other ways to make rewards denser / track progress better, creating tools for oneself, various IDA ideas (beyond those already mentioned), etc.. There are also various ways humans get smarter over centuries and over a lifetime that should also count for our purposes as “algo progress” if the AIs can carry them out, e.g. inventing+acquiring new concepts, questions, methods, and skills, and just knowing more.
In our scenario, coming up with an arbitrarily different new AI design is also legitimate, as long as this AI can be created/trained/grown in at most 1 month.
Tbh a lot of my belief that you get a lot of progress just comes from it being an extremely high-dimensional design space and there surely being lots of things one can do so much better in there.
In such a case I expect these AI researchers to pick all the low- and medium-hanging fruit at the then-current compute level/hardware technology, and then the algorithmic progress gets saturated until new-gen chips are produced in quality. Check this: https://www.lesswrong.com/posts/sGNFtWbXiLJg2hLzK
Since my Claim 1 is about the conceptual work input being 100x sped up, not some final output being 100x sped up, I’ll take you to be disagreeing with Claim 2. So the question is: is 10 years of thinking about AI algorithms followed by 1 month of retraining sufficient [to get from AI that causes
I think we should imagine the fruits at the beginning of this to not have been well-picked (supposing a crazy smart AI does not already exist).
Trusting Byrnes’s decomposition of the 7 year 600x nanochat cost improvement, that’s 6x from hardware and 100x from non-hardware. That would give some sort of baseline guess of
Byrnes explicitly does not include algo ideas that “are not about doing the same thing more efficiently, but rather about trying to do something different instead”. See Section 1.5 of his post. But these clearly should be included in our context here, and are majorly important imo. E.g., curating curricula, creating problems for oneself in a different way, coming up with good ways to reward problem-creation, creating more nested levels of problem-solving with their own rewards, coming up with other ways to make rewards denser / track progress better, creating tools for oneself, various IDA ideas (beyond those already mentioned), etc.. There are also various ways humans get smarter over centuries and over a lifetime that should also count for our purposes as “algo progress” if the AIs can carry them out, e.g. inventing+acquiring new concepts, questions, methods, and skills, and just knowing more.
In our scenario, coming up with an arbitrarily different new AI design is also legitimate, as long as this AI can be created/trained/grown in at most 1 month.
Tbh a lot of my belief that you get a lot of progress just comes from it being an extremely high-dimensional design space and there surely being lots of things one can do so much better in there.