I’m not sure I get it. What are the automated researchers doing that makes frontier progress move so much faster? They’re coming up with experiment designs and inventions which are so much better than the human teams designs and inventions that they need half or a quarter the number of experiments per success? But they’ve (ex hypothesi) only just got to automated researcher level. Where is that taste coming from?
I think you get taste almost entirely from experience (or second-hand records) of relevantly-similar experiments. You can also reason to better experiment design, but that looks to have steeply diminishing returns both empirically and theoretically. So it’s almost entirely about experimental compute supply and sample-efficiency of taste-accumulation.
I do think there’s likely to be something like a speed/efficiency boost (perhaps explosion… unsure) from automated coding (again dependent on experimental verification, but this is one where you can extrapolate decently well from the prior art of compute optimisation). So maybe that’s enough to massively increase experimental compute budget, effectively multiplying experimental throughput. Perhaps this alone is enough to get the overall rate of progress up—returns to experimental throughput are pretty good?
Additionally, maybe once enough context is logged and that training pipeline is in place, taste-accrual is just better even if sample-efficiency is worse, because the AI sees all the context of all experiments, as opposed to human teams, which have some discount for needing to communicate between people and such.
On TEDAI—what are the automated researchers automating? How does that end up colonising all other capabilities? Where are they getting the data?
On TEDAI—what are the automated researchers automating? How does that end up colonising all other capabilities? Where are they getting the data?
They are making methods for creating general intelligence combined with methods for generally making AIs better at specific downstream domains (e.g., mechanical engineering). This requires data collection in that downstream domains, but I’d guess this data collection is pretty easy (e.g., you can deploy the AI to learn huge amounts in parallel in addition to learnings as much as possible in sim and you can compensate for weakness with additional time/effort or with specific superhuman abilities in many cases).
I’m not sure I get it. What are the automated researchers doing that makes frontier progress move so much faster? They’re coming up with experiment designs and inventions which are so much better than the human teams designs and inventions that they need half or a quarter the number of experiments per success? But they’ve (ex hypothesi) only just got to automated researcher level. Where is that taste coming from?
I think you get taste almost entirely from experience (or second-hand records) of relevantly-similar experiments. You can also reason to better experiment design, but that looks to have steeply diminishing returns both empirically and theoretically. So it’s almost entirely about experimental compute supply and sample-efficiency of taste-accumulation.
I do think there’s likely to be something like a speed/efficiency boost (perhaps explosion… unsure) from automated coding (again dependent on experimental verification, but this is one where you can extrapolate decently well from the prior art of compute optimisation). So maybe that’s enough to massively increase experimental compute budget, effectively multiplying experimental throughput. Perhaps this alone is enough to get the overall rate of progress up—returns to experimental throughput are pretty good?
Additionally, maybe once enough context is logged and that training pipeline is in place, taste-accrual is just better even if sample-efficiency is worse, because the AI sees all the context of all experiments, as opposed to human teams, which have some discount for needing to communicate between people and such.
On TEDAI—what are the automated researchers automating? How does that end up colonising all other capabilities? Where are they getting the data?
I think your questions are mostly better answered by How quick and big would a software intelligence explosion be? and AI Futures Model.
They are making methods for creating general intelligence combined with methods for generally making AIs better at specific downstream domains (e.g., mechanical engineering). This requires data collection in that downstream domains, but I’d guess this data collection is pretty easy (e.g., you can deploy the AI to learn huge amounts in parallel in addition to learnings as much as possible in sim and you can compensate for weakness with additional time/effort or with specific superhuman abilities in many cases).