I agree it helps to run experiments at small scales first, but I’d be pretty surprised if that helped to the point of enabling a 30x speedup—that means that the AI labor allows you get 30x improvement in compute needed beyond what would be done by default by humans (though the 30x can include e.g. improving utilization, it’s not limited just to making individual experiments take less time).
I think the most plausible case for your position would be that the compute costs for ML research scale much less than quadratically with the size of the pretrained model, e.g. maybe (1) finetuning starts taking fewer data points as model size increases (sample efficiency improves with model capability), and so finetuning runs become a rounding error on compute, and (2) the vast majority of ML research progress involves nothing more expensive than finetuning runs. (Though in this world you have to wonder why we keep training bigger models instead of just investing solely in better finetuning the current biggest model.)
Another thing that occurred to me is that latency starts looking like another major bottleneck. Currently it seems feasible to make a paper’s worth of progress in ~6 months. With a 30x speedup, you now have to do that in 6 days. At that scale, introducing additional latency via experiments at small scales is a huge cost.
(I’m assuming here that the ideas and overall workflow are still managed by human researchers, since your hypothetical said that the AIs are just going from high level ideas to implemented experiments. If you have fully automated AI researchers then they don’t need to optimize latency as hard; they can instead get 30x speedup by having 30x as many researchers working but still producing a paper every 6 months.)
(Another possibility is that human ML researchers get really good at multi-tasking, and so e.g. they have 5 paper-equivalents at any given time, each of which takes 30 calendar days to complete. But I don’t believe that (most) human ML researchers are that good at multitasking on research ideas, and there isn’t that much time for them to learn.)
It also seems hard for the human researchers to have ideas good enough to turn into paper-equivalents every 6 days. Also hard for those researchers to keep on top of the literature well enough to be proposing stuff that actually makes progress rather than duplicating existing work they weren’t aware of, even given AI tools that help with understanding the literature.
Further, the current scaling laws imply huge inference availablity if huge amounts of compute are used for training.
Tbc the fact that running your automated ML implementers takes compute was a side point; I’d be making the same claims even if running the AIs was magically free.
Though even at a billion token-equivalents per second it seems plausible to me that your automated ML experiment implementers end up being a significant fraction of that compute. It depends quite significantly on how capable a single forward pass is, e.g. can the AI just generate an entire human-level pull request autoregressively (i.e. producing each token of the PR one at a time, without going back to fix errors) vs does it do similar things as humans (write tests and code, test, debug, eventually submit) vs. does it do way more iteration and error correction than humans (in parallel to avoid crazy high latency), do we use best-of-N sampling or similar tricks to improve quality of generations, etc.
Overall, this has updated me to some extent and it seems less plausible to me that ML research can achieve 30x speedups while having human researchers do all of the high level ideas. (I think the picture looks importantly better when AIs are removing this bottleneck.)
I think the most plausible case for your position would be that the compute costs for ML research scale much less than quadratically
The situation I was imagining is where most experiments use some combination of:
A relatively small amount of finetuning/inference on the biggest models (including for the actual AI researcher)
Larger (possibly full) training runs, but at much smaller scale (e.g. GPT-3 level performance models)
Then, we can in total afford ~training dataset sized amounts of finetuning/inference for the biggest models (by the inference availability argument). And GPT-3 performance experiments will be pretty cheap. So assuming our base model looks like GPT-6 with the expected compute requirement and model size, this is a huge amount of possible inference availability.
So, the implicit claim is that compute costs scale much less than quadratically. It’s certainly not obvious ML research can be progressed fast enough with this little compute.
Another possibility is that human ML researchers get really good at multi-tasking
I was certainly imagining at least some amount of multi-tasking (e.g. 4 projects at once each of which runs 8x faster). This doesn’t feel that crazy to me, I already do a moderate amount of multi-tasking.
Currently it seems feasible to make a paper’s worth of progress in ~6 months.
Note that this often involves multiple people working on the same paper. In the AI case, the division of labor might look at least somewhat different. (Though I don’t think this changes the picture very much from what you’re describing because most people now aren’t the “ideas” people.)
I was certainly imagining at least some amount of multi-tasking (e.g. 4 projects at once each of which runs 8x faster). This doesn’t feel that crazy to me, I already do a moderate amount of multi-tasking.
Multi-tasking where you are responsible for the entire design of the project? (Designing the algorithm, choosing an experimental setting and associated metrics, knowing the related work, interpreting the results of the experiments, figuring out what the next experiment should be, …)
Suppose today I gave you a device where you put in moderately detailed instructions for experiments, and the device returns the results[1] with N minutes of latency and infinite throughput. Do you think you can spend 1 working day using this device to produce the same output as 4 copies of yourself working in parallel for a week (and continue to do that for months, after you’ve exhausted low-hanging fruit)?
… Having written this hypothetical out, I am finding it more plausible than before, at least for small enough N, though it still feels quite hard at e.g. N = 60.
Assuming the box is pretty smart at understanding instructions (and has an understanding of my typical ontology to the extent that you would get after working with me a few weeks and reading various posts) and the box will ask follow-up questions in cases where the instructions are unclear. (And we can do small diffs with reduced latency like asking the results to be plotted in a different way.)
My main concern is running out of ideas after a while despite copies of myself with more thinking time having more time to generate ideas.
Sounds reasonable, though idk what you think realistic values of N are (my wild guess with hardly any thought is 15 minutes − 1 day).
EDIT: Tbc in the 1 day case I’m imagining that most of the time goes towards running the experiment—it’s more a claim about what experiments we want to run. If we just talk about the time to write the code and launch the experiment I’m thinking of N in the range of 5 minutes to 1 hour.
I agree it helps to run experiments at small scales first, but I’d be pretty surprised if that helped to the point of enabling a 30x speedup—that means that the AI labor allows you get 30x improvement in compute needed beyond what would be done by default by humans (though the 30x can include e.g. improving utilization, it’s not limited just to making individual experiments take less time).
I think the most plausible case for your position would be that the compute costs for ML research scale much less than quadratically with the size of the pretrained model, e.g. maybe (1) finetuning starts taking fewer data points as model size increases (sample efficiency improves with model capability), and so finetuning runs become a rounding error on compute, and (2) the vast majority of ML research progress involves nothing more expensive than finetuning runs. (Though in this world you have to wonder why we keep training bigger models instead of just investing solely in better finetuning the current biggest model.)
Another thing that occurred to me is that latency starts looking like another major bottleneck. Currently it seems feasible to make a paper’s worth of progress in ~6 months. With a 30x speedup, you now have to do that in 6 days. At that scale, introducing additional latency via experiments at small scales is a huge cost.
(I’m assuming here that the ideas and overall workflow are still managed by human researchers, since your hypothetical said that the AIs are just going from high level ideas to implemented experiments. If you have fully automated AI researchers then they don’t need to optimize latency as hard; they can instead get 30x speedup by having 30x as many researchers working but still producing a paper every 6 months.)
(Another possibility is that human ML researchers get really good at multi-tasking, and so e.g. they have 5 paper-equivalents at any given time, each of which takes 30 calendar days to complete. But I don’t believe that (most) human ML researchers are that good at multitasking on research ideas, and there isn’t that much time for them to learn.)
It also seems hard for the human researchers to have ideas good enough to turn into paper-equivalents every 6 days. Also hard for those researchers to keep on top of the literature well enough to be proposing stuff that actually makes progress rather than duplicating existing work they weren’t aware of, even given AI tools that help with understanding the literature.
Tbc the fact that running your automated ML implementers takes compute was a side point; I’d be making the same claims even if running the AIs was magically free.
Though even at a billion token-equivalents per second it seems plausible to me that your automated ML experiment implementers end up being a significant fraction of that compute. It depends quite significantly on how capable a single forward pass is, e.g. can the AI just generate an entire human-level pull request autoregressively (i.e. producing each token of the PR one at a time, without going back to fix errors) vs does it do similar things as humans (write tests and code, test, debug, eventually submit) vs. does it do way more iteration and error correction than humans (in parallel to avoid crazy high latency), do we use best-of-N sampling or similar tricks to improve quality of generations, etc.
Overall, this has updated me to some extent and it seems less plausible to me that ML research can achieve 30x speedups while having human researchers do all of the high level ideas. (I think the picture looks importantly better when AIs are removing this bottleneck.)
The situation I was imagining is where most experiments use some combination of:
A relatively small amount of finetuning/inference on the biggest models (including for the actual AI researcher)
Larger (possibly full) training runs, but at much smaller scale (e.g. GPT-3 level performance models)
Then, we can in total afford ~training dataset sized amounts of finetuning/inference for the biggest models (by the inference availability argument). And GPT-3 performance experiments will be pretty cheap. So assuming our base model looks like GPT-6 with the expected compute requirement and model size, this is a huge amount of possible inference availability.
So, the implicit claim is that compute costs scale much less than quadratically. It’s certainly not obvious ML research can be progressed fast enough with this little compute.
I was certainly imagining at least some amount of multi-tasking (e.g. 4 projects at once each of which runs 8x faster). This doesn’t feel that crazy to me, I already do a moderate amount of multi-tasking.
Note that this often involves multiple people working on the same paper. In the AI case, the division of labor might look at least somewhat different. (Though I don’t think this changes the picture very much from what you’re describing because most people now aren’t the “ideas” people.)
Cool, that all roughly makes sense to me :)
Multi-tasking where you are responsible for the entire design of the project? (Designing the algorithm, choosing an experimental setting and associated metrics, knowing the related work, interpreting the results of the experiments, figuring out what the next experiment should be, …)
Suppose today I gave you a device where you put in moderately detailed instructions for experiments, and the device returns the results[1] with N minutes of latency and infinite throughput. Do you think you can spend 1 working day using this device to produce the same output as 4 copies of yourself working in parallel for a week (and continue to do that for months, after you’ve exhausted low-hanging fruit)?
… Having written this hypothetical out, I am finding it more plausible than before, at least for small enough N, though it still feels quite hard at e.g. N = 60.
The experiments can’t use too much compute. No solving the halting problem.
Probably yes for realistic values of N?
Assuming the box is pretty smart at understanding instructions (and has an understanding of my typical ontology to the extent that you would get after working with me a few weeks and reading various posts) and the box will ask follow-up questions in cases where the instructions are unclear. (And we can do small diffs with reduced latency like asking the results to be plotted in a different way.)
My main concern is running out of ideas after a while despite copies of myself with more thinking time having more time to generate ideas.
Sounds reasonable, though idk what you think realistic values of N are (my wild guess with hardly any thought is 15 minutes − 1 day).
EDIT: Tbc in the 1 day case I’m imagining that most of the time goes towards running the experiment—it’s more a claim about what experiments we want to run. If we just talk about the time to write the code and launch the experiment I’m thinking of N in the range of 5 minutes to 1 hour.