I’ve said this elsewhere, but I think it bears repeating. I’ve done my own in-depth research into the state of the field of machine learning and potential novel algorithmic advances which have not yet been incorporated into frontier models, and in-depth research into the state of neuroscience’s understanding of the brain. I have written a report detailing the ways in which I think Joe Carlsmith’s and Ajeya Cotra’s estimates are overestimating the AGI-relevant compute of the human brain by somewhere between 10x to 100x.
Furthermore, I think that there are compelling arguments for why the compute in frontier algorithms is not being deployed as efficiently as it could be, resulting in higher training costs and data requirements than is theoretically possible.
In combination, these findings lead me to believe we are primarily algorithm-constrained not hardware or data constrained. Which, in turn, means that once frontier models have progressed to the point of being able to automate research for improved algorithms I expect that substantial progress will follow. This progress will, if I am correct, be untethered to further increases in compute hardware or training data.
My best guess is that a frontier model of the approximate expected capability of GPT-5 or GPT-6 (equivalently Claude 4 or 5, or similar advances in Gemini) will be sufficient for the automation of algorithmic exploration to an extent that the necessary algorithmic breakthroughs will be made. I don’t expect the search process to take more than a year. So I think we should expect a time of algorithmic discovery in the next 2 − 3 years which leads to a strong increase in AGI capabilities even holding compute and data constant.
I feel very uncertain what the full implications of that will be, or how fast things will proceed after that point. I do think it would be reasonable, if this situation does come to pass, to approach such novel unprecedentedly powerful AI systems with great caution.
I agree with the general shape of your argument, including that Cotra and Carlsmith are likely to overestimate the compute of the human brain, and that frontier algorithms are not as efficient as algorithms could be.
My best guess is that a frontier model of the approximate expected capability of GPT-5 or GPT-6 (equivalently Claude 4 or 5, or similar advances in Gemini) will be sufficient for the automation of algorithmic exploration to an extent that the necessary algorithmic breakthroughs will be made.
But I disagree that it will happen this quickly. :)
I’ve said this elsewhere, but I think it bears repeating. I’ve done my own in-depth research into the state of the field of machine learning and potential novel algorithmic advances which have not yet been incorporated into frontier models, and in-depth research into the state of neuroscience’s understanding of the brain. I have written a report detailing the ways in which I think Joe Carlsmith’s and Ajeya Cotra’s estimates are overestimating the AGI-relevant compute of the human brain by somewhere between 10x to 100x.
Furthermore, I think that there are compelling arguments for why the compute in frontier algorithms is not being deployed as efficiently as it could be, resulting in higher training costs and data requirements than is theoretically possible.
In combination, these findings lead me to believe we are primarily algorithm-constrained not hardware or data constrained. Which, in turn, means that once frontier models have progressed to the point of being able to automate research for improved algorithms I expect that substantial progress will follow. This progress will, if I am correct, be untethered to further increases in compute hardware or training data.
My best guess is that a frontier model of the approximate expected capability of GPT-5 or GPT-6 (equivalently Claude 4 or 5, or similar advances in Gemini) will be sufficient for the automation of algorithmic exploration to an extent that the necessary algorithmic breakthroughs will be made. I don’t expect the search process to take more than a year. So I think we should expect a time of algorithmic discovery in the next 2 − 3 years which leads to a strong increase in AGI capabilities even holding compute and data constant.
I feel very uncertain what the full implications of that will be, or how fast things will proceed after that point. I do think it would be reasonable, if this situation does come to pass, to approach such novel unprecedentedly powerful AI systems with great caution.
I agree with the general shape of your argument, including that Cotra and Carlsmith are likely to overestimate the compute of the human brain, and that frontier algorithms are not as efficient as algorithms could be.
But I disagree that it will happen this quickly. :)