In my humble opinion, the AI 2040 document falls short of being such a plan because it does nothing to stop or slow research that might find (and I’m tempted to say, “will probably find”) a fundamentally more efficient learning algorithm than what the leading labs are currently using.
This is overstated. Plan A involves significant attempts to slow down algorithmic progress. See e.g.:
In the past, companies have trained bigger and better AIs using both compute scaling (bigger training runs) and software progress (advances in AI algorithms—new paradigms, better training recipes, better data, etc.). Now, the Consortium tries to steer things so that the majority of improvement comes from increasing training compute.
Algorithms are information; it’s inherently difficult to stop them from proliferating, and the total research transparency means we aren’t even trying. [FN: There are a few nuances here. Some algorithmic progress is easily communicated (e.g., new architectures, better optimization algorithms), while other types cannot easily diffuse (e.g., huge libraries of RL environments, hardware-software codesign, scale or compute dependent algorithms). Regulations that the US and China agree to should steer companies towards making the latter type of algorithm progress when possible.] Once a new paradigm is discovered, it’ll go straight to the hypothetical covert projects and there’s no way to undo that.
and:
So the goal is to develop AI at a reasonable pace: Not too fast, not too slow. Ban the unsafe kinds, allow the safe kinds. (...) The Consortium countries come up with a simple high-level framework: we’ll agree to let each other see all the AI research. Then, if we don’t like something someone is doing, we’ll talk about it and perhaps agree to ban it. (...)
Transparency offers many benefits. (...) Third, there’s no longer as much incentive to race to discover new AI paradigms and more powerful algorithms, because companies wouldn’t be able to hoard such discoveries and profit greatly from them. This buys the world time.
Plan A involves significant attempts to slow down algorithmic progress
Plan A does not advocate for what I consider the most potent brake on that “progress”, namely restrictions enforced with penalties on dissemination of new knowledge from one researcher to another (and related measures like bans on founding a new AI lab or seeking additional investment for an existing lab).
there’s no longer as much incentive to race to discover new AI paradigms and more powerful algorithms, because companies wouldn’t be able to hoard such discoveries and profit greatly from them.
Most of the advances in AI so far have not been the result of any hoarding, but rather of researchers and labs freely giving away knowledge. The most salient example is Google Deep Mind’s freely giving away the knowledge described in the “Attention is all you need” paper that started the transformer revolution on which the success of OpenAI and Anthropic depended, and I got the impression from my brief study of the history of the field that every breakthrough in machine learning on which the transformer breakthrough depended was also freely given away shortly after its discovery. Although I know little about the history of AI research, I know enough about the history of research and development in general to say with confidence that at least over the last 200 years, the vast majority of technological and scientific progress over all fields was caused by these acts of freely giving away insights. I’d be very surprised to learn that AI is an exception to that generality: the vast amount of potential revenue and profit to be made with AI makes giving-away less likely, but the idealism and (false) sense of an altruistic purpose makes giving-away more likely.
To slow down the rate of advance of a field, we would prevent these acts of giving-away from happening: if we could somehow prevent all insights discovered inside OpenAI from disseminating beyond OpenAI, that would IMHO be an improvement (though shutting OpenAI down and requiring all its employees to find something else to do other than AI research would be strictly better)!
This is overstated. Plan A involves significant attempts to slow down algorithmic progress. See e.g.:
and:
Plan A does not advocate for what I consider the most potent brake on that “progress”, namely restrictions enforced with penalties on dissemination of new knowledge from one researcher to another (and related measures like bans on founding a new AI lab or seeking additional investment for an existing lab).
Most of the advances in AI so far have not been the result of any hoarding, but rather of researchers and labs freely giving away knowledge. The most salient example is Google Deep Mind’s freely giving away the knowledge described in the “Attention is all you need” paper that started the transformer revolution on which the success of OpenAI and Anthropic depended, and I got the impression from my brief study of the history of the field that every breakthrough in machine learning on which the transformer breakthrough depended was also freely given away shortly after its discovery. Although I know little about the history of AI research, I know enough about the history of research and development in general to say with confidence that at least over the last 200 years, the vast majority of technological and scientific progress over all fields was caused by these acts of freely giving away insights. I’d be very surprised to learn that AI is an exception to that generality: the vast amount of potential revenue and profit to be made with AI makes giving-away less likely, but the idealism and (false) sense of an altruistic purpose makes giving-away more likely.
To slow down the rate of advance of a field, we would prevent these acts of giving-away from happening: if we could somehow prevent all insights discovered inside OpenAI from disseminating beyond OpenAI, that would IMHO be an improvement (though shutting OpenAI down and requiring all its employees to find something else to do other than AI research would be strictly better)!