In regards to AI, what about regulation of advanced GPU manufacturing, to try to stop the creation of increasingly powerful GPUs? The US currently has regulations against exporting powerful chips to China; maybe an international agreement (between US, EU, China, and Russia) would be possible to globally prohibit manufacture of chips that exceed certain specs? I think something like this would be eventually be necessary, because regulation of AI training would be impractical to enforce if bad actors can easily acquire and run the hardware needed for such AI training.
That only works if China, the EU, and other powers large enough to build their own chips agree with this.
Otherwise what you do with these restrictions create an incredible financial opportunity. Nvidia is up to 40/billion a year just for AI chips, with a huge profit margin over 50 percent. Presumably once AI chips can inference ai models that can do useful work—like run a robot or work independently as a gpt well enough to pay for—that number will 10x or 100x. At 4 trillion a year just for AI chips that would be larger than opec, which seems to be a paltry 880 billion per year.
This is what bothers me about the future prospect of any restrictions at all. How much lobbying money does Nvidia have to let the chips flow? What about when it’s many other companies including state owned Chinese firms, where the interests of the company and the interests of the Chinese government are the same?
At a certain level of scale, AI silicon companies will have a larger lobbying budget than oil companies, their profitability is high so this can happen at smaller levels of scale. And their incentives are to sell as much silicon as possible.
Though this has some alignment incentives: safer designs using isolated dedicated silicon for isolated AI models (makes escapes and model collusion difficult). and ternary redundancy for robotics (protects from hardware and some forms of software fault) means more silicon sales for the same amount of useful work from the model.
In regards to AI, what about regulation of advanced GPU manufacturing, to try to stop the creation of increasingly powerful GPUs? The US currently has regulations against exporting powerful chips to China; maybe an international agreement (between US, EU, China, and Russia) would be possible to globally prohibit manufacture of chips that exceed certain specs? I think something like this would be eventually be necessary, because regulation of AI training would be impractical to enforce if bad actors can easily acquire and run the hardware needed for such AI training.
That only works if China, the EU, and other powers large enough to build their own chips agree with this. Otherwise what you do with these restrictions create an incredible financial opportunity. Nvidia is up to 40/billion a year just for AI chips, with a huge profit margin over 50 percent. Presumably once AI chips can inference ai models that can do useful work—like run a robot or work independently as a gpt well enough to pay for—that number will 10x or 100x. At 4 trillion a year just for AI chips that would be larger than opec, which seems to be a paltry 880 billion per year.
This is what bothers me about the future prospect of any restrictions at all. How much lobbying money does Nvidia have to let the chips flow? What about when it’s many other companies including state owned Chinese firms, where the interests of the company and the interests of the Chinese government are the same?
At a certain level of scale, AI silicon companies will have a larger lobbying budget than oil companies, their profitability is high so this can happen at smaller levels of scale. And their incentives are to sell as much silicon as possible.
Though this has some alignment incentives: safer designs using isolated dedicated silicon for isolated AI models (makes escapes and model collusion difficult). and ternary redundancy for robotics (protects from hardware and some forms of software fault) means more silicon sales for the same amount of useful work from the model.