Note that this probably doesn’t change the story much for GPU restrictions, though. For purposes of software improvements, one needs compute for lots of relatively small runs rather than one relatively big run, and lots of relatively small runs is exactly what GPU restrictions (as typically envisioned) would not block.
Couldn’t GPU restrictions still make them more expensive? Like let’s say tomorrow that we impose a tax on all new hardware that can be used to train neural networks such that any improvements in performance will be cancelled out by additional taxes. Wouldn’t that also slow down or even stop the growth of smaller training runs?
That would, and in general restrictions aimed at increasing price/reducing supply could work, though that doesn’t describe most GPU restriction proposals I’ve heard.
Note that this probably doesn’t change the story much for GPU restrictions, though. For purposes of software improvements, one needs compute for lots of relatively small runs rather than one relatively big run, and lots of relatively small runs is exactly what GPU restrictions (as typically envisioned) would not block.
Couldn’t GPU restrictions still make them more expensive? Like let’s say tomorrow that we impose a tax on all new hardware that can be used to train neural networks such that any improvements in performance will be cancelled out by additional taxes. Wouldn’t that also slow down or even stop the growth of smaller training runs?
That would, and in general restrictions aimed at increasing price/reducing supply could work, though that doesn’t describe most GPU restriction proposals I’ve heard.