IDK about “worried”, but I do think a lot of folks around here aren’t orienting to the (IMO likely) possibility that everything with AI goes pretty well in the short term, with the benefits being unevenly distributed but relatively broad-based.
Right now, most of the economic impacts from AI are still in capital markets and capex / buildout; the consumer surplus and effects on the larger economy from inference are still pretty small, relatively speaking. So AI sentiment among investors, construction workers, municipal governments etc. who are already directly impacted positively by the capex and investment are more positive on AI generally, and I expect that this will continue as we get more stories like this:
I expect that this dynamic will pretty quickly make any policy work that is not laser-focused on the “sci-fi” risks of actual superintelligence an uphill, losing battle.
The main counterforce IMO is that more and more severe warning shots are quite plausible in worlds where we are doomed, because models are getting destructive faster than they are good at planning (modulo full-blown neuralese/strong continual learning, not mere looping of models.)
(The main caveat is that centaurs/human + AI teams are likely to outperform AIs alone, and thus pure AI warning shots will probably diminish relatively speaking, and the secondary caveat is that some of the good properties are almost certainly lost in future events because of regression to the mean effects, but it’s very plausible there’s a warning shot that’s good enough to make the USG consider an outright pause.)
I suspect this is largely due to models becoming relatively good at SWE/Cyberhacking way faster than with other skills, with the exception of easily-parallelizable mathematics.
IDK about “worried”, but I do think a lot of folks around here aren’t orienting to the (IMO likely) possibility that everything with AI goes pretty well in the short term, with the benefits being unevenly distributed but relatively broad-based.
Right now, most of the economic impacts from AI are still in capital markets and capex / buildout; the consumer surplus and effects on the larger economy from inference are still pretty small, relatively speaking. So AI sentiment among investors, construction workers, municipal governments etc. who are already directly impacted positively by the capex and investment are more positive on AI generally, and I expect that this will continue as we get more stories like this:
https://www.wsj.com/us-news/education/the-teachers-getting-50-000-bonuses-thanks-to-a-massive-meta-data-center-b4631d05
I expect that this dynamic will pretty quickly make any policy work that is not laser-focused on the “sci-fi” risks of actual superintelligence an uphill, losing battle.
The main counterforce IMO is that more and more severe warning shots are quite plausible in worlds where we are doomed, because models are getting destructive faster than they are good at planning (modulo full-blown neuralese/strong continual learning, not mere looping of models.)
(The main caveat is that centaurs/human + AI teams are likely to outperform AIs alone, and thus pure AI warning shots will probably diminish relatively speaking, and the secondary caveat is that some of the good properties are almost certainly lost in future events because of regression to the mean effects, but it’s very plausible there’s a warning shot that’s good enough to make the USG consider an outright pause.)
I suspect this is largely due to models becoming relatively good at SWE/Cyberhacking way faster than with other skills, with the exception of easily-parallelizable mathematics.
I got this point from @davekasten.