Should we be worrying about “reverse Huggingface”, wherein a frontier AI company (or anyone) does something really really beneficial like curing a disease using AI, and swings public opinion / political winds away from a pause? Is there anything to do to prepare for that?
Curing a disease likely won’t work—even if there’s a cure, clinical trials take a very long time. Now, maybe inventing something like SR-17018 (a chemical which can treat addiction to opioids, and had communities spring up online dedicated to synthesizing and using it) would, but this would be bound to be controversial, and lots of people would be pointing out why this is actually a bad thing, etc.
Releasing science results in general also runs the risk of angering the scientific community, much like mathematics has angered the math community. Unsure what the popular perception of the math advances is, though.
A breakthrough in fundamental theoretical physics would likely be treated similarly to the math breakthrough, while releasing something like a room-temperature superconductor could be beneficial, though I think it’ll take a long time for this tech to percolate through enough to improve the average Joe’s life, similar to drugs.
Other tech breakthroughs—an amazing video game, an incredibly effective matchmaking service, something to manage the boring parts of people’s lives might do this, but also a lot of people really don’t like new technology in general.
I’d be skeptical that good things from AI will be sudden enough to make the news cycle—it feels like most good things take time, whereas destroying/hacking something can happen pretty quickly. Maybe there are things I haven’t considered yet.
AI-powered medical research could rapidly show clinical results by deploying investigational drugs under expanded access/compassionate use/right to try. No clinical trial necessary. If AI-developed investigational drugs started showing clear, fast results on the sickest patients, I think that would change minds very quickly. But if it makes patients sicker, then that will have the opposite effect.
It’s possible that the first AI-developed medications will be from somebody who uses AI to identify a new drug to treat their own severe illness.
tl;dr: Pause/pace advocates should take pains to be clear they’re also in favor of medical cures/curing aging, allow for work in that direction in their proposals for pause/pace scenarios, and clearly explain that rushing AI progress may kill you rather than cure you. (or both, but the sum of “cure” and “kill” is still “dead”)
By far the most important reason people hate big pharma is its reputation for its efforts to keep prices higher for longer on existing drug classes. If AI starts producing a slew of genuinely innovative new drugs and doesn’t charge an arm and a leg for them, I think people will view that in a much more positive light.
My basic take is that people are risk-averse and change-averse. ‘We will change absolutely everything- but in a good way!’ is just a much less compelling pitch than ‘this is basically normal and might make life more convenient/ make stocks go up.’
IDK about “worried”, but I do think a lot of folks around here aren’t orienting to the (IMO likely) possibility that everything with AI goes pretty well in the short term, with the benefits being unevenly distributed but relatively broad-based.
Right now, most of the economic impacts from AI are still in capital markets and capex / buildout; the consumer surplus and effects on the larger economy from inference are still pretty small, relatively speaking. So AI sentiment among investors, construction workers, municipal governments etc. who are already directly impacted positively by the capex and investment are more positive on AI generally, and I expect that this will continue as we get more stories like this:
I expect that this dynamic will pretty quickly make any policy work that is not laser-focused on the “sci-fi” risks of actual superintelligence an uphill, losing battle.
The main counterforce IMO is that more and more severe warning shots are quite plausible in worlds where we are doomed, because models are getting destructive faster than they are good at planning (modulo full-blown neuralese/strong continual learning, not mere looping of models.)
(The main caveat is that centaurs/human + AI teams are likely to outperform AIs alone, and thus pure AI warning shots will probably diminish relatively speaking, and the secondary caveat is that some of the good properties are almost certainly lost in future events because of regression to the mean effects, but it’s very plausible there’s a warning shot that’s good enough to make the USG consider an outright pause.)
I suspect this is largely due to models becoming relatively good at SWE/Cyberhacking way faster than with other skills, with the exception of easily-parallelizable mathematics.
Should we be worrying about “reverse Huggingface”, wherein a frontier AI company (or anyone) does something really really beneficial like curing a disease using AI, and swings public opinion / political winds away from a pause? Is there anything to do to prepare for that?
Curing a disease likely won’t work—even if there’s a cure, clinical trials take a very long time. Now, maybe inventing something like SR-17018 (a chemical which can treat addiction to opioids, and had communities spring up online dedicated to synthesizing and using it) would, but this would be bound to be controversial, and lots of people would be pointing out why this is actually a bad thing, etc.
Releasing science results in general also runs the risk of angering the scientific community, much like mathematics has angered the math community. Unsure what the popular perception of the math advances is, though.
A breakthrough in fundamental theoretical physics would likely be treated similarly to the math breakthrough, while releasing something like a room-temperature superconductor could be beneficial, though I think it’ll take a long time for this tech to percolate through enough to improve the average Joe’s life, similar to drugs.
Other tech breakthroughs—an amazing video game, an incredibly effective matchmaking service, something to manage the boring parts of people’s lives might do this, but also a lot of people really don’t like new technology in general.
I’d be skeptical that good things from AI will be sudden enough to make the news cycle—it feels like most good things take time, whereas destroying/hacking something can happen pretty quickly. Maybe there are things I haven’t considered yet.
AI-powered medical research could rapidly show clinical results by deploying investigational drugs under expanded access/compassionate use/right to try. No clinical trial necessary. If AI-developed investigational drugs started showing clear, fast results on the sickest patients, I think that would change minds very quickly. But if it makes patients sicker, then that will have the opposite effect.
It’s possible that the first AI-developed medications will be from somebody who uses AI to identify a new drug to treat their own severe illness.
See Wagner’s “We’re not ready for the e/Acc × Longevity preference cascade” for a discussion of that idea re: medicine specifically.
tl;dr: Pause/pace advocates should take pains to be clear they’re also in favor of medical cures/curing aging, allow for work in that direction in their proposals for pause/pace scenarios, and clearly explain that rushing AI progress may kill you rather than cure you. (or both, but the sum of “cure” and “kill” is still “dead”)
Big Pharma bakes miracles like pies and it’s one of the most hated industries in US. I would worry about nascent superpersuasion/propaganda instead.
By far the most important reason people hate big pharma is its reputation for its efforts to keep prices higher for longer on existing drug classes. If AI starts producing a slew of genuinely innovative new drugs and doesn’t charge an arm and a leg for them, I think people will view that in a much more positive light.
Awesome science progress + Alignment faking = We will most likely look like conspiracy theorists right up to the treacherous turn.
The defense is probably education about this possibility now, but I would most likely bet on that failing as a defense.
My basic take is that people are risk-averse and change-averse. ‘We will change absolutely everything- but in a good way!’ is just a much less compelling pitch than ‘this is basically normal and might make life more convenient/ make stocks go up.’
IDK about “worried”, but I do think a lot of folks around here aren’t orienting to the (IMO likely) possibility that everything with AI goes pretty well in the short term, with the benefits being unevenly distributed but relatively broad-based.
Right now, most of the economic impacts from AI are still in capital markets and capex / buildout; the consumer surplus and effects on the larger economy from inference are still pretty small, relatively speaking. So AI sentiment among investors, construction workers, municipal governments etc. who are already directly impacted positively by the capex and investment are more positive on AI generally, and I expect that this will continue as we get more stories like this:
https://www.wsj.com/us-news/education/the-teachers-getting-50-000-bonuses-thanks-to-a-massive-meta-data-center-b4631d05
I expect that this dynamic will pretty quickly make any policy work that is not laser-focused on the “sci-fi” risks of actual superintelligence an uphill, losing battle.
The main counterforce IMO is that more and more severe warning shots are quite plausible in worlds where we are doomed, because models are getting destructive faster than they are good at planning (modulo full-blown neuralese/strong continual learning, not mere looping of models.)
(The main caveat is that centaurs/human + AI teams are likely to outperform AIs alone, and thus pure AI warning shots will probably diminish relatively speaking, and the secondary caveat is that some of the good properties are almost certainly lost in future events because of regression to the mean effects, but it’s very plausible there’s a warning shot that’s good enough to make the USG consider an outright pause.)
I suspect this is largely due to models becoming relatively good at SWE/Cyberhacking way faster than with other skills, with the exception of easily-parallelizable mathematics.
I got this point from @davekasten.