Senator Bernie Sanders is planning to introduce legislation that would ban the construction of new AI data centers. You can find his video announcement here, and here is the transcript:
Thanks very much for joining me. I will soon be introducing legislation calling for a moratorium on the construction of new data centers.
Now, as a result, I’ve been called a luddite, anti-innovation, anti-progress, pro-Chinese, among many other things. So why am I doing that? Why am I calling for a moratorium on the construction of new data centers?
Bottom line: We are at the beginning of the most profound technological revolution in world history. That’s the truth. This is a revolution which will bring unimaginable changes to our world. This is a revolution which will impact our economy with massive job displacement. It will threaten our democratic institutions. It will impact our emotional well-being, and what it even means to be a human being. It will impact how we educate and raise our kids. It will impact the nature of warfare, something we are seeing right now in Iran.
Further, and frighteningly, some very knowledgeable people fear that that what was once seen as science fiction could soon become a reality—and that is that superintelligent AI could become smarter than human beings, could become independent of human control, and pose an existential threat to the entire human race. In other words, human beings could actually lose control over the planet.
And in the midst of all of that, all of this transformative change, what I have to tell you is that the United States Congress hasn’t a clue, not a clue, as to how to respond to these revolutionary technologies and protect the American people. And it’s not only not having a clue, they’re out busy raising money all day long from AI and their super PACs, which is a whole other problem.
As many of you know, the AI revolution is being pushed by the wealthiest people in our country, including Elon Musk, Jeff Bezos, Larry Ellison, Mark Zuckerberg, Peter Thiel, and others. All of these people are multi-billionaires who, if they are successful at AI, will become even richer and more powerful than they are today.
What I want to do now is not tell you my fears regarding AI and robotics. I want you to actually hear from them, the billionaires who are pushing these technologies. Listen carefully to what they are saying.
Elon Musk, wealthiest person alive, stated that quote, “AI and robots will replace all jobs.” All jobs. “Working will be optional.” End of quote.
Dario Amodai, the CEO of Anthropic, predicted that quote, “AI could displace half of all entry-level white collar jobs in the next 1 to 5 years.” And that quote, “Humanity is about to be handed almost unimaginable power, and it is deeply unclear whether our social, political, and technological systems possess the maturity to wield it.” End quote. That’s Amodai.
According to Demis Hassabis, the head of Google’s DeepMind—this is Google’s DeepMind—the AI revolution will be 10 times bigger than the industrial revolution, and 10 times faster. All right, you got that? That means it will have a 100 times greater impact on society than the industrial revolution had.
Jeff Bezos, the fourth richest person in the world, has been pushing his staff for years to think big and envision what it would take for Amazon, which he owns, to fully automate its operations and replace at least 600,000 warehouse workers with robots. 600,000 jobs gone. Robots doing the work.
Bill Gates, also one of the wealthiest people on Earth, predicted that humans, quote, “won’t be needed for most things,” end quote, such as manufacturing products, delivering packages, or growing food over the next decade, due to artificial intelligence.
Mustafa Suleyman, the CEO of Microsoft AI, said most white-collar work quote, “will be fully automated by an AI within the next 12 to 18 months” end quote.
Jim Farley, the CEO of Ford, predicted that AI will eliminate quote, “nearly half, literally half, of all white-collar jobs in the US” end quote, within the next decade.
I want you to hear this one. Larry Ellison—also one of the richest people on Earth, and a major investor in AI—said that there will be an artificial intelligence-powered surveillance state where, quote, “citizens will be on their best behavior because we’re constantly recording and reporting everything that is going on.” End quote.
Dr. Jeffrey Hinton, considered to be the “godfather of AI,” believes there is a quote “10% to 20% chance for AI to wipe us out.” End quote.
Mark Zuckerberg, the fifth richest person in the world, is building a data center in the state of Louisiana—a data center that is the size of Manhattan, and will use three times the quantity of electricity that the entire city of New Orleans uses every year.
All right. Now, for many years now, leading experts have called for regulation and reasonable pauses to the development of artificial intelligence, to ensure the safety—the very safety—of humanity. Let’s go back to our good friend Elon Musk. He said back in 2018, quote—this is Elon Musk—“Mark my words, AI is far more dangerous than nukes. So why do we have no regulatory oversight? This is insane.” End quote, Elon Musk.
In March of 2023, over 1,000 business leaders in the big tech industry, prominent scientists, AI researchers, and academics co-signed an open letter entitled, quote, “Pause Giant AI Experiments” end quote, stating,
“We must ask ourselves: should we let machines flood our information channels with propaganda and untruth? Should we automate away all the jobs, including the fulfilling ones? Should we develop nonhuman minds that might eventually outnumber, outsmart, obsolete, and replace us? Should we risk control—loss of control—of our civilization?”
“Should we risk loss of control of our civilization? Such decisions must not be delegated to unelected tech leaders. Therefore, we call on all AI labs to immediately pause for at least 6 months the training of AI systems more more powerful than GPT4; this pause should be public and verifiable and include all key actors. If such a pause cannot be enacted quickly, governments should step in and institute a moratorium.” End of quote.
That is what some of the leaders in the AI industry have said. And clearly where we are right now, is that there has not been any pause. There has been massive amounts of competition between one company and the other, between the United States and China. So: bottom line is that, in my view, to protect our workers from losing their jobs, to protect human beings from attacks on their mental health, to protect our kids, to protect the safety of human life: yeah, we need a moratorium on data centers. We need to take a deep breath. We need to make sure that AI and robotics work for all of us, not just a handful of billionaires. Thanks very much.
It’s the kind of action that when universalized does indeed end the AGI death race! That is in an important sense proposing an end to the AGI death race.
If everyone stopped building datacenters you really have made a lot of progress towards stopping the death race (and of course, the algorithm that produces banning datacenter construction would probably not stop there).
I think this is a common misconception. I’m pretty sure algorithmic progress will eventually reach a point where what currently takes a datacenter will be possible on a single machine for a slightly longer training period. If that same algorithm runs on a datacenter it would produce something superhuman, but cutting down to only single-gpu training would then not be enough to completely stop. Algo progress is a slow slog of “grad student descent”, so it likely takes quite a bit longer, and maybe it takes enough longer to figure out alignment. But it doesn’t stop the death race, it just slows it down. Actual stopping would require shredding all silicon big enough to even run the fully trained AI, which doesn’t seem to be in the cards. I’m not saying datacenter construction is good or should continue, or that this won’t buy time, but I think people are wishful-thinking about how much time it buys.
Agree qualitatively (and possible quantitatively). However, there’s a quite large knock-on effect, which is a strong bundle of signals of “AGI is bad, don’t make AGI”. These signals move in various directions between different entities, carrying various messages, but they generally push against AGI. (E.g. signaling legitimacy of the Stop position; the US signaling to other states; society signaling to would-be capabilities researchers; Congress self-signaling “we’re trying to ban this whole thing and will continue to add patches to ban dangerous stuff”; etc.)
I mean, sure, eventually. The key question is how much of algorithmic progress is downstream of hardware scaling. My sense is around 50% of it, maybe a bit more, so that if you cut scaling, progress now happens at around 1/4th of the speed, which is of course huge and makes things a lot better.
Thinking this through step by step in the framework of the AI Futures Model:
First, I’ll check what the model says, then I’ll reconstruct the reasoning behind why it predicts that.
By default, with Daniel’s parameters, Automated Coder (AC) happens in 2030 and ASI happens in 2031 1.33 years later.
If I stop experiment and training compute growth at the start of 2027, then the model predictsAutomated Coder in 2039 rather than 2030. So 4x slower in calendar time (exactly matching habyrka’s guess). It also looks to have well over a 5 year takeoff from AC to ASI as opposed to the default of 1.33 years.
However, this is highly sensitive to the timing of the compute growth pause, because it’s a shock to the flow rather than the stock. e.g. if I instead stop growth at the start of 2029 as in this worksheet, then AC happens in Mar 2031, taking ~2.2 years instead of ~1.2, so slowing things down by <2x. It does still slow down takeoff from AC to ASI to 4 years, so by ~3x (and this is probably at least a slight underestimate because we don’t model hardware R&D automation).
Now I’ll reconstruct why this is the case using simplifications to the model (I actually did these calcluations before plugging the time series things into our model).
Currently, experiment compute is growing at around 3x/year, and human labor around 2x/year. Conditional on no AGI, we’re projecting experiment compute growth to slow to around 2x/year by 2030, and human labor growth to slow to around 1.5x/year.
Figuring out the effect of removing experiment growth is a bit complicated for various reasons.
On the margin, informed by interviews/surveys, we model a ~2.5x gain in “research effort” from 10x more experiment compute. Which if applied instantaneously, would mean a 2.5x slowdown in algorithmic progress.
We estimate that a 100x in human parallel coding labor gives a ~2x in research effort on the margin. We don’t model quantity of research labor, but I expect probably the gains would be relatively small as quality matters a lot more than quantity; let’s shade up to 3x.
So naively based on our model parameters and a simplified version of our model (Cobb-Douglas used to locally approximate a CES), by default the growth in research effort per year from experiment compute is 2.5^log(2-3)=~1.3-1.55x, and from human labor is sqrt(3)^log(1.5-2)=~1.1-1.2x. Meaning that roughly log(1.4)/(log(1.4)+log(1.15))=~70% of research effort growth is coming from experiment compute.
So to simplify from now on, let’s think about what happens if research effort growth has a one-time shock a constant 30% of what it otherwise would have been.
What does this actually mean in terms of the effect on algorithmic/software progess? This means that the shock in research effort growth will eventually cause the software growth rate to be 30% of what it would have been otherwise, but it steadily decreases toward 30% over time (intuitively, the immediate effect is 0 because you have to actually wait for the “missing new experiment compute” to take effect).
The above graph seems to give decent intuition for why the averaged-over-the-relevant-period slowdown in software progress from no more experiment compute might be about 2x (with the 2027.0 growth stoppage), and therefore the overall slowdown from no more compute might be about 4x. It looks like the average of the first 12 years might be close to 0.5.
All of the above is ignoring automation for simplicity. Taking into account automation would mean that more of the gains from labor, so the slowdown is smaller. But on the other hand, as you get more coding labor you get more bottlenecked on experiment compute (which the Cobb-Douglas doesn’t take into account); in the AIFM, you’d eventually get hard bottlenecked, you can have a maximum of 15x research effort gain from coding labor increases alone. Looks like these factors and other deviations from the model might roughly cancel out in the case we’re considering.
But it doesn’t stop the death race, it just slows it down.
My current default assumption is that, yes, someone will build something that obsoletes human intellectual and physical labor. In those futures, the best-case scenario is “Maybe the AIs like keeping humans as pets (and won’t breed us the way we breed pugs).”[1] And the other alternative futures go sharply downhill from there.
So I think of delay in terms of survivor curves, like someone with terminal cancer. How much time can we buy the human race? Can the children alive today get to enjoy a decent lifetime for however long we all have? So I heavily favor delay, in much the same way that I’d favor cancer remission.
A global AI halt might even buy us quite a bit of time.
If I had to bet on a specific model as liking humans and being a responsible “pet owner”, then I currently suspect we might have the best odds with a descendant of Claude. I do actually think that “enculturation” and building morally thoughtful models that like humans gives us non-zero chance of a more acceptable outcome. But I would still prefer humans to control their own destiny.
“Eventually”, sure, but I don’t think that’s operative here. If we had the ASI recipe and could study it safely for ten years, we’d find a way to implement it in a single datacenter. But discovering it in a single data center is much harder. There is actually something missing from current LLMs, there’s a part of intelligence they just don’t have, and the only thing that seems to mitigate that issue is model size, so without ever-increasing model size and analysis of their training dynamics, I think any attempts to get the missing piece are throwing darts with the lights off. (To be fair I have pretty unusual timelines compared to most of LW so maybe what’s convincing to me shouldn’t be to you.)
I agree with the general concern, but it’d be clearly a move in the right direction on that front?
With this kind of proposal I’m more worried that it could lead to a unilateral slowdown just after having animated China to be much more aggressive on AI.
I agree with the general concern, but it’d be clearly a move in the right direction on that front?
agreed, I’m not saying don’t do it. I’m replying to habryka saying it proposes an end to the race.
With this kind of proposal I’m more worried that it could lead to a unilateral slowdown just after having animated China to be much more aggressive on AI.
Doesn’t the USA have enough datacenters to stay ahead of china in the race to fully human-replacing AI for quite a while, even with a unilateral hardware pause? I’m not sure of this claim, but it’s currently my impression that the compute ratio is pretty dramatic.
I expect there’s already enough silicon in place to produce overwhelmingly superhuman AI that still has idiot-savant going on; completely stopping CPU and GPU production worldwide seems to me like it’d be a timeline-extending move, but not by more than a few years. Somewhere between 1.3x and 3x.
Doesn’t the USA have enough datacenters to stay ahead of china in the race to fully human-replacing AI for quite a while, even with a unilateral hardware pause?
As of June 2025 the US has 5x as much compute as China, I’d expect the gap has grown with substantially more American than Chinese data centers coming online in the past ~9 months
I dont see how. If AGI/ASI is powerful, then the existing deployed compute will suddenly become more powerful when AGI happens. If that isn’t X risk, then a few more data centers won’t change things. This only matter in long timelines where more data centers are required to get AGI. I don’t think that is the case. I think copying the mammal neocortex etc will get us there and that doesn’t need more compute.
This is cool! I’m sad he spends so much of his time criticising the good part (AI doing tonnes of productive labour). I say this not because I want to demand every ally agree with me on every point, but because I want to early disavow beliefs that political expediency might want me to endorse.
It seems to me a meaningfully open question whether automating all human labor will end up net benefiting humans, even assuming we survive; of course it might, but I think much more dystopian outcomes also seem plausible. Markets tend to benefit humans because the price signals we send tend to correlate with our relative needs, and hence with our welfare; I think it is not obvious that this correlation will persist once humans become unable to generate economic value.
While I’m glad that these things are starting to be seriously discussed in the open by those in positions of power, this sounds like a blanket ban on all new datacenters? This seems too broad to me. The ideal moratorium would be on large GPU clusters, or large AI specific infrastructure, right?
On another note, I’m somewhat concerned that banning large compute infrastructure will concentrate pressure on algorithmic progress if there is no “low effort”[1] outlet of just scaling the compute. Maybe regulating the rate at which large scale compute infrastructure is constructed will still allow researchers to be “intellectually lazy”[2] by offloading progress onto compute scaling instead of innovating on algorithms.
I don’t actually think the researchers are intellectually lazy or that scaling is low effort, these just seem like the most succinct terms to express a lack of pressure to adapt in a specific direction.
I don’t think people are currently being intellectually lazy. They might be rationally spending effort in ways that produce less insight per compute but faster insight per month than they would if they had less compute. I do think that despite the way compute limitation would make people try harder, things are still worse than they would be with less compute. But not as much worse as it naively seems, because of what you’ve mentioned here.
Senator Bernie Sanders is planning to introduce legislation that would ban the construction of new AI data centers. You can find his video announcement here, and here is the transcript:
You can just do things (propose an end to the AGI death race)
Unfortunately, this policy action is not that.
It’s the kind of action that when universalized does indeed end the AGI death race! That is in an important sense proposing an end to the AGI death race.
It’s also the kind of action that’s within the Overton window and if passed moves the window.
It slows it down a bit, perhaps.
If everyone stopped building datacenters you really have made a lot of progress towards stopping the death race (and of course, the algorithm that produces banning datacenter construction would probably not stop there).
I think this is a common misconception. I’m pretty sure algorithmic progress will eventually reach a point where what currently takes a datacenter will be possible on a single machine for a slightly longer training period. If that same algorithm runs on a datacenter it would produce something superhuman, but cutting down to only single-gpu training would then not be enough to completely stop. Algo progress is a slow slog of “grad student descent”, so it likely takes quite a bit longer, and maybe it takes enough longer to figure out alignment. But it doesn’t stop the death race, it just slows it down. Actual stopping would require shredding all silicon big enough to even run the fully trained AI, which doesn’t seem to be in the cards. I’m not saying datacenter construction is good or should continue, or that this won’t buy time, but I think people are wishful-thinking about how much time it buys.
Agree qualitatively (and possible quantitatively). However, there’s a quite large knock-on effect, which is a strong bundle of signals of “AGI is bad, don’t make AGI”. These signals move in various directions between different entities, carrying various messages, but they generally push against AGI. (E.g. signaling legitimacy of the Stop position; the US signaling to other states; society signaling to would-be capabilities researchers; Congress self-signaling “we’re trying to ban this whole thing and will continue to add patches to ban dangerous stuff”; etc.)
I mean, sure, eventually. The key question is how much of algorithmic progress is downstream of hardware scaling. My sense is around 50% of it, maybe a bit more, so that if you cut scaling, progress now happens at around 1/4th of the speed, which is of course huge and makes things a lot better.
Thinking this through step by step in the framework of the AI Futures Model:
First, I’ll check what the model says, then I’ll reconstruct the reasoning behind why it predicts that.
By default, with Daniel’s parameters, Automated Coder (AC) happens in 2030 and ASI happens in 2031 1.33 years later.
If I stop experiment and training compute growth at the start of 2027, then the model predicts Automated Coder in 2039 rather than 2030. So 4x slower in calendar time (exactly matching habyrka’s guess). It also looks to have well over a 5 year takeoff from AC to ASI as opposed to the default of 1.33 years.
I got this by plugging in this modified version of our time series to this unreleased branch of our website.
However, this is highly sensitive to the timing of the compute growth pause, because it’s a shock to the flow rather than the stock. e.g. if I instead stop growth at the start of 2029 as in this worksheet, then AC happens in Mar 2031, taking ~2.2 years instead of ~1.2, so slowing things down by <2x. It does still slow down takeoff from AC to ASI to 4 years, so by ~3x (and this is probably at least a slight underestimate because we don’t model hardware R&D automation).
Now I’ll reconstruct why this is the case using simplifications to the model (I actually did these calcluations before plugging the time series things into our model).
Currently, experiment compute is growing at around 3x/year, and human labor around 2x/year. Conditional on no AGI, we’re projecting experiment compute growth to slow to around 2x/year by 2030, and human labor growth to slow to around 1.5x/year.
Figuring out the effect of removing experiment growth is a bit complicated for various reasons.
On the margin, informed by interviews/surveys, we model a ~2.5x gain in “research effort” from 10x more experiment compute. Which if applied instantaneously, would mean a 2.5x slowdown in algorithmic progress.
We estimate that a 100x in human parallel coding labor gives a ~2x in research effort on the margin. We don’t model quantity of research labor, but I expect probably the gains would be relatively small as quality matters a lot more than quantity; let’s shade up to 3x.
So naively based on our model parameters and a simplified version of our model (Cobb-Douglas used to locally approximate a CES), by default the growth in research effort per year from experiment compute is 2.5^log(2-3)=~1.3-1.55x, and from human labor is sqrt(3)^log(1.5-2)=~1.1-1.2x. Meaning that roughly log(1.4)/(log(1.4)+log(1.15))=~70% of research effort growth is coming from experiment compute.
So to simplify from now on, let’s think about what happens if research effort growth has a one-time shock a constant 30% of what it otherwise would have been.
What does this actually mean in terms of the effect on algorithmic/software progess? This means that the shock in research effort growth will eventually cause the software growth rate to be 30% of what it would have been otherwise, but it steadily decreases toward 30% over time (intuitively, the immediate effect is 0 because you have to actually wait for the “missing new experiment compute” to take effect).
(possibly wrong) I had Claude generate roughly the trajectory of the growth rate change:
The above graph seems to give decent intuition for why the averaged-over-the-relevant-period slowdown in software progress from no more experiment compute might be about 2x (with the 2027.0 growth stoppage), and therefore the overall slowdown from no more compute might be about 4x. It looks like the average of the first 12 years might be close to 0.5.
All of the above is ignoring automation for simplicity. Taking into account automation would mean that more of the gains from labor, so the slowdown is smaller. But on the other hand, as you get more coding labor you get more bottlenecked on experiment compute (which the Cobb-Douglas doesn’t take into account); in the AIFM, you’d eventually get hard bottlenecked, you can have a maximum of 15x research effort gain from coding labor increases alone. Looks like these factors and other deviations from the model might roughly cancel out in the case we’re considering.
My current default assumption is that, yes, someone will build something that obsoletes human intellectual and physical labor. In those futures, the best-case scenario is “Maybe the AIs like keeping humans as pets (and won’t breed us the way we breed pugs).” [1] And the other alternative futures go sharply downhill from there.
So I think of delay in terms of survivor curves, like someone with terminal cancer. How much time can we buy the human race? Can the children alive today get to enjoy a decent lifetime for however long we all have? So I heavily favor delay, in much the same way that I’d favor cancer remission.
A global AI halt might even buy us quite a bit of time.
If I had to bet on a specific model as liking humans and being a responsible “pet owner”, then I currently suspect we might have the best odds with a descendant of Claude. I do actually think that “enculturation” and building morally thoughtful models that like humans gives us non-zero chance of a more acceptable outcome. But I would still prefer humans to control their own destiny.
Do you expect algorithmic progress to never hit diminishing returns?
“Eventually”, sure, but I don’t think that’s operative here. If we had the ASI recipe and could study it safely for ten years, we’d find a way to implement it in a single datacenter. But discovering it in a single data center is much harder. There is actually something missing from current LLMs, there’s a part of intelligence they just don’t have, and the only thing that seems to mitigate that issue is model size, so without ever-increasing model size and analysis of their training dynamics, I think any attempts to get the missing piece are throwing darts with the lights off. (To be fair I have pretty unusual timelines compared to most of LW so maybe what’s convincing to me shouldn’t be to you.)
I agree with the general concern, but it’d be clearly a move in the right direction on that front?
With this kind of proposal I’m more worried that it could lead to a unilateral slowdown just after having animated China to be much more aggressive on AI.
agreed, I’m not saying don’t do it. I’m replying to habryka saying it proposes an end to the race.
Doesn’t the USA have enough datacenters to stay ahead of china in the race to fully human-replacing AI for quite a while, even with a unilateral hardware pause? I’m not sure of this claim, but it’s currently my impression that the compute ratio is pretty dramatic.
I expect there’s already enough silicon in place to produce overwhelmingly superhuman AI that still has idiot-savant going on; completely stopping CPU and GPU production worldwide seems to me like it’d be a timeline-extending move, but not by more than a few years. Somewhere between 1.3x and 3x.
As of June 2025 the US has 5x as much compute as China, I’d expect the gap has grown with substantially more American than Chinese data centers coming online in the past ~9 months
https://epoch.ai/data-insights/ai-supercomputers-performance-share-by-country
I dont see how. If AGI/ASI is powerful, then the existing deployed compute will suddenly become more powerful when AGI happens. If that isn’t X risk, then a few more data centers won’t change things. This only matter in long timelines where more data centers are required to get AGI. I don’t think that is the case. I think copying the mammal neocortex etc will get us there and that doesn’t need more compute.
This is cool! I’m sad he spends so much of his time criticising the good part (AI doing tonnes of productive labour). I say this not because I want to demand every ally agree with me on every point, but because I want to early disavow beliefs that political expediency might want me to endorse.
It seems to me a meaningfully open question whether automating all human labor will end up net benefiting humans, even assuming we survive; of course it might, but I think much more dystopian outcomes also seem plausible. Markets tend to benefit humans because the price signals we send tend to correlate with our relative needs, and hence with our welfare; I think it is not obvious that this correlation will persist once humans become unable to generate economic value.
What would AI labs do differently if this were made law? Couldn’t they build datacenters outside the US?
While I’m glad that these things are starting to be seriously discussed in the open by those in positions of power, this sounds like a blanket ban on all new datacenters? This seems too broad to me. The ideal moratorium would be on large GPU clusters, or large AI specific infrastructure, right?
On another note, I’m somewhat concerned that banning large compute infrastructure will concentrate pressure on algorithmic progress if there is no “low effort” [1] outlet of just scaling the compute. Maybe regulating the rate at which large scale compute infrastructure is constructed will still allow researchers to be “intellectually lazy” [2] by offloading progress onto compute scaling instead of innovating on algorithms.
I don’t actually think the researchers are intellectually lazy or that scaling is low effort, these just seem like the most succinct terms to express a lack of pressure to adapt in a specific direction.
Same as above.
I don’t think people are currently being intellectually lazy. They might be rationally spending effort in ways that produce less insight per compute but faster insight per month than they would if they had less compute. I do think that despite the way compute limitation would make people try harder, things are still worse than they would be with less compute. But not as much worse as it naively seems, because of what you’ve mentioned here.