Policy makers do not know this. They know that someone is telling them this. They definitely do not know that they will get the economic promises of AGI on the timescales they care about, if they support this particular project.
I feel differently here. It seems that a lot of governments have woken up to AI in the past few years, and are putting it at the forefront of national strategies, e.g. see the headline here. In the past year there has been a lot of movement in the regulatory space, but I’m still getting undertones of ‘we realise that AI is going to be huge, and we want to establish global leadership in this technology’.
So going back to your nuclear example, I think the relevant question is: ‘What allowed policymakers to gain the necessary support to push stringent nuclear regulation through, even though it offered huge economic benefits?’. I think there are two things:
It takes a significant amount of time, ~6-8 years, for a nuclear powerplant to be built and begin operating (and even longer for it to breakeven). So whilst they are economically practical in the long-term, it can be hard to garner the support for the huge initial investment. To make this clearer, imagine if it took ~1 year to build a nuclear power plant, and 2 years for it to breakeven. If that were the case, I think it would have been harder to push stringent regulation through.
There was a lot of irrational public fear about anything nuclear, due to powerplant accidents, the Cold War and memories of nuclear weapon use during WWII.
With respect to AI, I don’t think (1) holds. That is, the economic benefits of AI will be far easier to realise than that for nuclear (you can train and deploy an AI system within a year, and likely breakeven a few years after that), meaning that policymaker support for regulation will be harder.
(2) might hold, this really depends on the nature of AI accidents over the next few years, and their impacts on public perception. I’d be interested in your thoughts here.
Yep, buy-in from the majority of frontier labs seems pretty important here. If OpenAI went out and said ‘We think that there’s a 10% chance that AGI we develop kills over 1 billion people’, but Meta kept their current stance (along the lines of ‘we think that the AI x-risk discussion is fearmongering and the systems we’re building will be broadly beneficial for humanity’) then I’d guess that OpenAI would lose a ton of business. From the point of view of an enterprise using OpenAI’s products, it can’t help your public image to be using the products of a lab that thinks it has a 10% chance of ending the world—especially if there are other labs offering similar products that don’t carry this burden. In a worst-case scenario, I can imagine that this puts OpenAI directly in the firing line of regulators, whilst Meta gets off far more lightly.