This seems like a good thing for labs to do[1]. I’d go one step earlier and propose that labs make a clear and explicit page (on their website or similar) stating their views on the risk from powerful AI systems. The proposal given in this post seems somewhat more ambitious and costly than the thing I’m proposing in this comment, though the proposal in the post is correspondingly somewhat better.
As far as what a “page stating their views on risk” looks like, I’m imagining something like (numbers are made up):
Views within [AI LAB] vary, but leadership thinks that the risk of AI killing over 1 billion people is around 10%. We think the probability of an AI originating from [AI LAB] killing over 1 billion people in the next 5 years is around 3%.
We specifically think that the probability of an AI system mostly autonomously taking over a large chunk of the world is around 10%. The risk of an AI system assisting with terrorism which kills over 1 billion people is around 3%.
…
AI labs often use terms like “AI safety” and “catastrophe”. It’s probably unclear what problem these terms are pointing at. I’d like it if whenever they said “catastrophe” they say something like:
We do XYZ to reduce the probability of AI caused catastrophe (e.g. the deaths of over 1 billion people, see here for our views on AI risk)
Where here links to the page discussed above.
And similar for using the terms AI safety:
by AI safety, we primarily mean the problem of avoiding AI caused catastrophe (e.g. the deaths of over 1 billion people, see here for our views on AI risk)
I’d consider this ask fulfilled even if this page stated quite optimistic views. At that point, there would be a clear disagreement to highlight.
I’m not sure about how costly these sorts of proposals are (e.g. because it makes customers think you’re crazy). Possibly, labs could coordinate to release things like this simultaneously to avoid tragedy of the commons (there might be anti-trust issues with this).
I’m not sure about how costly these sorts of proposals are (e.g. because it makes customers think you’re crazy). Possibly, labs could coordinate to release things like this simultaneously to avoid tragedy of the commons (there might be anti-trust issues with this).
Yep, buy-in from the majority of frontier labs seems pretty important here. If OpenAI went out and said ‘We think that there’s a 10% chance that AGI we develop kills over 1 billion people’, but Meta kept their current stance (along the lines of ‘we think that the AI x-risk discussion is fearmongering and the systems we’re building will be broadly beneficial for humanity’) then I’d guess that OpenAI would lose a ton of business. From the point of view of an enterprise using OpenAI’s products, it can’t help your public image to be using the products of a lab that thinks it has a 10% chance of ending the world—especially if there are other labs offering similar products that don’t carry this burden. In a worst-case scenario, I can imagine that this puts OpenAI directly in the firing line of regulators, whilst Meta gets off far more lightly.
I’m not sure this effect is as strong as one might think. For one, Dario Amodei (CEO of Anthropic) claimed his P(doom) was around 25% (specifically, “the chance that his technology could end human civilisation”). I remember Sam Altman saying something similar, but can’t find an exact figure right now. Meanwhile, Yann LeCun (Chief AI Scientist at Meta) maintains approximately the stance you describe. None of this, as far as I’m aware, has led to significant losses for OpenAI or Anthropic.
Is it really the case that making these claims at an institutional level, on a little corner of one’s website, is so much stronger than the CEO of one’s company espousing these views very publicly in interviews? Intuitively, this seems like it wouldn’t make a massive difference.
In a worst-case scenario, I can imagine that this puts OpenAI directly in the firing line of regulators, whilst Meta gets off far more lightly.
I’m interested to know if there’s any precedent for this, ie. a company being regulated further because they claimed their industry needed it, while those restrictions weren’t applied universally.
This seems like a good thing for labs to do[1]. I’d go one step earlier and propose that labs make a clear and explicit page (on their website or similar) stating their views on the risk from powerful AI systems. The proposal given in this post seems somewhat more ambitious and costly than the thing I’m proposing in this comment, though the proposal in the post is correspondingly somewhat better.
As far as what a “page stating their views on risk” looks like, I’m imagining something like (numbers are made up):
AI labs often use terms like “AI safety” and “catastrophe”. It’s probably unclear what problem these terms are pointing at. I’d like it if whenever they said “catastrophe” they say something like:
Where here links to the page discussed above.
And similar for using the terms AI safety:
I’d consider this ask fulfilled even if this page stated quite optimistic views. At that point, there would be a clear disagreement to highlight.
I’m not sure about how costly these sorts of proposals are (e.g. because it makes customers think you’re crazy). Possibly, labs could coordinate to release things like this simultaneously to avoid tragedy of the commons (there might be anti-trust issues with this).
Though I maybe disagree with various specific statements in this post.
Yep, buy-in from the majority of frontier labs seems pretty important here. If OpenAI went out and said ‘We think that there’s a 10% chance that AGI we develop kills over 1 billion people’, but Meta kept their current stance (along the lines of ‘we think that the AI x-risk discussion is fearmongering and the systems we’re building will be broadly beneficial for humanity’) then I’d guess that OpenAI would lose a ton of business. From the point of view of an enterprise using OpenAI’s products, it can’t help your public image to be using the products of a lab that thinks it has a 10% chance of ending the world—especially if there are other labs offering similar products that don’t carry this burden. In a worst-case scenario, I can imagine that this puts OpenAI directly in the firing line of regulators, whilst Meta gets off far more lightly.
I’m not sure this effect is as strong as one might think. For one, Dario Amodei (CEO of Anthropic) claimed his P(doom) was around 25% (specifically, “the chance that his technology could end human civilisation”). I remember Sam Altman saying something similar, but can’t find an exact figure right now. Meanwhile, Yann LeCun (Chief AI Scientist at Meta) maintains approximately the stance you describe. None of this, as far as I’m aware, has led to significant losses for OpenAI or Anthropic.
Is it really the case that making these claims at an institutional level, on a little corner of one’s website, is so much stronger than the CEO of one’s company espousing these views very publicly in interviews? Intuitively, this seems like it wouldn’t make a massive difference.
I’m interested to know if there’s any precedent for this, ie. a company being regulated further because they claimed their industry needed it, while those restrictions weren’t applied universally.