I hear a lot of discussion of treaties to monitor compute or ban AI or whatever. But the word “treaty” has a specific meaning that often isn’t what people mean. Specifically, treaties are a particular kind of international agreement that (in the US) require Senate approval, and there are a lot of other types of international agreement (e.g. executive agreements that are just made by the president). Treaties are generally more serious and harder to withdraw from.
As a total non-expert, “treaty” does actually seem to me like the vibe that e.g. our MIRI friends are going for—they want something analogous to major nuclear arms control agreements, which are usually treaties. So to me, “treaty” seems like a good word for the central example of what MIRI wants, but international agreements that they’re very happy about won’t necessarily be treaties.
I wouldn’t care about this point, except that I’ve heard that national security experts are prickly about the word “treaty” being used when “international agreement” could have been used instead (and if you talk about treaties they might assume you’re using the word ignorantly even if you aren’t). My guess is that you should usually conform to their shibboleth here and say “international agreement” when you don’t specifically want to talk about treaties.
But the word “treaty” has a specific meaning that often isn’t what people mean.
It doesn’t really have a specific meaning. It has multiple specific meanings. In US constitution it means an agreement with two-thirds approval of the Senate. The Vienna Convention on the Law of Treaties, Article 2(1)(a) defines a treaty as:
“an international agreement concluded between States in written form and governed by international law, whether embodied in a single instrument or in two or more related instruments and whatever its particular designation.”
Among international agreements, in US law you have a treaty (executive + 2⁄3 of the Senate), congressional–executive agreements (executive + 1⁄2 of the Congress + 1⁄2 of the Senate) and sole executive agreements (just the executive).
Both treaties (according to the US constitution) and congressional–executive agreements can be supreme law of the land and thus do things that a sole executive agreements which does not create law. From the perspective of other countries both treaties (according to the US constitution) and congressional–executive agreements are treaties.
If you would try to do an AI ban via a sole executive agreement, companies would likely sue and argue that the US president does not have the power to do that and the current US Supreme Court would likely declare the AI ban ineffectual.
When passing serious agreements like NAFTA where there’s a lot of opposition to parts of the agreement turned out to be easier to gather simply majorities + 60 Senate votes to move past the filibuster than 67 Senate votes, they are passed as congressional–executive agreements. If getting votes for the AI treaty is hard, it’s likely that it would also be passed as a congressional–executive agreements. If you do lobbying you probably should know the term congressional–executive agreements and be able to say it when a national security expert asks you what you want.
Using “international agreement” in the MIRI writing has the problem that it suggests a sole executive agreement would do the job when it wouldn’t.
(disclaimer, not actually at MIRI but involved in some discussions, reporting my understanding of their position)
There was some internal discussion about whether to call the things in the IABIED website “treaties” or “agreements”, which included asking for recommendations from people more politically savvy.
I don’t know much about who the politically savvy people were or how competent they were, but, the advice they returned was:
Yep, “agreement” is indeed easier to get political buy-in for. But, it does have the connotation of being easier to back out of. And, from our understanding of your political goals, “treaty” is probably more like the thing you actually want. But, either is kinda reasonable to say, in this context.
And, they decided to stick with Treaty, because kind of the whole point of the MIRI agenda is say clearly/explicitly “this is what you actually would need to do to reliably survive, according to our models”, as opposed to “here’s the nearest within-current-overton-window political option that maybe helps a reasonable amount,” With the goal to enable the more complete/reliable solution to be talked about, and increase the odds that whatever political agreements end up happening are likely to actually matter.
And, whatever politically workable things that actually happen will end up actually happening, and other people can push forward on whatever the best available compromise seems to be.
(I think the MIRI-decisionmakers agreed that “international agreement” was potentially a reasonable thing to ask for, depending on the exact connotations and landscape, but “treaty” seemed like the right tradeoff for them)
I maybe agree with part of your point, that, people (other than MIRI), who are using the word “treaty” without really reflecting on whether they should be using “treaty” or “agreement”, should at least think through what their goals are and which word better serves them.
Yeah, when Thomas Larsen mentioned that to me (presumably from a shared source that you’re getting your info from), I mentioned it to MIRI, and they went and asked for feedback and ended up getting the above-mentioned-advice which seemed to indicate it has some costs, but wasn’t like an obviously wrong call.
And, from our understanding of your political goals, “treaty” is probably more like the thing you actually want. But, either is kinda reasonable to say, in this context.
What I understand Yudkowsky and Soares want might be summarized as something like:
“Large numbers of GPUs should be treated like large numbers of uranium gas centrifuges.”
“Publishing details of certain AI algorithms should be treated like publishing detailed engineering guidelines for improving the yield of a nuclear device.”
“Researching certain kinds of AI algorithms should be treated like doing gain of function research into highly contagious airborne Ebola.” Actually, we probably don’t take bio threats nearly as seriously as we should.
The thing they want here includes a well-written treaty. Or an “international agreement.” If you buy their assumptions, then yes, you would want to lay out bright lines around things like data center capacity and monitoring, chip fabs, and possibly what kind of AI research is publishable.
But nuclear deterence also has quite a few other moving parts beyond the international agreements, including:
The gut knowledge of the superpowers that if they screw this up, then their entire civilization dies.[1]
A tense, paranoid standoff between the key players.[2]
A system of economic sanctions strongly backed by major powers.
Quiet conversations between government officials and smart people where the officials say things like, “Pretty please never mention that idea again.”[3]
The key point in all of these circumstances is that powerful people and countries believe that “If we get this wrong, we might die.” This isn’t a case of “We want 80% fewer of our cities to be blown up with fusion bombs”. It’s a case of “We want absolutely none of our cities blown up by fusion bombs, because it won’t stop with just one or two.”
And so the rule that “You may only own up to X uranium gas centrifuges” is enforced using multiple tools, ranging from treaties/agreements to quiet requests to unilateral exercises of state power.
Possibly while singing “Duck and Cover”. Which is actually decent advice for a nuclear war. Think of a nuclear explosion as a cross between a tornado and a really bright light that kills you. Getting away from a window and under a desk is not the worst heuristic, and even simple walls provide some shielding against gamma radiation. Sadly, this probably doesn’t work against SkyNet, no matter what the meme suggests. But an entire generation of children saw these videos and imagined their deaths. And some of those people still hold power. When the last of them retire, nuclear deterence will likely weaken.
This isn’t at all a settled question internally; some folks at MIRI prefer the ‘international agreement’ language, and some prefer the ‘treaty’ language, and the contents of the proposals (basically only one of which is currently public, the treaty from the online resources for the book) vary (some) based on whether it’s a treaty or an international agreement, since they’re different instruments.
Afaict the mechanism by which NatSec folks think treaty proposals are ‘unserious’ is that treaties are the lay term for the class of objects (and are a heavy lift in a way most lay treaty-advocates don’t understand). So if you say “treaty” and somehow indicate that you in fact know what that is, it mitigates the effect significantly.
I think most TGT outputs are going to use the international agreement language, since they’re our ‘next steps’ arm (you usually get some international agreement ahead of a treaty; I currently expect a lot of sentences like “An international agreement and, eventually, a treaty” in future TGT outputs).
My current understanding is that Nate wanted to emphasize what would actually be sufficient by his lights, looked into the differences in the various types of instruments, and landed back on treaty, which is generally in line with the ‘end points’ emphasis of the book project as a whole.
In the >a dozen interactions where we brought this up with our most authoritative NatSec contact (many of which I was present for), he did not vomit blood even once!
It’s definitely plausible the treaty draft associated with the book is taking some hits here, but I think this was weighed against ’well, if we tell them what we want, and we actually get it, and it’s too weak, that’s a loss.” Strategically, I would not endorse everyone operating from that frame, but I do endorse it existing as part of the portfolio of approaches here, and am glad to support MIRI as the org in the room most willing to make that kind of call.
I’m glad Buck is calling this out, so that other actors don’t blindly follow the book’s lead and deploy ‘treaty’ unwisely.
(I think Ray’s explanation is coherent with mine, but speaks to the experience of someone who only saw something like ‘user-facing-book-side’, whereas I was in a significant subset of the conversations where this was being discussed internally, although never with Nate, so I wouldn’t be shocked if he’s seeing it differently.)
Agreed! I’d be curious to know if the natsec experts that you’ve engaged with would recommend other international law instruments that do not require ratification.
This presentation by UNECE actually does a decent job at providing straight-forward explanations for what a Treaty is, differences with Conventions, Charters, declarations… (In case anyone following this thread finds it useful).
Personally, I am more concerned about avoiding agreement terms or definitions that are too easy to game- specially if the chosen legal instrument requires transposition into national law (e.g.: to make it binding in the EU, having to pass a regulation).
My opinion, FWIW, is that both treaty and international agreement (or “deal”, etc.) have upsides and downsides. And it’s hard to predict those considerations’ political salience or direction in the long term—e.g., just a few years ago, Republicans’ main complaint against the JCPOA (aka “the Iran Nuclear Deal”) was that it wasn’t an actual treaty, and should have been, which would be a very odd argument in 2025.
I think as long as MIRI says things like “or other international agreement or set of customary norms” on occasion it should be fine. It certainly doesn’t nails on the chalkboard me to hear “treaty” on a first glance, and in any long convo I model MIRI as saying something like “or look, we’d be open to other things that get this done too, we think a treaty is preferable but are open to something else that solves the same problem.”
I hear a lot of discussion of treaties to monitor compute or ban AI or whatever. But the word “treaty” has a specific meaning that often isn’t what people mean. Specifically, treaties are a particular kind of international agreement that (in the US) require Senate approval, and there are a lot of other types of international agreement (e.g. executive agreements that are just made by the president). Treaties are generally more serious and harder to withdraw from.
As a total non-expert, “treaty” does actually seem to me like the vibe that e.g. our MIRI friends are going for—they want something analogous to major nuclear arms control agreements, which are usually treaties. So to me, “treaty” seems like a good word for the central example of what MIRI wants, but international agreements that they’re very happy about won’t necessarily be treaties.
I wouldn’t care about this point, except that I’ve heard that national security experts are prickly about the word “treaty” being used when “international agreement” could have been used instead (and if you talk about treaties they might assume you’re using the word ignorantly even if you aren’t). My guess is that you should usually conform to their shibboleth here and say “international agreement” when you don’t specifically want to talk about treaties.
It doesn’t really have a specific meaning. It has multiple specific meanings. In US constitution it means an agreement with two-thirds approval of the Senate. The Vienna Convention on the Law of Treaties, Article 2(1)(a) defines a treaty as:
Among international agreements, in US law you have a treaty (executive + 2⁄3 of the Senate), congressional–executive agreements (executive + 1⁄2 of the Congress + 1⁄2 of the Senate) and sole executive agreements (just the executive).
Both treaties (according to the US constitution) and congressional–executive agreements can be supreme law of the land and thus do things that a sole executive agreements which does not create law. From the perspective of other countries both treaties (according to the US constitution) and congressional–executive agreements are treaties.
If you would try to do an AI ban via a sole executive agreement, companies would likely sue and argue that the US president does not have the power to do that and the current US Supreme Court would likely declare the AI ban ineffectual.
When passing serious agreements like NAFTA where there’s a lot of opposition to parts of the agreement turned out to be easier to gather simply majorities + 60 Senate votes to move past the filibuster than 67 Senate votes, they are passed as congressional–executive agreements. If getting votes for the AI treaty is hard, it’s likely that it would also be passed as a congressional–executive agreements. If you do lobbying you probably should know the term congressional–executive agreements and be able to say it when a national security expert asks you what you want.
Using “international agreement” in the MIRI writing has the problem that it suggests a sole executive agreement would do the job when it wouldn’t.
Strong upvoted to signal boost, but again note I don’t know what I’m talking about.
(disclaimer, not actually at MIRI but involved in some discussions, reporting my understanding of their position)
There was some internal discussion about whether to call the things in the IABIED website “treaties” or “agreements”, which included asking for recommendations from people more politically savvy.
I don’t know much about who the politically savvy people were or how competent they were, but, the advice they returned was:
And, they decided to stick with Treaty, because kind of the whole point of the MIRI agenda is say clearly/explicitly “this is what you actually would need to do to reliably survive, according to our models”, as opposed to “here’s the nearest within-current-overton-window political option that maybe helps a reasonable amount,” With the goal to enable the more complete/reliable solution to be talked about, and increase the odds that whatever political agreements end up happening are likely to actually matter.
And, whatever politically workable things that actually happen will end up actually happening, and other people can push forward on whatever the best available compromise seems to be.
(I think the MIRI-decisionmakers agreed that “international agreement” was potentially a reasonable thing to ask for, depending on the exact connotations and landscape, but “treaty” seemed like the right tradeoff for them)
I maybe agree with part of your point, that, people (other than MIRI), who are using the word “treaty” without really reflecting on whether they should be using “treaty” or “agreement”, should at least think through what their goals are and which word better serves them.
What I hear is that the natsec people judge people for using “treaty” in cases like this. Maybe MIRI looked into it and has better info than me; idk.
Yeah, when Thomas Larsen mentioned that to me (presumably from a shared source that you’re getting your info from), I mentioned it to MIRI, and they went and asked for feedback and ended up getting the above-mentioned-advice which seemed to indicate it has some costs, but wasn’t like an obviously wrong call.
Thinking about the advisor’s comments:
What I understand Yudkowsky and Soares want might be summarized as something like:
“Large numbers of GPUs should be treated like large numbers of uranium gas centrifuges.”
“Publishing details of certain AI algorithms should be treated like publishing detailed engineering guidelines for improving the yield of a nuclear device.”
“Researching certain kinds of AI algorithms should be treated like doing gain of function research into highly contagious airborne Ebola.”Actually, we probably don’t take bio threats nearly as seriously as we should.The thing they want here includes a well-written treaty. Or an “international agreement.” If you buy their assumptions, then yes, you would want to lay out bright lines around things like data center capacity and monitoring, chip fabs, and possibly what kind of AI research is publishable.
But nuclear deterence also has quite a few other moving parts beyond the international agreements, including:
The gut knowledge of the superpowers that if they screw this up, then their entire civilization dies.[1]
A tense, paranoid standoff between the key players.[2]
A system of economic sanctions strongly backed by major powers.
Mysterious bad things happening to uranium centrifuges.
Quiet conversations between government officials and smart people where the officials say things like, “Pretty please never mention that idea again.”[3]
The key point in all of these circumstances is that powerful people and countries believe that “If we get this wrong, we might die.” This isn’t a case of “We want 80% fewer of our cities to be blown up with fusion bombs”. It’s a case of “We want absolutely none of our cities blown up by fusion bombs, because it won’t stop with just one or two.”
And so the rule that “You may only own up to X uranium gas centrifuges” is enforced using multiple tools, ranging from treaties/agreements to quiet requests to unilateral exercises of state power.
Possibly while singing “Duck and Cover”. Which is actually decent advice for a nuclear war. Think of a nuclear explosion as a cross between a tornado and a really bright light that kills you. Getting away from a window and under a desk is not the worst heuristic, and even simple walls provide some shielding against gamma radiation. Sadly, this probably doesn’t work against SkyNet, no matter what the meme suggests. But an entire generation of children saw these videos and imagined their deaths. And some of those people still hold power. When the last of them retire, nuclear deterence will likely weaken.
“The whole point of a Doomsday machine is lost if you keep it a secret!”
This is paraphrased, but it’s from a real example.
This isn’t at all a settled question internally; some folks at MIRI prefer the ‘international agreement’ language, and some prefer the ‘treaty’ language, and the contents of the proposals (basically only one of which is currently public, the treaty from the online resources for the book) vary (some) based on whether it’s a treaty or an international agreement, since they’re different instruments.
Afaict the mechanism by which NatSec folks think treaty proposals are ‘unserious’ is that treaties are the lay term for the class of objects (and are a heavy lift in a way most lay treaty-advocates don’t understand). So if you say “treaty” and somehow indicate that you in fact know what that is, it mitigates the effect significantly.
I think most TGT outputs are going to use the international agreement language, since they’re our ‘next steps’ arm (you usually get some international agreement ahead of a treaty; I currently expect a lot of sentences like “An international agreement and, eventually, a treaty” in future TGT outputs).
My current understanding is that Nate wanted to emphasize what would actually be sufficient by his lights, looked into the differences in the various types of instruments, and landed back on treaty, which is generally in line with the ‘end points’ emphasis of the book project as a whole.
In the >a dozen interactions where we brought this up with our most authoritative NatSec contact (many of which I was present for), he did not vomit blood even once!
It’s definitely plausible the treaty draft associated with the book is taking some hits here, but I think this was weighed against ’well, if we tell them what we want, and we actually get it, and it’s too weak, that’s a loss.” Strategically, I would not endorse everyone operating from that frame, but I do endorse it existing as part of the portfolio of approaches here, and am glad to support MIRI as the org in the room most willing to make that kind of call.
I’m glad Buck is calling this out, so that other actors don’t blindly follow the book’s lead and deploy ‘treaty’ unwisely.
(I think Ray’s explanation is coherent with mine, but speaks to the experience of someone who only saw something like ‘user-facing-book-side’, whereas I was in a significant subset of the conversations where this was being discussed internally, although never with Nate, so I wouldn’t be shocked if he’s seeing it differently.)
Agreed! I’d be curious to know if the natsec experts that you’ve engaged with would recommend other international law instruments that do not require ratification.
This presentation by UNECE actually does a decent job at providing straight-forward explanations for what a Treaty is, differences with Conventions, Charters, declarations… (In case anyone following this thread finds it useful).
Personally, I am more concerned about avoiding agreement terms or definitions that are too easy to game- specially if the chosen legal instrument requires transposition into national law (e.g.: to make it binding in the EU, having to pass a regulation).
My opinion, FWIW, is that both treaty and international agreement (or “deal”, etc.) have upsides and downsides. And it’s hard to predict those considerations’ political salience or direction in the long term—e.g., just a few years ago, Republicans’ main complaint against the JCPOA (aka “the Iran Nuclear Deal”) was that it wasn’t an actual treaty, and should have been, which would be a very odd argument in 2025.
I think as long as MIRI says things like “or other international agreement or set of customary norms” on occasion it should be fine. It certainly doesn’t nails on the chalkboard me to hear “treaty” on a first glance, and in any long convo I model MIRI as saying something like “or look, we’d be open to other things that get this done too, we think a treaty is preferable but are open to something else that solves the same problem.”