But it’s more blocked on political will rather than physical impossibility, at least for now.
Sorry, but I don’t see that.
For example, how one would prevent one of the Latin American drug cartels from doing a successful ASI project?
They are already very interested in all kinds of non-standard tech and weaponry. They don’t seem to be easily defeatable, especially in a sustainable way. They can muster tons of resources. And there will be so many existing useless GPU chips, not gainfully usable legally under the new regime, and therefore up for grabs.
If everyone else stops, trying to achieve decisive strategic advantage via an ASI would look very tempting to some of those cartels. A surprisingly high number of AI researchers would be ready to defect to them in a “prohibition world”.
That is, unless one hopes to muster political will to force the political recognition of those cartels as state-level actors in hope that they would become parties to this kind of agreement. That’s a really big ask, all things considering...
Hey, thanks for your comment! My colleague Robi will have a blog post out soon dealing with the question “how does the agreement prevent ASI development in countries that are not part of the agreement?”, which it sounds like is basically what you’re getting at. I think this is a non-trivial problem, and Latin American drug cartels seem like an interesting case study—good point!
As a clarification, I don’t expect this to be true in our agreement: “there will be so many existing useless GPU chips”. Our agreement lets existing GPUs stick around as long as they’re being sufficiently monitored (Chip Use Verification), and it’s fair game for people to do inference on today’s models. So I think there will be a strong legal market for GPUs, albeit not as strong as today’s because there will be some restrictions on how they can be used.
As a tldr on my thinking here: It’s similar to existing WMD non-proliferation problems. Coalition countries will crack down pretty hard on chip smuggling and try to deny cartel access to chips. The know-how for frontier AI development is pretty non-trivial and there aren’t all that many relevant people. While some of them might defect to join the cartels, I think there will be a lot of ideological/social/peer pressure against doing this, in addition to various prohibition efforts from the government (e.g., how governments intervene on terrorist recruitment).
Sorry, but I don’t see that.
For example, how one would prevent one of the Latin American drug cartels from doing a successful ASI project?
They are already very interested in all kinds of non-standard tech and weaponry. They don’t seem to be easily defeatable, especially in a sustainable way. They can muster tons of resources. And there will be so many existing useless GPU chips, not gainfully usable legally under the new regime, and therefore up for grabs.
If everyone else stops, trying to achieve decisive strategic advantage via an ASI would look very tempting to some of those cartels. A surprisingly high number of AI researchers would be ready to defect to them in a “prohibition world”.
That is, unless one hopes to muster political will to force the political recognition of those cartels as state-level actors in hope that they would become parties to this kind of agreement. That’s a really big ask, all things considering...
Hey, thanks for your comment! My colleague Robi will have a blog post out soon dealing with the question “how does the agreement prevent ASI development in countries that are not part of the agreement?”, which it sounds like is basically what you’re getting at. I think this is a non-trivial problem, and Latin American drug cartels seem like an interesting case study—good point!
As a clarification, I don’t expect this to be true in our agreement: “there will be so many existing useless GPU chips”. Our agreement lets existing GPUs stick around as long as they’re being sufficiently monitored (Chip Use Verification), and it’s fair game for people to do inference on today’s models. So I think there will be a strong legal market for GPUs, albeit not as strong as today’s because there will be some restrictions on how they can be used.
As a tldr on my thinking here: It’s similar to existing WMD non-proliferation problems. Coalition countries will crack down pretty hard on chip smuggling and try to deny cartel access to chips. The know-how for frontier AI development is pretty non-trivial and there aren’t all that many relevant people. While some of them might defect to join the cartels, I think there will be a lot of ideological/social/peer pressure against doing this, in addition to various prohibition efforts from the government (e.g., how governments intervene on terrorist recruitment).