We might put more effort into thinking of one later. (b) Even the relatively structureless thing we proposed—research transparency + MACD but otherwise the nations of the world just have to muddle through and handle things on a case by case basis—is a significant improvement over the status quo, for reasons we’ve articulated in the piece. You seem to disagree with this but I don’t see why.
(1) Part of my objection is that I anticipate the structurelessness of it to be an obstacle to its adoption.
Like this is what a senior decision-maker in Bejing or Washington will be contemplating: They’re going to put their economy at the mercy of their greatest geopolitical rival. That is, after this deal, it will become relatively trivial for either the US or China to cripple the economy of the other, albeit at the cost of their own economy being subsequently crippled. Of course, you could say that before making the deal they were at risk of being taken over by some AI, but this risk was diffuse and uncertain; the risk that they’re signing up for now is concrete and definite.
And this lever of destruction could be used, of course, for reasons other than to stop an AI takeoff, and decision-makers in both countries will be acutely aware of this. Suppose China decides that, if it cripples everyone’s compute, it would gain a vast differential advantage because its economy depends less on GPUS than the USA’s economy depends: then it would be in China’s advantage to get in a MACD situation, then to trigger it, and subsequently dominate the US; and a US political figure, anticipating this, would object to MACD. Or suppose that China thinks, “Hrm, the US is an unreliable actor, and a quick and bloodless MACD might be triggered by, for instance, a senile or unstable US President, of which the US recently has had a fair number.” Thus, because China doesn’t want to trigger MACD for reasons of random shit in the future, it would object to it. And so on and so forth.
What makes this worse is that part of what makes nuclear MAD a plausible means of peace is that there’s a clear signal of “Have the nukes been launched,” while there isn’t as clear signal of danger in the case of AI takeoff; and a rational decision-maker, seeing this, will update downwards about whether MACD will be triggered for reasons relating to AI takeoff and upwards about whether MACD will be triggered for some other random shit.
That is, part of what makes nuclear MAD work is that there are radar stations in Siberia and Greenland and Canada, which can detect ICBMs and bombers that have been launched. The US knows that it could detect things being launched, and Russia knows that the US knows, and the US knows that Russia knows that the US knows, and so forth. Imagine if, by contrast, the sign for “the nukes have been launched,” was that panel of experts, notorious for disagreeing among themselves, came to a consensus that the nukes had been launched or might have been launched. If this were so, then nuclear MAD would be much less effective as a game-theoretic means of peace. But of course this is the situation that we’re in with regards to AI.
(2) Part of my confusion I just don’t know what parts of the scenario are predictions and which ones are hopes. Like in the response to Thane:
A consortium of multiple governments—some of which are actual democracies thanks to the transparency requirements which help prevent AI-assisted executive power grabs—is way less bad than a single global dictator, for example.
In general, I’m dubious whether democracies other than the US (is the US to be an actual democracy?) to have any decision-making clout in the Consortium, because China would object to the possibility of being outvoted. But like, I don’t know how much of “Consortium influenced by many democracies” is part of the prediction of what (“transparency + MACD”) gets you, given those two goals; or if “Consortium influenced by many democracies” is maybe a bonus that we might get after setting up the structure of “transparency + MACD,” but a bonus that we’re unlikely to get.
(1) Part of my objection is that I anticipate the structurelessness of it to be an obstacle to its adoption.
Like this is what a senior decision-maker in Bejing or Washington will be contemplating: They’re going to put their economy at the mercy of their greatest geopolitical rival. That is, after this deal, it will become relatively trivial for either the US or China to cripple the economy of the other, albeit at the cost of their own economy being subsequently crippled. Of course, you could say that before making the deal they were at risk of being taken over by some AI, but this risk was diffuse and uncertain; the risk that they’re signing up for now is concrete and definite.
And this lever of destruction could be used, of course, for reasons other than to stop an AI takeoff, and decision-makers in both countries will be acutely aware of this. Suppose China decides that, if it cripples everyone’s compute, it would gain a vast differential advantage because its economy depends less on GPUS than the USA’s economy depends: then it would be in China’s advantage to get in a MACD situation, then to trigger it, and subsequently dominate the US; and a US political figure, anticipating this, would object to MACD. Or suppose that China thinks, “Hrm, the US is an unreliable actor, and a quick and bloodless MACD might be triggered by, for instance, a senile or unstable US President, of which the US recently has had a fair number.” Thus, because China doesn’t want to trigger MACD for reasons of random shit in the future, it would object to it. And so on and so forth.
What makes this worse is that part of what makes nuclear MAD a plausible means of peace is that there’s a clear signal of “Have the nukes been launched,” while there isn’t as clear signal of danger in the case of AI takeoff; and a rational decision-maker, seeing this, will update downwards about whether MACD will be triggered for reasons relating to AI takeoff and upwards about whether MACD will be triggered for some other random shit.
That is, part of what makes nuclear MAD work is that there are radar stations in Siberia and Greenland and Canada, which can detect ICBMs and bombers that have been launched. The US knows that it could detect things being launched, and Russia knows that the US knows, and the US knows that Russia knows that the US knows, and so forth. Imagine if, by contrast, the sign for “the nukes have been launched,” was that panel of experts, notorious for disagreeing among themselves, came to a consensus that the nukes had been launched or might have been launched. If this were so, then nuclear MAD would be much less effective as a game-theoretic means of peace. But of course this is the situation that we’re in with regards to AI.
(2) Part of my confusion I just don’t know what parts of the scenario are predictions and which ones are hopes. Like in the response to Thane:
In general, I’m dubious whether democracies other than the US (is the US to be an actual democracy?) to have any decision-making clout in the Consortium, because China would object to the possibility of being outvoted. But like, I don’t know how much of “Consortium influenced by many democracies” is part of the prediction of what (“transparency + MACD”) gets you, given those two goals; or if “Consortium influenced by many democracies” is maybe a bonus that we might get after setting up the structure of “transparency + MACD,” but a bonus that we’re unlikely to get.
__ Re. 2: Yeah checks out