Many of the same people who publicly make a great fuss of this don’t seem to care that Nvidia is selling GPUs to China by way of Singapore. Their purpose is to drag up China as an excuse for continuing what they are doing. If China has more GPUs, all the better for them in the future; China will be a greater threat that they can use to continue what they are doing.
Their model hasn’t chosen an explanation like “China will play the role of the movie bad guy at any cost to their own selfish interests”. They aren’t reasoning at GPT 3.5-level, only GPT-2 level. China is a scary word-vector, so they can argue that if the USA tries to slow down against extinction risk, then China won’t slow down, because that’s a scary thing so “China” is a word-vector that the scary thing can be attributed to.
I think the point is that there aren’t just two black and white possibilities:
get extinct by developing superintelligence,
stop AI R&D.
I would argue that it is much more difficult to understand at which point the AI is too dangerous to be improved than to understand at which point YOUR AI is “better enough” than YOUR ENEMY’S AI (China or the US) to provide you some strategic advantage. Under this premise, if your adversary is rational, once you stop your AI R&D, he may realize that his optimal choice is to do just a bit more R&D and only then stop, in order to get some strategic advantages out of it, and still avoid the catastrophic risk of extinction.
If both adversaries consider the other two possibilities:
develop “just a bit more AI” to catch up with the other country,
develop “just a bit more AI” to get some strategic advantage over the other country...
they can push the Nash equilibrium into “get extinct by developing superintelligence”.
On the other hand, I’d say that in order to hope for some international cooperation you need to have some more nuance (maybe GPT-5.5 level) and model the fact that possibly both good and bad guys in their heart believe to be the good guys, don’t want to see the world burn, and may be happy to take a leap of faith and stop/regulate AI R&D in the same way as we did for weapons/pollution: not in pursuit of their selfish interest, but because it is good for the world.
Suppose that alignment was TRIVIAL. Then it would make PERFECT sense for Nvidia (and for Nvidia-controlled politicians?) to sell the GPUs to China, since the result would merely accelerate both sides’ progress and improve Nvidia’s cash flow.
As for China failing to slow down, I would like @Daniel Kokotajlo to comment on whether China not slowing down in the tabletop exercises was due to the player roleplaying as China acting too in-character or due to quasirational considerations like the ones with which one could “justify” betrayal in the Prisoner’s Dilemma.
To state the obvious:
Many of the same people who publicly make a great fuss of this don’t seem to care that Nvidia is selling GPUs to China by way of Singapore. Their purpose is to drag up China as an excuse for continuing what they are doing. If China has more GPUs, all the better for them in the future; China will be a greater threat that they can use to continue what they are doing.
Their model hasn’t chosen an explanation like “China will play the role of the movie bad guy at any cost to their own selfish interests”. They aren’t reasoning at GPT 3.5-level, only GPT-2 level. China is a scary word-vector, so they can argue that if the USA tries to slow down against extinction risk, then China won’t slow down, because that’s a scary thing so “China” is a word-vector that the scary thing can be attributed to.
I think the point is that there aren’t just two black and white possibilities:
get extinct by developing superintelligence,
stop AI R&D.
I would argue that it is much more difficult to understand at which point the AI is too dangerous to be improved than to understand at which point YOUR AI is “better enough” than YOUR ENEMY’S AI (China or the US) to provide you some strategic advantage. Under this premise, if your adversary is rational, once you stop your AI R&D, he may realize that his optimal choice is to do just a bit more R&D and only then stop, in order to get some strategic advantages out of it, and still avoid the catastrophic risk of extinction.
If both adversaries consider the other two possibilities:
develop “just a bit more AI” to catch up with the other country,
develop “just a bit more AI” to get some strategic advantage over the other country...
they can push the Nash equilibrium into “get extinct by developing superintelligence”.
On the other hand, I’d say that in order to hope for some international cooperation you need to have some more nuance (maybe GPT-5.5 level) and model the fact that possibly both good and bad guys in their heart believe to be the good guys, don’t want to see the world burn, and may be happy to take a leap of faith and stop/regulate AI R&D in the same way as we did for weapons/pollution: not in pursuit of their selfish interest, but because it is good for the world.
Suppose that alignment was TRIVIAL. Then it would make PERFECT sense for Nvidia (and for Nvidia-controlled politicians?) to sell the GPUs to China, since the result would merely accelerate both sides’ progress and improve Nvidia’s cash flow.
As for China failing to slow down, I would like @Daniel Kokotajlo to comment on whether China not slowing down in the tabletop exercises was due to the player roleplaying as China acting too in-character or due to quasirational considerations like the ones with which one could “justify” betrayal in the Prisoner’s Dilemma.