Talking about alternate realistic[1] scenarios, Zvi mentioned in his post that “Nvidia even outright advocates that it should be allowed to sell to China openly, and no one in Washington seems to hold them accountable for this.”
Were Washington to let NVIDIA sell chips to China, the latter would receive far more compute, which would likely end up in DeepCent’s hands. Then the slowdown might cause the aligned AI created in OpenBrain to be weaker than the misaligned AI created in DeepCent. What would the two AIs do?
I think that unrealistic scenarios like destruction of Taiwan and South Korea due to the nuclear war between India and Pakistan in May 2025 can also provide useful insights. For example, if we make the erroneous assumption that the total compute in the USA stops increasing and the compute in China increases linearly, while the AI takeoff potential per compute stays the same, then by May 2030 OpenBrain and DeepCent will have created misaligned AGIs and be unable to slow down and reassess.
Talking about alternate realistic[1] scenarios, Zvi mentioned in his post that “Nvidia even outright advocates that it should be allowed to sell to China openly, and no one in Washington seems to hold them accountable for this.”
Were Washington to let NVIDIA sell chips to China, the latter would receive far more compute, which would likely end up in DeepCent’s hands. Then the slowdown might cause the aligned AI created in OpenBrain to be weaker than the misaligned AI created in DeepCent. What would the two AIs do?
I think that unrealistic scenarios like destruction of Taiwan and South Korea due to the nuclear war between India and Pakistan in May 2025 can also provide useful insights. For example, if we make the erroneous assumption that the total compute in the USA stops increasing and the compute in China increases linearly, while the AI takeoff potential per compute stays the same, then by May 2030 OpenBrain and DeepCent will have created misaligned AGIs and be unable to slow down and reassess.