Very few people have taken us up on our offer to create scenarios so far; yours is currently the frontrunner in terms of overall quality. However it doesn’t meet our bar, mainly because it ends too soon—OK, so we don’t get AGI in 2027 due to the chip production slowdown, but what about 2028 and 2029? I’d love it if you continued the scenario giving your best guess of when AGI would arrive and what that would look like and how it would end for humanity.
I have already made a comment where I wrote the collapsible section “How the nuclear conflict would affect the AI race”. I think that the section there is a similarly oversimplified[1] PoC for the event that destroying Taiwan causes OpenBrain&FormerRivals and DeepCent&FormerRivals to arrive in 2030 or earlier at similarly poorly aligned models without the potential to slow down and reassess. If one side chooses to do so, and the other doesn’t, then mankind gets a powerful misaligned AI and a weak aligned one; I tried to explore the results here[2] and another person expressed similar concerns here.
P.S. It is rather funny to watch people express concerns which I have already explored and even posted my takes on this very forum…
For example, I made the oversimplified assumption that the US won’t produce a single chip, while China will produce the chips linearly, not exponentially; back then I estimated that DeepCent’s compute after China’s awakening would increase at the rate of 1.4E27 operations/month/year. ChatGPT estimates that after Taiwan is destroyed, the USA and China will produce compute at respective rates 0.1-0.2E18 operations/second/month and 0.05-0.15E18 operations/second/month. The two estimates of compute in the USA and China overlap; ChatGPT estimates the probability that China will have at least twice more compute as 5-15%. However, I suspect that it might be biased towards overestimating American capabilities.
I also made the assumption that it will be the American AI who will end up misaligned; this is possible if the AI-2027 forecast is read by DeepCent’s researchers and leaders. Another problem is that the USA would have a reason to race even if DeepCent didn’t exist, since, as I have already remarked, America is in trouble.
Very few people have taken us up on our offer to create scenarios so far; yours is currently the frontrunner in terms of overall quality. However it doesn’t meet our bar, mainly because it ends too soon—OK, so we don’t get AGI in 2027 due to the chip production slowdown, but what about 2028 and 2029? I’d love it if you continued the scenario giving your best guess of when AGI would arrive and what that would look like and how it would end for humanity.
I have already made a comment where I wrote the collapsible section “How the nuclear conflict would affect the AI race”. I think that the section there is a similarly oversimplified[1] PoC for the event that destroying Taiwan causes OpenBrain&FormerRivals and DeepCent&FormerRivals to arrive in 2030 or earlier at similarly poorly aligned models without the potential to slow down and reassess. If one side chooses to do so, and the other doesn’t, then mankind gets a powerful misaligned AI and a weak aligned one; I tried to explore the results here[2] and another person expressed similar concerns here.
P.S. It is rather funny to watch people express concerns which I have already explored and even posted my takes on this very forum…
For example, I made the oversimplified assumption that the US won’t produce a single chip, while China will produce the chips linearly, not exponentially; back then I estimated that DeepCent’s compute after China’s awakening would increase at the rate of 1.4E27 operations/month/year. ChatGPT estimates that after Taiwan is destroyed, the USA and China will produce compute at respective rates 0.1-0.2E18 operations/second/month and 0.05-0.15E18 operations/second/month. The two estimates of compute in the USA and China overlap; ChatGPT estimates the probability that China will have at least twice more compute as 5-15%. However, I suspect that it might be biased towards overestimating American capabilities.
I also made the assumption that it will be the American AI who will end up misaligned; this is possible if the AI-2027 forecast is read by DeepCent’s researchers and leaders. Another problem is that the USA would have a reason to race even if DeepCent didn’t exist, since, as I have already remarked, America is in trouble.