I think that if an extremely efficient ASI algorithm is discovered and disseminated to the public, it will almost certainly be too late to prevent our extinction (or some other fate just as bad) unless perhaps the discovery and dissemination is done by someone competent like the leaders of MIRI, Steven Byrnes or John Wentworth. (Note that the latter 2 have never worked for an AI lab.)
I was trying to understand: Do you have much more hope about the situation if an extremely efficient ASI algorithm is discovered and not immediately disseminated to the public, but where the Chinese and US government both have access to it and can run it without the others’ knowledge; and if so why? That seems like an essential piece of info to understand the cost/benefit of moving from the high-transparency world to the world where USG and China are overseeing each others’ tech but aren’t publishing.
(My two parantheses were me speculating about possible answers to this question.)
For me the main benefit of restrictions on publication and other forms of dissemination of breakthroughs, discoveries, insights and plain old technical information about AI is to delay the creation of an ASI. If the ASI is a very efficient learning algorithm, then it would definitely be better if only Washington and Beijing have it and are effectively preventing its dissemination, but it would still be very bad news: IMHO human extinction would still follow within 3 years with p = .95.
Any delay in the arrival of ASI gives humanity more time for someone to come up with some miracle to get us out of our dire situation. The nature of that miracle I probably cannot guess. If I had to guess, I might guess that space aliens will show up and save us from extinction while imposing some of their values on us, values that we would consider bizarre, or something vaguely like that.
I think that if an extremely efficient ASI algorithm is discovered and disseminated to the public, it will almost certainly be too late to prevent our extinction (or some other fate just as bad) unless perhaps the discovery and dissemination is done by someone competent like the leaders of MIRI, Steven Byrnes or John Wentworth. (Note that the latter 2 have never worked for an AI lab.)
I was trying to understand: Do you have much more hope about the situation if an extremely efficient ASI algorithm is discovered and not immediately disseminated to the public, but where the Chinese and US government both have access to it and can run it without the others’ knowledge; and if so why? That seems like an essential piece of info to understand the cost/benefit of moving from the high-transparency world to the world where USG and China are overseeing each others’ tech but aren’t publishing.
(My two parantheses were me speculating about possible answers to this question.)
For me the main benefit of restrictions on publication and other forms of dissemination of breakthroughs, discoveries, insights and plain old technical information about AI is to delay the creation of an ASI. If the ASI is a very efficient learning algorithm, then it would definitely be better if only Washington and Beijing have it and are effectively preventing its dissemination, but it would still be very bad news: IMHO human extinction would still follow within 3 years with p = .95.
Any delay in the arrival of ASI gives humanity more time for someone to come up with some miracle to get us out of our dire situation. The nature of that miracle I probably cannot guess. If I had to guess, I might guess that space aliens will show up and save us from extinction while imposing some of their values on us, values that we would consider bizarre, or something vaguely like that.