We believe AI is pivotal because we think it’s going to surpass human intelligence soon. So it’s not just another technology, it’s our successor.
The original plan of MIRI, the AI research institute somewhat associated with this website, was to identify a value system and a software architecture for AI, that would still be human-friendly, even after it bootstrapped itself to a level completely beyond human control or understanding, becoming the metaphorical “operating system” in charge of all life on Earth.
More recently, given the rapidity of advances in the raw power of AI, they have decided that there just isn’t time to solve these design problems, before some AI lab somewhere unwittingly hatches a superintelligent AI system that steamrolls the human race, not out of malice, but simply because it has goals that aren’t sufficiently finetuned to respect human life, liberty, or happiness.
Instead, their current aim is to buy time for humanity, by using early superintelligent AI, to neutralize all other dangerous AI projects, and establish a temporary regime in which civilization can deliberate on what to do with the incredible promise and peril of AI and related technologies.
There is therefore some similarity with your own idea to slow things down, but in this scenario, it is to be done by force, and by using the dangerous technology of superintelligent AI, when it first appears. Continuing the operating system metaphor, this amounts to putting AI-augmented civilization into a “safe mode” before it can do anything too destructive.
This suggests a model of the future, in which there is a kind of temporary world government, equipped with a superintelligent AI that monitors everything everywhere, and which steps in to sabotage any unapproved technology that threatens to create unfriendly superintelligence. Ideally, this period lasts as long as it takes, for humanity’s wise ones to figure out how to make fully autonomous superintelligence, something that we can safely coexist with. At that point the temporary world government can be permanently replaced with that self-governing planetary operating system.
You may be wondering, why rely on AI to restrain AI? Why not just have e.g. the UN Security Council declare that AI research worldwide will be frozen indefinitely, and use the existing tools of human governance to enforce that? The problem is that technological culture is decentralized and self-enhancing. In the short term, we might throttle the development of deep learning AI by restricting access to TPU chips worldwide. But you can also run the algorithms on sufficiently large networks of ordinary computers. And ultimately, you even have to worry about things like superintelligence achieved via neuron-hacking, polymeric nanocomputers, and so forth.
The premise is that the world is too out of control to stop everyone in the entire world from ever crossing the dangerous threshold. So instead, one must work towards an outcome whereby, the first ones across the threshold will use that power to slow things down for everyone else, while responsibly trying to figure out how to safely integrate that power into our world.
OK, that’s a glimpse of how some people are thinking. AI is seen as the crux of everything, because it is at the hub of everything: it can control other technologies, it can discover new technologies, it can even replace us as the chief decision-making entity in the world. And it’s really “AGI” (artificial general intelligence), and especially AGI that is more intelligent than human, which is the focus of all this concern, “Narrow AI” that just drives cars or recognizes faces has its own safety issues, but isn’t as all-encompassing in its implications.