Google, Microsoft, and Facebook are all developing AI. So are Tencent, Sberbank, and the NSA. So are thousands of startups.
The most important thing is to solve the real problem—to design AI which, even when autonomous and smarter than human, has “friendly” values. (Just to be concrete, 22:30 here gives an example of what such a design could look like, albeit while still at a very theoretical and schematic stage.)
So you can try to speed up AI alignment research, and you can also try to slow down unaligned AI research. But there are thousands of independent research centers doing unaligned AI. OK, in the present, maybe there’s just a handful which are the ones at immediate risk of creating unaligned, autonomous, smarter-than-human AI. They’re mostly in huge powerful organizations, and there would be secret projects at the same level that you don’t know about—but OK, maybe you can make an impact somehow, by sabotaging TPU production, or by getting the UNSC to work with NeurIPS on a global regulatory regime, or…
You know, it’s going to be hard to stop these organizations from reaching for ultimate power. Putin already said a few years back, the nation that controls AI, controls the world. But also, the level of computing power and software knowledge required to become an immediate risk, gets lower over time. Maybe you can achieve a temporary tactical success through these preventive interventions, but for a primarily preventive strategy to work, you would have to change the entire direction of technological culture in this multipolar world. Also, the work that we already know about, from the big organizations we know about, is already alarmingly close to the point of no return. Proliferation will keep happening, but maybe we already know the name of the organization that will be the architect of our fate, we just don’t know which it shall be.
On the other hand: if you can solve the problem of friendliness, the problem of alignment, in practice and not just in theory (meaning you have enough computational resources to actually run your actually aligned AI program), then that’s it. Thus the focus on solving alignment rather than preventing nonalignment.
If we’re in a situation where it’s an open secret that a certain specific research area leads to general artificial intelligence, we’re doomed. If we get into a position where compute is the only limiting factor, we’re doomed. There’s no arguing there. The goal is to prevent us from getting into that situation.
As it stands now, certainly lots of companies are practicing machine learning. I have secondhand descriptions of a lot of the “really advanced” NSA programs and they fit that bill. Not a lot of organizations I know of however are actually pushing the clock hand forward on AGI and meta. Even fewer are doing that consistently, or getting over the massive intellectual hurdles that require a team like DeepMind’s. Completing AGI will probably be the result of a marriage of increased computing power, which we can’t really control, and insights pioneered and published by top labs, which I legitimately think we could to some degree by modifying their goals and talking to their members. OpenAI is a nonprofit. At the absolute bare minimum, none of these companies’ publish their meta research for money. The worst things they seem to do at this stage aren’t achieved when they’re reaching for power so much as playing an intellectual & status game amongst themselves, and fulfilling their science fiction protagonist syndromes.
I don’t doubt that it would be better for us to have AI alignment solved than to rely on these speculations about how AI will be engineered, but I do not see any good argument as to why it’s a bad strategy.
If we were “doomed” in this way, would you agree that the thing to do—for those who could do it—is to keep trying to solve the problem of alignment? i.e. trying to identify an AI design that could be autonomous, and smarter than human, and yet still safe?
Let me articulate my intuitions in a little bit more of a refined way: “If we ever get to a point where there are few secrets left, or that it’s common knowledge one can solve AGI with ~1000-10,000 million dollars, then delaying tactics probably wouldn’t work, because there’s nothing left for DeepMind to publish that speeds up the timeline.”
Inside those bounds, yes. I still think that people should keep working on alignment today, I just think other dumber people like me should try the delaying tactics I articulated in addition to funding alignment research.
Google, Microsoft, and Facebook are all developing AI. So are Tencent, Sberbank, and the NSA. So are thousands of startups.
The most important thing is to solve the real problem—to design AI which, even when autonomous and smarter than human, has “friendly” values. (Just to be concrete, 22:30 here gives an example of what such a design could look like, albeit while still at a very theoretical and schematic stage.)
So you can try to speed up AI alignment research, and you can also try to slow down unaligned AI research. But there are thousands of independent research centers doing unaligned AI. OK, in the present, maybe there’s just a handful which are the ones at immediate risk of creating unaligned, autonomous, smarter-than-human AI. They’re mostly in huge powerful organizations, and there would be secret projects at the same level that you don’t know about—but OK, maybe you can make an impact somehow, by sabotaging TPU production, or by getting the UNSC to work with NeurIPS on a global regulatory regime, or…
You know, it’s going to be hard to stop these organizations from reaching for ultimate power. Putin already said a few years back, the nation that controls AI, controls the world. But also, the level of computing power and software knowledge required to become an immediate risk, gets lower over time. Maybe you can achieve a temporary tactical success through these preventive interventions, but for a primarily preventive strategy to work, you would have to change the entire direction of technological culture in this multipolar world. Also, the work that we already know about, from the big organizations we know about, is already alarmingly close to the point of no return. Proliferation will keep happening, but maybe we already know the name of the organization that will be the architect of our fate, we just don’t know which it shall be.
On the other hand: if you can solve the problem of friendliness, the problem of alignment, in practice and not just in theory (meaning you have enough computational resources to actually run your actually aligned AI program), then that’s it. Thus the focus on solving alignment rather than preventing nonalignment.
If we’re in a situation where it’s an open secret that a certain specific research area leads to general artificial intelligence, we’re doomed. If we get into a position where compute is the only limiting factor, we’re doomed. There’s no arguing there. The goal is to prevent us from getting into that situation.
As it stands now, certainly lots of companies are practicing machine learning. I have secondhand descriptions of a lot of the “really advanced” NSA programs and they fit that bill. Not a lot of organizations I know of however are actually pushing the clock hand forward on AGI and meta. Even fewer are doing that consistently, or getting over the massive intellectual hurdles that require a team like DeepMind’s. Completing AGI will probably be the result of a marriage of increased computing power, which we can’t really control, and insights pioneered and published by top labs, which I legitimately think we could to some degree by modifying their goals and talking to their members. OpenAI is a nonprofit. At the absolute bare minimum, none of these companies’ publish their meta research for money. The worst things they seem to do at this stage aren’t achieved when they’re reaching for power so much as playing an intellectual & status game amongst themselves, and fulfilling their science fiction protagonist syndromes.
I don’t doubt that it would be better for us to have AI alignment solved than to rely on these speculations about how AI will be engineered, but I do not see any good argument as to why it’s a bad strategy.
If we were “doomed” in this way, would you agree that the thing to do—for those who could do it—is to keep trying to solve the problem of alignment? i.e. trying to identify an AI design that could be autonomous, and smarter than human, and yet still safe?
Let me articulate my intuitions in a little bit more of a refined way: “If we ever get to a point where there are few secrets left, or that it’s common knowledge one can solve AGI with ~1000-10,000 million dollars, then delaying tactics probably wouldn’t work, because there’s nothing left for DeepMind to publish that speeds up the timeline.”
Inside those bounds, yes. I still think that people should keep working on alignment today, I just think other dumber people like me should try the delaying tactics I articulated in addition to funding alignment research.