Yes, nevertheless the S-risk and X-risk problems don’t go away. There are humans who like causing suffering. There are human advocating for human extinction (and some of them might act on that given the capabilities). There are humans who are ready to fight wars with weapons which might cause extinction, or would be ready to undertake projects which might cause extinction or widespread suffering.
Stepping back, we know that Eliezer was very much against anthropomorphic superintelligences in 2011. He thought we needed much higher levels of safety (“provably friendly AI”, which would not be possible with something as messy as human-like systems). Since then he strongly updated towards pessimism regarding our chances to create beneficial artificial superintelligence, and he arrived at the conclusion that our chances with biological superintelligence might be higher.
But it would be good to try to articulate what are the reasons for our chances with biological superintelligence to be higher.
One aspect is that we do have an intuition that biology-based systems are likely to self-improve slower, and thus would have more time to ponder solutions to various issues as they get smarter. So they might be not superintelligent, but just very smart for quite a while, and during that period they would decide what to do next. Another aspect is that biology-based systems are more likely to be automatically sentient, and their sentience is more likely to be at least somewhat similar to ours, and so even if things go badly initially, the chances for having a lot of value in the future lightcone are higher, because it is more likely that there would be first-person experiencers.
But it would be good to pause and think whether we are sure. Also speaking of
Brain implants/brain-computer interfaces: Devices under development by companies such as Neuralink, Kernel, Openwater, and Meta’s Reality Labs. Could hypothetically enhance human intelligence.
these devices can also lead to the hybrid human-AI systems, and that might be a more technologically likely route. The hybrid system becomes smarter, both because of its biological part working better, but also because of a tight coupling with an AI thinking part. In some cases of BCI use, it might be difficult to distinguish between straight human intelligence enhancement and creation of a hybrid thinker. We might want to ponder whether this is a desirable route. (I personally find this route very attractive for a number of reasons, but safety issues along this route are quite acute as well).
Potential solutions to foreseeable problems with biological superintelligence include: a) only upgrading particularly moral and trustworthy humans or b) ensuring that upgrading is widely accessible, so that lots of people can do it.
b) does not solve it without a lot of successful work on multipolar safety (it’s almost an equivalent of giving nuclear weapons to lots of people, making them widely accessible; and yes, giving gain-of-function labs equipment too)
a) is indeed very reasonable, but we should keep in mind that upgrade is a potentially stronger impact than any psychoactive drugs, a potentially stronger impact than any most radical psychedelic experiences. Here the usual “AI alignment problem” one is normally dealing with is replaced by the problem of conservation of one’s values and character.
In fact these problems are closely related. The most intractable part of AI safety is what happens when AI ecosystems starts to rapidly recursively self-improve, perhaps with significant acceleration. We might have current members of AI ecosystem behave in a reasonably safe and beneficial way, but would future members (or same members after they self-improve) behave safely, or would “a sharp left turn” happen?
Here it is the same problem for a rapidly improving and changing “enhanced human”, would that person continue to maintain the original character and values while undergoing radical changes and enhancements, or would drastic new realizations (potentially more radical than any psychedelic revelations) lead to unpredictable revisions of that original character and values?
It might be the case that it’s easier to smooth these changes for a human (compared to AI), but the success is not automatic by any means.
I guess we could say governance remains a problem with biological superintelligence? As it does with normal humans, just more so.
Yes, nevertheless the S-risk and X-risk problems don’t go away. There are humans who like causing suffering. There are human advocating for human extinction (and some of them might act on that given the capabilities). There are humans who are ready to fight wars with weapons which might cause extinction, or would be ready to undertake projects which might cause extinction or widespread suffering.
Stepping back, we know that Eliezer was very much against anthropomorphic superintelligences in 2011. He thought we needed much higher levels of safety (“provably friendly AI”, which would not be possible with something as messy as human-like systems). Since then he strongly updated towards pessimism regarding our chances to create beneficial artificial superintelligence, and he arrived at the conclusion that our chances with biological superintelligence might be higher.
But it would be good to try to articulate what are the reasons for our chances with biological superintelligence to be higher.
One aspect is that we do have an intuition that biology-based systems are likely to self-improve slower, and thus would have more time to ponder solutions to various issues as they get smarter. So they might be not superintelligent, but just very smart for quite a while, and during that period they would decide what to do next. Another aspect is that biology-based systems are more likely to be automatically sentient, and their sentience is more likely to be at least somewhat similar to ours, and so even if things go badly initially, the chances for having a lot of value in the future lightcone are higher, because it is more likely that there would be first-person experiencers.
But it would be good to pause and think whether we are sure. Also speaking of
these devices can also lead to the hybrid human-AI systems, and that might be a more technologically likely route. The hybrid system becomes smarter, both because of its biological part working better, but also because of a tight coupling with an AI thinking part. In some cases of BCI use, it might be difficult to distinguish between straight human intelligence enhancement and creation of a hybrid thinker. We might want to ponder whether this is a desirable route. (I personally find this route very attractive for a number of reasons, but safety issues along this route are quite acute as well).
Potential solutions to foreseeable problems with biological superintelligence include: a) only upgrading particularly moral and trustworthy humans or b) ensuring that upgrading is widely accessible, so that lots of people can do it.
b) does not solve it without a lot of successful work on multipolar safety (it’s almost an equivalent of giving nuclear weapons to lots of people, making them widely accessible; and yes, giving gain-of-function labs equipment too)
a) is indeed very reasonable, but we should keep in mind that upgrade is a potentially stronger impact than any psychoactive drugs, a potentially stronger impact than any most radical psychedelic experiences. Here the usual “AI alignment problem” one is normally dealing with is replaced by the problem of conservation of one’s values and character.
In fact these problems are closely related. The most intractable part of AI safety is what happens when AI ecosystems starts to rapidly recursively self-improve, perhaps with significant acceleration. We might have current members of AI ecosystem behave in a reasonably safe and beneficial way, but would future members (or same members after they self-improve) behave safely, or would “a sharp left turn” happen?
Here it is the same problem for a rapidly improving and changing “enhanced human”, would that person continue to maintain the original character and values while undergoing radical changes and enhancements, or would drastic new realizations (potentially more radical than any psychedelic revelations) lead to unpredictable revisions of that original character and values?
It might be the case that it’s easier to smooth these changes for a human (compared to AI), but the success is not automatic by any means.