I understand that there are many ways in which nanotechnology could be dangerous, even to the point of posing extinction risks, but I do not understand why these risks seem inevitable. I would find it much more likely that humanity will invent some nanotech device that gets out of hand, poisons a water supply, kills several thousand people, and needs to be contained/quarantined, leading to massive nano-tech development regulation, rather than a nano-tech mistake that immediately depressurizes the whole space suit, is impossible to contain, and kills us all.
A recursively improving, superintelligent AI, on the other hand, seems much more likely to fuck us over, especially if we’re convinced it’s acting in our best interest for the beginning of its ‘life,’ and problems only become obvious after it’s already become far more ‘intelligent’ than we are.
I understand that there are many ways in which nanotechnology could be dangerous, even to the point of posing extinction risks, but I do not understand why these risks seem inevitable. I would find it much more likely that humanity will invent some nanotech device that gets out of hand, poisons a water supply, kills several thousand people, and needs to be contained/quarantined, leading to massive nano-tech development regulation, rather than a nano-tech mistake that immediately depressurizes the whole space suit, is impossible to contain, and kills us all.
A recursively improving, superintelligent AI, on the other hand, seems much more likely to fuck us over, especially if we’re convinced it’s acting in our best interest for the beginning of its ‘life,’ and problems only become obvious after it’s already become far more ‘intelligent’ than we are.