Here’s a doubt for you: I’m a nerd, I like nerds, I’ve worked on technology, and I’ve loved techie projects since I was a kid. Grew up on SF, all of that.
My problem lately is that I can’t take Friendly AI arguments seriously. I do think AI is possible, that we will invent it. I do think that at some point in the next hundreds of years, it will be game over for the human race. We will be replaced and/or transformed.
I kind of like the human race! And I’m forced to conclude that a human race without that tiny fraction of nerds could last a good long time yet (tens of thousands of years) and would change only slowly, through biological evolution. They would not do much technology, since it takes nerds (in the broadest sense) to do this. But, they would still have fulfilling, human, lives.
On the other hand, I don’t think a human race with nerds can forever avoid inventing a self-destructive technology like AI. So much as I have been brought up to think of politicians and generals as destroyers, and scientists and other nerds as creators, I have to admit that it’s the other way around, ultimately.
The non-nerds can’t destroy the human race. Only we nerds can do that.
That’s my particular crisis of faith. Care to take a side?
I would say that the non-nerds can’t save the human race either though. Without nerds our population never exceeds what can be supported by hunting, gathering, and maybe some primitive agriculture.
Which isn’t much. We’d be constantly hovering just short of being wiped out by some global cataclysm. And there’s some evidence that we’ve narrowly missed just that at least once in our history. If we want to survive long-term we need to get off this rock, and then we need to find at least one other solar system. After that we can take a breather while we think about finding another galaxy to colonize.
Yes, we might destroy ourselves with new technology. But we’re definitely dead without it. And if you look at how many new technologies have been denounced as being harbingers for the end of the world vs how many times the world has actually ended, I’d have to think that gut feelings about what technologies are the most dangerous and how badly we’ll handle them are probably wrong more often than they’re right.
If you can’t imagine ways in which the human race can be destroyed by non-nerds then that shows a lack of imagination not that it can not be done. Also, it isn’t like nerds and non-nerds are actually a different species, people that do not have a natural aptitude for a subject are still capable of learning the subject,. If nerds all moved to nerdtopia other people would study what material there was on the subject and attempt to continue on. If this is not possible then you have applied the term nerd to be too broad such that it contains the majority of people and all that would be left are people that are incapable to fully taking care of themselves without some form of outside assistance and would thus destroy the human race by sheer ineptitude at basic survival skills.
The vast majority of people is both incapable of and uninterested in creating new technology OR doing science (and their incapability supports their lack of interest). So, if nerds move to nerdtopia taking some already-deadly technologies with them, the remaining world will never create something AI-like… well, given that newborns with nerds’ skills are taken away early. People are generally stupid—not only in the sense of exhibiting specific biases discussed by Eliezer but also in the sense of lack of both curiosity and larger-than-three working memory (or larger-than-120 IQ, whitherever you prefer) in the majority (and larger-than-two/larger-than-100 in a big group). Having intelligence—IQ above roughly 120 or any isomorphic measure—is something so rare that from standard p<0.05 view it’s inexistent (Bell’s curve, 100 as mean, 10 as sigma).
Here’s a doubt for you: I’m a nerd, I like nerds, I’ve worked on technology, and I’ve loved techie projects since I was a kid. Grew up on SF, all of that.
My problem lately is that I can’t take Friendly AI arguments seriously. I do think AI is possible, that we will invent it. I do think that at some point in the next hundreds of years, it will be game over for the human race. We will be replaced and/or transformed.
I kind of like the human race! And I’m forced to conclude that a human race without that tiny fraction of nerds could last a good long time yet (tens of thousands of years) and would change only slowly, through biological evolution. They would not do much technology, since it takes nerds (in the broadest sense) to do this. But, they would still have fulfilling, human, lives.
On the other hand, I don’t think a human race with nerds can forever avoid inventing a self-destructive technology like AI. So much as I have been brought up to think of politicians and generals as destroyers, and scientists and other nerds as creators, I have to admit that it’s the other way around, ultimately.
The non-nerds can’t destroy the human race. Only we nerds can do that.
That’s my particular crisis of faith. Care to take a side?
I would say that the non-nerds can’t save the human race either though. Without nerds our population never exceeds what can be supported by hunting, gathering, and maybe some primitive agriculture.
Which isn’t much. We’d be constantly hovering just short of being wiped out by some global cataclysm. And there’s some evidence that we’ve narrowly missed just that at least once in our history. If we want to survive long-term we need to get off this rock, and then we need to find at least one other solar system. After that we can take a breather while we think about finding another galaxy to colonize.
Yes, we might destroy ourselves with new technology. But we’re definitely dead without it. And if you look at how many new technologies have been denounced as being harbingers for the end of the world vs how many times the world has actually ended, I’d have to think that gut feelings about what technologies are the most dangerous and how badly we’ll handle them are probably wrong more often than they’re right.
Have you ever heard of the term hubris?
If you can’t imagine ways in which the human race can be destroyed by non-nerds then that shows a lack of imagination not that it can not be done. Also, it isn’t like nerds and non-nerds are actually a different species, people that do not have a natural aptitude for a subject are still capable of learning the subject,. If nerds all moved to nerdtopia other people would study what material there was on the subject and attempt to continue on. If this is not possible then you have applied the term nerd to be too broad such that it contains the majority of people and all that would be left are people that are incapable to fully taking care of themselves without some form of outside assistance and would thus destroy the human race by sheer ineptitude at basic survival skills.
The vast majority of people is both incapable of and uninterested in creating new technology OR doing science (and their incapability supports their lack of interest). So, if nerds move to nerdtopia taking some already-deadly technologies with them, the remaining world will never create something AI-like… well, given that newborns with nerds’ skills are taken away early. People are generally stupid—not only in the sense of exhibiting specific biases discussed by Eliezer but also in the sense of lack of both curiosity and larger-than-three working memory (or larger-than-120 IQ, whitherever you prefer) in the majority (and larger-than-two/larger-than-100 in a big group). Having intelligence—IQ above roughly 120 or any isomorphic measure—is something so rare that from standard p<0.05 view it’s inexistent (Bell’s curve, 100 as mean, 10 as sigma).