Agreed we’re in the successful group, easily. Sucks that NK exists as it is, but they are isolated and have no way to expand or threaten the world system. The fact that we can have websites like this, and people actually can sit in the basement and work on the math means we are doing well, up to this point at least.
My guess is more of the failed earths were long ago. Like humans continue to wander around in small tribes for millions of years and never get beyond stone-age technology. We are eventually wiped out by a meteor or some such.
Many people here seem to think increasing intelligence will help us solve our problems. History indicates that increasing intelligence will increase the number and severity of our problems.
Phil, I’m sorry if this sounds negative, but I don’t understand this attitude at all. Intelligence is how accurately you can mine the information of the past in order to predict the future. You can’t possibly think that all of our problems would go away if you gave everybody in the world a lobotomy? Or is there just some preferable lower-limit to intelligence we should engineer people to?
I think the historical problems with intelligence is an uneven increase in different fields or typical misuse. This isn’t a problem with the tool, but the protocols and practices surrounding it.
This isn’t a problem with the tool, but the protocols and practices surrounding it.
Joe, I couldn’t discern any verifiable meaning in this sentence. Both the tool and the protocols/practices contribute to the problem.
You can’t possibly think that all of our problems would go away if you gave everybody in the world a lobotomy?
No more nukes. No more surveillance states. No more military/police robots. No more AI threat, nanotech threat, biotech threat. A lobotomy for everyone would increase our chances of surviving the century as a species.
I guess most people would object to an intelligence decrease because they feel it would make them worse off. In a world where growth of intelligence and knowledge leads to multiple high-probability scenarios of global collapse, this stance looks eerily equivalent to defecting in the Prisoner’s Dilemma. I wonder what Eliezer, an outspoken proponent of cooperating, would say about global lobotomies viewed in this light.
Intelligence is how accurately you can mine the information of the past in order to predict the future.
Very few people would claim that is an adequate definition of intelligence. The fact that there is a substantial clique of such people here, at this site, does not constitute a validation of the definition.
Well, let’s look at the short list google gives us:
the ability to comprehend; to understand and profit from experience
Capacity of mind, especially to understand principles, truths, facts or meanings, acquire knowledge, and apply it to practice; the ability to learn and comprehend.
Intelligence is an umbrella term used to describe a property of the mind that encompasses many related abilities, such as the capacities to reason, to plan, to solve problems, to think abstractly, to comprehend ideas, to use language, and to learn.
My definition was:
To mine information from the past in order to predict the future.
My definition is certainly more broad, but I think it breaks down to the same thing. In order to do anything in the list you have to use resources of past experience in order to apply information for future experience. I’ve been using that definition roughly since ‘Consciousness Explained’ came out, and I’ve never been in a discussion with somebody who felt it was a ridiculous way of describing the functional nature of intelligence especially in terms of biological utility. Do you just dislike summation and want a long, drawn out definition?
Either way, no matter which definition you use, the capabilities granted by extra intelligence do not generate extra problems, but help you mitigate the problems that come up. I don’t really understand the criticism in context with the discussion.
Joe, what you are forgetting is that human beings are not governed solely by their intelligence. I think Annoyance was referring to a sort of moral intelligence which isn’t in your definition.
And intelligence doesn’t generate extra problems? Ever hear of the Cuban Missile Crisis? Or, say, pollution?
I think those problems weren’t caused by too much intelligence, but by too little. I know, intelligence enables these problems to form in the first place—These entities wouldn’t be making the problems if they weren’t volitional agents with intelligence, but that seems like a kind of cop-out complaint—Without intelligence there wouldn’t be any problems, sure, but there also wouldn’t be anything positive either, no concepts whatsoever.
Pollution is a great example: It’s intelligent thought that allowed us to start making machines that polluted. Intelligence allowed us to realize we could capitalize on the well-being of the environment and save money by trashing it.
More intelligence realizes that this is still a value trade off, that you aren’t getting something for nothing—Depending on the rate at which you do this, you could seriously damage yourself and the people around you for the trade-off. You have to weigh the costs with the benefits, and if the benefit is ‘some money’ and the cost is ‘destroying the world’, the intelligent choice becomes clear. To continue to act for the money isn’t intelligence, it’s just insanity, overpowering greed.
The cuban missile crisis may have been caused by intelligence building the structures that led up to it, but the solution wasn’t making everyone dumber so they couldn’t build that kind of thing—that just reduces overall utility. The solution is to act intelligently in ways that don’t destroy the world.
I see your point about moral intelligence being considered separately though, I hadn’t thought of that in the context. It’s a more elegant package to wrap everything up together, but not always the right thing to do… Thanks for the reply.
So you agree that yes, intelligence is continually generating “extra problems” for us to deal with. As you point out, many of the most pressing problems in the modern world are unforeseen consequences of useful technologies. You just believe that increases in human intelligence will invariably outpace the destructive power of the problems, whereas I don’t.
The premise of this diary was many earths, so I’d submit that certainly there are many earths for which the problem of nuclear warfare outpaced humanity’s capacity to intelligently deal with it, and that in the end we could very well share their fate.
I’ll also note that I fail to see how anyone could conclude from what I’ve written above that my prescription for humanity is stupid pills.
I’m glad we’ve hashed this out. I think that bias about the messianic/apocalyptic role of technology has largely been overlooked on this site, so I was glad to see this entry of Eliezer’s.
Regardless of whether or not they’re true I tend to think that arguments about the arc of history etc are profoundly counterproductive. People won’t vote if they think it’s a landslide, either for their guy or against. And I suspect I differ from others on this site in this respect, but I find it hard to get ginned up about cosmic endeavors, simply because they seem so remote from my experience.
And I don’t think we need an alternative! What I was trying to point out from the start was that increasing our predictive ability is necessary but not sufficient to save the world. Entirely selfish, entirely rational actors will doom the planet if we let them.
If everyone’s intelligence were increased, and society stayed the same, and society’s problems stayed the same, that higher intelligence would do better at solving today’s problems than our current intelligence.
But society wouldn’t stay the same, and society’s problems wouldn’t stay the same. People would come up with more complicated organizations, more complicated laws, more complicated financial interactions (credit default swaps), more complicated social roles, et cetera.
We can only hope there is a level of intelligence at which people can appreciate this effect, and design societies that will avoid black swan crashes even at the cost of everyone’s expected personal gain.
Agreed we’re in the successful group, easily. Sucks that NK exists as it is, but they are isolated and have no way to expand or threaten the world system. The fact that we can have websites like this, and people actually can sit in the basement and work on the math means we are doing well, up to this point at least.
My guess is more of the failed earths were long ago. Like humans continue to wander around in small tribes for millions of years and never get beyond stone-age technology. We are eventually wiped out by a meteor or some such.
My guess is that there is a bottleneck still ahead of us.
Evolution ⇒ intelligence ⇒ complexity ⇒ instability ⇒ collapse
Many people here seem to think increasing intelligence will help us solve our problems. History indicates that increasing intelligence will increase the number and severity of our problems.
Phil, I’m sorry if this sounds negative, but I don’t understand this attitude at all. Intelligence is how accurately you can mine the information of the past in order to predict the future. You can’t possibly think that all of our problems would go away if you gave everybody in the world a lobotomy? Or is there just some preferable lower-limit to intelligence we should engineer people to?
I think the historical problems with intelligence is an uneven increase in different fields or typical misuse. This isn’t a problem with the tool, but the protocols and practices surrounding it.
Devil’s advocate mode!
Joe, I couldn’t discern any verifiable meaning in this sentence. Both the tool and the protocols/practices contribute to the problem.
No more nukes. No more surveillance states. No more military/police robots. No more AI threat, nanotech threat, biotech threat. A lobotomy for everyone would increase our chances of surviving the century as a species.
I guess most people would object to an intelligence decrease because they feel it would make them worse off. In a world where growth of intelligence and knowledge leads to multiple high-probability scenarios of global collapse, this stance looks eerily equivalent to defecting in the Prisoner’s Dilemma. I wonder what Eliezer, an outspoken proponent of cooperating, would say about global lobotomies viewed in this light.
Very few people would claim that is an adequate definition of intelligence. The fact that there is a substantial clique of such people here, at this site, does not constitute a validation of the definition.
Well, let’s look at the short list google gives us:
the ability to comprehend; to understand and profit from experience
Capacity of mind, especially to understand principles, truths, facts or meanings, acquire knowledge, and apply it to practice; the ability to learn and comprehend.
Intelligence is an umbrella term used to describe a property of the mind that encompasses many related abilities, such as the capacities to reason, to plan, to solve problems, to think abstractly, to comprehend ideas, to use language, and to learn.
My definition was:
To mine information from the past in order to predict the future.
My definition is certainly more broad, but I think it breaks down to the same thing. In order to do anything in the list you have to use resources of past experience in order to apply information for future experience. I’ve been using that definition roughly since ‘Consciousness Explained’ came out, and I’ve never been in a discussion with somebody who felt it was a ridiculous way of describing the functional nature of intelligence especially in terms of biological utility. Do you just dislike summation and want a long, drawn out definition?
Either way, no matter which definition you use, the capabilities granted by extra intelligence do not generate extra problems, but help you mitigate the problems that come up. I don’t really understand the criticism in context with the discussion.
Joe, what you are forgetting is that human beings are not governed solely by their intelligence. I think Annoyance was referring to a sort of moral intelligence which isn’t in your definition.
And intelligence doesn’t generate extra problems? Ever hear of the Cuban Missile Crisis? Or, say, pollution?
I think those problems weren’t caused by too much intelligence, but by too little. I know, intelligence enables these problems to form in the first place—These entities wouldn’t be making the problems if they weren’t volitional agents with intelligence, but that seems like a kind of cop-out complaint—Without intelligence there wouldn’t be any problems, sure, but there also wouldn’t be anything positive either, no concepts whatsoever.
Pollution is a great example: It’s intelligent thought that allowed us to start making machines that polluted. Intelligence allowed us to realize we could capitalize on the well-being of the environment and save money by trashing it.
More intelligence realizes that this is still a value trade off, that you aren’t getting something for nothing—Depending on the rate at which you do this, you could seriously damage yourself and the people around you for the trade-off. You have to weigh the costs with the benefits, and if the benefit is ‘some money’ and the cost is ‘destroying the world’, the intelligent choice becomes clear. To continue to act for the money isn’t intelligence, it’s just insanity, overpowering greed.
The cuban missile crisis may have been caused by intelligence building the structures that led up to it, but the solution wasn’t making everyone dumber so they couldn’t build that kind of thing—that just reduces overall utility. The solution is to act intelligently in ways that don’t destroy the world.
I see your point about moral intelligence being considered separately though, I hadn’t thought of that in the context. It’s a more elegant package to wrap everything up together, but not always the right thing to do… Thanks for the reply.
So you agree that yes, intelligence is continually generating “extra problems” for us to deal with. As you point out, many of the most pressing problems in the modern world are unforeseen consequences of useful technologies. You just believe that increases in human intelligence will invariably outpace the destructive power of the problems, whereas I don’t.
The premise of this diary was many earths, so I’d submit that certainly there are many earths for which the problem of nuclear warfare outpaced humanity’s capacity to intelligently deal with it, and that in the end we could very well share their fate.
I’ll also note that I fail to see how anyone could conclude from what I’ve written above that my prescription for humanity is stupid pills.
I agree completely. If intelligence-generated problems cannot outpace the solutions total destruction awaits.
I apologize if the stupid pill characterization feels wrong, I just was trying to think of a viable alternative to increasing intelligence.
I’m glad we’ve hashed this out. I think that bias about the messianic/apocalyptic role of technology has largely been overlooked on this site, so I was glad to see this entry of Eliezer’s.
Regardless of whether or not they’re true I tend to think that arguments about the arc of history etc are profoundly counterproductive. People won’t vote if they think it’s a landslide, either for their guy or against. And I suspect I differ from others on this site in this respect, but I find it hard to get ginned up about cosmic endeavors, simply because they seem so remote from my experience.
And I don’t think we need an alternative! What I was trying to point out from the start was that increasing our predictive ability is necessary but not sufficient to save the world. Entirely selfish, entirely rational actors will doom the planet if we let them.
It’s not an “attitude”.
If everyone’s intelligence were increased, and society stayed the same, and society’s problems stayed the same, that higher intelligence would do better at solving today’s problems than our current intelligence.
But society wouldn’t stay the same, and society’s problems wouldn’t stay the same. People would come up with more complicated organizations, more complicated laws, more complicated financial interactions (credit default swaps), more complicated social roles, et cetera.
We can only hope there is a level of intelligence at which people can appreciate this effect, and design societies that will avoid black swan crashes even at the cost of everyone’s expected personal gain.
Well, except for the nukes.