But even if it wants to help, can it? Is intelligence really such a powerful tool? I picture it as we are aiming an arrow at something, like building a Mars base, and it makes our arrow more accurate. But if it was accurate enough to do the job how much does that help? How does intelligence transform into power to generate outcomes?
I mean the issue is that EY defines pretty much as efficient goal-achievement or cross-domain optimization, so by using that definition intelligence is trivially about a power to generate outcomes, but this is simply not the common definition of intelligence. It is not what IQ tests measure. It is not what Mensa members are good at. It is not what Kasparov is good at. That is more of a puzzle-solving ability.
I mean, could efficient goal-achievement equal puzzle-solving? I would be really surprised if someone could prove that. In the real life the most efficient goal achievers I know are often stupid and have other virtues, like a bulldog like never, ever, ever, ever give up type attitude.
Might I ask, what exactly do you mean? Are you saying that the super intelligent AI would not be able to contribute that much to our intellectual endeavours? Or are you saying that its intelligence may not translate to achieving goals like ‘become a billionaire’ or ‘solve world hunger’? Or something else entirely?
I don’t yet know enough about AI to even attempt to answer that, I am just trying to form a position on intelligence itself, human intelligence. I don’t think IQ is predictive of how much power people have. I don’t think the world is an IQ meritocracy. I understand how it can look like one from the Silicon Valley, because the SV is an IQ meritocracy, where people who actually use intelligence to change the world go to, but in general the world is not so. IQ is more of a puzzle-solving ability and I think it transforms to world-changing power only when the bottleneck is specifically the lack of puzzle-solving ability. When we have infinite amounts of dedication, warm bodies, electricity and money to throw on a problem and the only thing missing is that we don’t know how, then yes, smart people are useful, they can figure that out. But that is not the usual case. Imagine you are a 220 IQ Russian and just solved cold fusion and offer it to Putin out of patriotism. You probably get shot and the blueprints buried because it threatens the energy exports so important for their economy and for the power of his oligarch supporters. This is IMHO how intelligence works, if there is everything there, especially the will, to change the world, and only the know-how is missing, then it is useful, but it is not the typical case, and in all the other cases it does not help much. Yes, of course a superintelligence could figure out, say, nanotechnology, but why should we assume that main reason why the world is not a utopia is the lack of such know-how?
This is why I am so suspicious about it. That I am afraid the whole thing is Silicon Valley culture written large, and this is not predictive of how the world works. SV is full of people eager to change the world, have the dedication, the money, all is missing is knowing how. A superintelligence could help them, but it does not mean intelligence, or superintelligence, is a general power, a general optimization ability, a general goal-achievement ability. I think it is only true in the special cases when achieving goals requires solving puzzles. Usually, it is not unsolved puzzles that are standing between you and your goal.
Imagine I wanted to be dictator of a first-world country. Is there any puzzle I could solve if I had a gazillion IQ that could achieve that? No. I could read all the books about how to manipulate people and figure out the best things to say and the result would still be that people don’t want to give up their freedom especially not to some uncharismatic fat nerds no matter how excellent sounding things he says. But maybe if it was an economic depression and I would be simply highly charismatic, ruthless, and have all the right ex-classmates… I would have a chance to pull that off without any puzzle-solving, just being smart enough to not sabotage myself with blunders, say, IQ 120 would do it.
And I am worried I am missing something huge. MIRI consists of far smarter people than I am, so I am almost certainly wrong… unless they have fallen in love with smartness so much that it created a pro-smart bias in them, unless it made them underestimate in how many cases efficient world changing has nothing to do with puzzle solving and has everything to do with beating obstacles with a huge hammer, or forging consensus or a million other things.
But I think the actual data about what 180+ IQ people are doing is on my side here. What is Kasparov doing? What is James Woods doing? Certainly not radically transforming the world. Nor taking it over.
I don’t think IQ is predictive of how much power people have. I don’t think the world is an IQ meritocracy.
Predictive isn’t a binary catergory. Statistically IQ is predictive of a lot of things including higher social skills and lifespan.
Imagine I wanted to be dictator of a first-world country. Is there any puzzle I could solve if I had a gazillion IQ that could achieve that? No. I could read all the books about how to manipulate people and figure out the best things to say and the result would still be that people don’t want to give up their freedom especially not to some uncharismatic fat nerds no matter how excellent sounding things he says.
Bernanke was during his federal reserve tenureship one of the most powerful people in the US and he scored 1590 out of 1600 on the SAT.
But I think the actual data about what 180+ IQ people are doing is on my side here. What is Kasparov doing?
Kasparov is not 180+ IQ. When given a real IQ test he scored 135. Lower than the average of LW people who submit their IQ on the census.
Over 20 IQ points lower than what Bernanke has and given that Bernanke scored at the top and the test SAT isn’t created to distinguish 160 from 180 he might be even smarter.
Bernake’s successor got described in her NYTimes profile by a collegue as a “small lady with a large I.Q.”.
While I can’t find direct scores she likely has a higher IQ than Kasparov.
Top bankers are high IQ people and at the moment the banker class has quite a lot of power in the US.
Banking likely needs more IQ than playing chess.
Bill O’Reilly is with a SAT score of 1585 also much smarter than Kasparov and the guy seems to have some influence in US political discussion.
Statistically IQ is predictive of a lot of things including higher social skills
How? Pretty sure it was the other way around in high school. Popular dumb shallow people, unpopular smart geeks.
I am not 100% sure what is a SAT is but if it is like a normal school test—memorize stuff like historical dates of battles, barf it back—the probably more related to dedication than intelligence.
Popular dumb shallow people, unpopular smart geeks.
That could be a problem of perception. If someone is book smart and unpopular, people say “he is smart”. If someone is smart and popular, people say “he is cool”.
There are sportsmen and actors with very high IQ, but no one remembers them as “having high IQ”, only as being a great sportsman or a great actor.
Do you know how high IQ Arnold Schwarzenegger has? Neither do I. My point is, when people say “Arnold Schwarzenegger”, no one thinks about “I wonder how high IQ that guy has… maybe that could explain some of his success”. Maybe he has much higher IQ than Kasparov, but most people would never even start thinking in that direction.
Also there is a difference between intelligence and signalling intelligence. Not everyone with high IQ must necessarily spend their days talking about relativity and quantum physics. Maybe they just invest all their intelligence into improving their private lives.
Do you know how high IQ Arnold Schwarzenegger has? Neither do I. My point is, when people say “Arnold Schwarzenegger”, no one thinks about “I wonder how high IQ that guy has… maybe that could explain some of his success”. Maybe he has much higher IQ than Kasparov, but most people would never even start thinking in that direction.
The score I find on the internet for him is 135 which puts him on the same ballpark as Kasparov.
Popular dumb shallow people, unpopular smart geeks.
You mistake what people signal with their inherent intelligence. Bill O’Reilly doesn’t behave as a geek. That change that he’s smart.
It’s like the mistake of thinking that being good at chess is about intelligence.
Unfortunately at the moment I don’t find the link to the studies about IQ and social skill, but I think we had previous LW discussions towards IQ positively correlating with social abilities in average society being well established.
I am not 100% sure what is a SAT is but if it is like a normal school test—memorize stuff like historical dates of battles, barf it back
It’s not about having memorized information. SAT tests are generally known to be a good proxy for IQ.
But again that is not power. That is just smart people getting paid better when and if there are enough jobs around where intelligence is actually useful. I think it is a very big jump from the fact that there seem to be relatively lot of those jobs around to saying it is a general world-changing, outcome-generating power. I cannot find it , but I think income could just as well be correlated with height.
If I understand, you are attempting a “proves too much” argument with height, however, this is irrelevant, if height is predictive of income then this is an interesting result in itself* (maybe tall people are more respected) and has no bearing on whether IQ is also predictive of income. I agree that IQ probably doesn’t scale indefinitely with success and power though. The tails are already starting to diverge at 130.
Well, a while back I was reading an article about Terrence Tao. Now, in it, it said that bout 1 in 40 child prodigies go on to become incredible academics. Part of the reason is that they are forced form an early age to learn as much as possible. There was one such child prodigy, who published papers at the age of 15 and just gradually faded away afterwards. Because of their environment, these child prodigies burn out very quickly, and grow jaded towards academia. Not wanting this to happen, Tao’s parent let him go at a pace he was comfortable with. What did that result in? Him becoming one of the greatest mathematicians on the planet.
So yes, many 180+iq individuals never go on to become great, but that’s largely due to their environment.And most geniuses just lack the drive/charisma/interest to take over the world. But that still doesn’t answer your question about ‘How could an AI take over the world?’ or something to that effect.
Well, you pointed out that some rich, charismatic individual with the right connections could make a big dent in society. But, charisma is something that can be learnt. Sure, it’s hard and maybe not everyone can do it, but it can be learnt. Now, if one has sufficient charisma, one can make a huge impact on the world i.e. Hitler. (The slaterstarcodex blog discussed something similar to this, and I’m just basically lifting the Hitler thing from there).
The Nazi party had about 55 members when he joined, and at the time he was just a failed painter/veteran soldier. And there are disturbing records that people would say things like ’Look at this new guy! We’ve got to promote him. Everyone he talks to is just joining us, its insane!” And people just flocked to him. Someone might hate him, go see a speech or two, and then become a lifelong convert. Even in WW2 when the German country was getting progressively worse, he still had 90% approval from his people. This is what charisma can do. And this is something the AI can learn.Now, imagine what this AI, who’ll be much more intelligent than Hitler, could do.
Furthermore, an AI would be connected to the entirety of the internet, and once it had learnt pretty much everything, it would be able to gain so much power. For example:
1) It could gain capital very rapidly. Certain humans have managed to gain huge amounts of money by overcoming their biases and making smart business decisions. Everything they learnt, so too could an AI, and with far fewer biases. So the AI could rapidly acquire business capital.
2) It could use this business capital to set up various companies, and with its intellectual capabilities, it could outpace other companies in terms of innovation. In a few years, it might well dominate the market.
3) Because of its vast knowledge drawn from the internet, it would be able to market far more succesfully than other organisations. Rather quickly it would draw people to its side, gaining a lot of social capital.
4) It would also be able to gain a lot of knowledge about any competitors and give it yet another edge over them.
5) Due to its advanced marketing strategies, wealth and social capital, it could make a party in some place and place a figurehead government in power. From there, it would be able to increase the countries power (probably in a careful fashion, not upsetting the people and keeping allies around it).
6) Now it has a country under its control, large sway over the rest of the world, and a huge amount of resources. From here, it could advance manufacturing and production to such a degree that it would need no humans to work for it.
7) Still acting carefully, the AI would now have the capability to build pretty much whatever it wanted. From there, it could institute more autonomous production plants around the world. It may provide many goods for free for the locals, in order to keep them on its side.
8) Now the AI can try and take over other countries, making parties with its backing, and promising a Golden Age for mankind.
9) The AI has transformed itself into the head of the worlds greatest superpower
10) Victory
This is just a rough outline of the path an AI could take. In all the stages it simply replicated the feats of extraordinary individuals throughout history. Nothing it did was impossible, and I would say not even implausible. Of course, it could do things very differently. Once we make autonomous production plants, the would AI just need to take them over, produce large amounts of robots and weaponry, and take the world over. Or maybe it would just hold its economic welfare hostage.
Certain humans have managed to gain huge amounts of money by overcoming their biases and making smart business decision
Thinking that one can outsmart the market is the biggest bias in this regard. People like Soros and Buffet were either lucky or had secret information others didn’t, because otherwise it should be very, very unlikely to outsmart the market.
I wasn’t referring to the stock market. I know that almost all money made by ‘playing the market’ is due to luck. What I meant was creating the right type of service/good with the right kind of marketing. More like Steve Jobs, Elon Musk and so forth.
A quant who does day trading can find that there some market inefficiency between the prices of different products and then make money with the effect.
That’s not how either Soros or Buffet made their fortunes but it’s still possible for other people.
But even if it wants to help, can it? Is intelligence really such a powerful tool? I picture it as we are aiming an arrow at something, like building a Mars base, and it makes our arrow more accurate. But if it was accurate enough to do the job how much does that help? How does intelligence transform into power to generate outcomes?
I mean the issue is that EY defines pretty much as efficient goal-achievement or cross-domain optimization, so by using that definition intelligence is trivially about a power to generate outcomes, but this is simply not the common definition of intelligence. It is not what IQ tests measure. It is not what Mensa members are good at. It is not what Kasparov is good at. That is more of a puzzle-solving ability.
I mean, could efficient goal-achievement equal puzzle-solving? I would be really surprised if someone could prove that. In the real life the most efficient goal achievers I know are often stupid and have other virtues, like a bulldog like never, ever, ever, ever give up type attitude.
Might I ask, what exactly do you mean? Are you saying that the super intelligent AI would not be able to contribute that much to our intellectual endeavours? Or are you saying that its intelligence may not translate to achieving goals like ‘become a billionaire’ or ‘solve world hunger’? Or something else entirely?
I don’t yet know enough about AI to even attempt to answer that, I am just trying to form a position on intelligence itself, human intelligence. I don’t think IQ is predictive of how much power people have. I don’t think the world is an IQ meritocracy. I understand how it can look like one from the Silicon Valley, because the SV is an IQ meritocracy, where people who actually use intelligence to change the world go to, but in general the world is not so. IQ is more of a puzzle-solving ability and I think it transforms to world-changing power only when the bottleneck is specifically the lack of puzzle-solving ability. When we have infinite amounts of dedication, warm bodies, electricity and money to throw on a problem and the only thing missing is that we don’t know how, then yes, smart people are useful, they can figure that out. But that is not the usual case. Imagine you are a 220 IQ Russian and just solved cold fusion and offer it to Putin out of patriotism. You probably get shot and the blueprints buried because it threatens the energy exports so important for their economy and for the power of his oligarch supporters. This is IMHO how intelligence works, if there is everything there, especially the will, to change the world, and only the know-how is missing, then it is useful, but it is not the typical case, and in all the other cases it does not help much. Yes, of course a superintelligence could figure out, say, nanotechnology, but why should we assume that main reason why the world is not a utopia is the lack of such know-how?
This is why I am so suspicious about it. That I am afraid the whole thing is Silicon Valley culture written large, and this is not predictive of how the world works. SV is full of people eager to change the world, have the dedication, the money, all is missing is knowing how. A superintelligence could help them, but it does not mean intelligence, or superintelligence, is a general power, a general optimization ability, a general goal-achievement ability. I think it is only true in the special cases when achieving goals requires solving puzzles. Usually, it is not unsolved puzzles that are standing between you and your goal.
Imagine I wanted to be dictator of a first-world country. Is there any puzzle I could solve if I had a gazillion IQ that could achieve that? No. I could read all the books about how to manipulate people and figure out the best things to say and the result would still be that people don’t want to give up their freedom especially not to some uncharismatic fat nerds no matter how excellent sounding things he says. But maybe if it was an economic depression and I would be simply highly charismatic, ruthless, and have all the right ex-classmates… I would have a chance to pull that off without any puzzle-solving, just being smart enough to not sabotage myself with blunders, say, IQ 120 would do it.
And I am worried I am missing something huge. MIRI consists of far smarter people than I am, so I am almost certainly wrong… unless they have fallen in love with smartness so much that it created a pro-smart bias in them, unless it made them underestimate in how many cases efficient world changing has nothing to do with puzzle solving and has everything to do with beating obstacles with a huge hammer, or forging consensus or a million other things.
But I think the actual data about what 180+ IQ people are doing is on my side here. What is Kasparov doing? What is James Woods doing? Certainly not radically transforming the world. Nor taking it over.
Predictive isn’t a binary catergory. Statistically IQ is predictive of a lot of things including higher social skills and lifespan.
Bernanke was during his federal reserve tenureship one of the most powerful people in the US and he scored 1590 out of 1600 on the SAT.
Kasparov is not 180+ IQ. When given a real IQ test he scored 135. Lower than the average of LW people who submit their IQ on the census. Over 20 IQ points lower than what Bernanke has and given that Bernanke scored at the top and the test SAT isn’t created to distinguish 160 from 180 he might be even smarter.
Bernake’s successor got described in her NYTimes profile by a collegue as a “small lady with a large I.Q.”. While I can’t find direct scores she likely has a higher IQ than Kasparov.
Top bankers are high IQ people and at the moment the banker class has quite a lot of power in the US. Banking likely needs more IQ than playing chess.
Bill O’Reilly is with a SAT score of 1585 also much smarter than Kasparov and the guy seems to have some influence in US political discussion.
How? Pretty sure it was the other way around in high school. Popular dumb shallow people, unpopular smart geeks.
I am not 100% sure what is a SAT is but if it is like a normal school test—memorize stuff like historical dates of battles, barf it back—the probably more related to dedication than intelligence.
That could be a problem of perception. If someone is book smart and unpopular, people say “he is smart”. If someone is smart and popular, people say “he is cool”.
There are sportsmen and actors with very high IQ, but no one remembers them as “having high IQ”, only as being a great sportsman or a great actor.
Do you know how high IQ Arnold Schwarzenegger has? Neither do I. My point is, when people say “Arnold Schwarzenegger”, no one thinks about “I wonder how high IQ that guy has… maybe that could explain some of his success”. Maybe he has much higher IQ than Kasparov, but most people would never even start thinking in that direction.
Also there is a difference between intelligence and signalling intelligence. Not everyone with high IQ must necessarily spend their days talking about relativity and quantum physics. Maybe they just invest all their intelligence into improving their private lives.
The score I find on the internet for him is 135 which puts him on the same ballpark as Kasparov.
You mistake what people signal with their inherent intelligence. Bill O’Reilly doesn’t behave as a geek. That change that he’s smart. It’s like the mistake of thinking that being good at chess is about intelligence.
Unfortunately at the moment I don’t find the link to the studies about IQ and social skill, but I think we had previous LW discussions towards IQ positively correlating with social abilities in average society being well established.
It’s not about having memorized information. SAT tests are generally known to be a good proxy for IQ.
More helpful than single data points, here is a scatterplot of IQ vs income in Figure 1.
But again that is not power. That is just smart people getting paid better when and if there are enough jobs around where intelligence is actually useful. I think it is a very big jump from the fact that there seem to be relatively lot of those jobs around to saying it is a general world-changing, outcome-generating power. I cannot find it , but I think income could just as well be correlated with height.
If I understand, you are attempting a “proves too much” argument with height, however, this is irrelevant, if height is predictive of income then this is an interesting result in itself* (maybe tall people are more respected) and has no bearing on whether IQ is also predictive of income. I agree that IQ probably doesn’t scale indefinitely with success and power though. The tails are already starting to diverge at 130.
*there is a correlation
It’s useful but it’s about comparing people between IQ 100 and 130. If we want to look at the power in society it’s worth looking at the extremes.
Well, a while back I was reading an article about Terrence Tao. Now, in it, it said that bout 1 in 40 child prodigies go on to become incredible academics. Part of the reason is that they are forced form an early age to learn as much as possible. There was one such child prodigy, who published papers at the age of 15 and just gradually faded away afterwards. Because of their environment, these child prodigies burn out very quickly, and grow jaded towards academia. Not wanting this to happen, Tao’s parent let him go at a pace he was comfortable with. What did that result in? Him becoming one of the greatest mathematicians on the planet.
So yes, many 180+iq individuals never go on to become great, but that’s largely due to their environment.And most geniuses just lack the drive/charisma/interest to take over the world. But that still doesn’t answer your question about ‘How could an AI take over the world?’ or something to that effect.
Well, you pointed out that some rich, charismatic individual with the right connections could make a big dent in society. But, charisma is something that can be learnt. Sure, it’s hard and maybe not everyone can do it, but it can be learnt. Now, if one has sufficient charisma, one can make a huge impact on the world i.e. Hitler. (The slaterstarcodex blog discussed something similar to this, and I’m just basically lifting the Hitler thing from there).
The Nazi party had about 55 members when he joined, and at the time he was just a failed painter/veteran soldier. And there are disturbing records that people would say things like ’Look at this new guy! We’ve got to promote him. Everyone he talks to is just joining us, its insane!” And people just flocked to him. Someone might hate him, go see a speech or two, and then become a lifelong convert. Even in WW2 when the German country was getting progressively worse, he still had 90% approval from his people. This is what charisma can do. And this is something the AI can learn.Now, imagine what this AI, who’ll be much more intelligent than Hitler, could do.
Furthermore, an AI would be connected to the entirety of the internet, and once it had learnt pretty much everything, it would be able to gain so much power. For example:
1) It could gain capital very rapidly. Certain humans have managed to gain huge amounts of money by overcoming their biases and making smart business decisions. Everything they learnt, so too could an AI, and with far fewer biases. So the AI could rapidly acquire business capital.
2) It could use this business capital to set up various companies, and with its intellectual capabilities, it could outpace other companies in terms of innovation. In a few years, it might well dominate the market.
3) Because of its vast knowledge drawn from the internet, it would be able to market far more succesfully than other organisations. Rather quickly it would draw people to its side, gaining a lot of social capital.
4) It would also be able to gain a lot of knowledge about any competitors and give it yet another edge over them.
5) Due to its advanced marketing strategies, wealth and social capital, it could make a party in some place and place a figurehead government in power. From there, it would be able to increase the countries power (probably in a careful fashion, not upsetting the people and keeping allies around it).
6) Now it has a country under its control, large sway over the rest of the world, and a huge amount of resources. From here, it could advance manufacturing and production to such a degree that it would need no humans to work for it.
7) Still acting carefully, the AI would now have the capability to build pretty much whatever it wanted. From there, it could institute more autonomous production plants around the world. It may provide many goods for free for the locals, in order to keep them on its side.
8) Now the AI can try and take over other countries, making parties with its backing, and promising a Golden Age for mankind.
9) The AI has transformed itself into the head of the worlds greatest superpower
10) Victory
This is just a rough outline of the path an AI could take. In all the stages it simply replicated the feats of extraordinary individuals throughout history. Nothing it did was impossible, and I would say not even implausible. Of course, it could do things very differently. Once we make autonomous production plants, the would AI just need to take them over, produce large amounts of robots and weaponry, and take the world over. Or maybe it would just hold its economic welfare hostage.
Thinking that one can outsmart the market is the biggest bias in this regard. People like Soros and Buffet were either lucky or had secret information others didn’t, because otherwise it should be very, very unlikely to outsmart the market.
I wasn’t referring to the stock market. I know that almost all money made by ‘playing the market’ is due to luck. What I meant was creating the right type of service/good with the right kind of marketing. More like Steve Jobs, Elon Musk and so forth.
A quant who does day trading can find that there some market inefficiency between the prices of different products and then make money with the effect.
That’s not how either Soros or Buffet made their fortunes but it’s still possible for other people.