What makes you think that those people were able to do those things because of high levels of intelligence? It seems to me that in most cases, the reported feat is probably driven by some capability / context combination that stretches the definition of intelligence to varying degrees. For instance I would guess that El Chapo pulled that off because he already had a lot of connections and money when he got to prison. The other examples seem to demonstrate that it is possible for a person to develop impressive capabilities in a restricted domain given enough experience.
We are exactly worried about that though. It is not that AGI will be inteligent (that is the name), but that it can and probably will develop dangerous capabilities. Inteligence is the word we use to describe it, since it is associated with the ability to gain capability, but even if the AGI is sometimes kind of brute force or dumb does not mean that it cannot also have dangerous enough capabilities to beat us out.
The post is an intuition pump for the idea that intelligence enables capabilities that look like “magic.”
It seems to me that all it really demonstrates is that some people have capabilities that look like magic, within domains where they are highly specialized to succeed. The only example that seems particularly dangerous (El Chapo) does not seem convincingly connected to intelligence. I am also not sure what the chess example is supposed to prove—we already have chess engines that can defeat multiple people at once blindfolded, including (presumably) Magnus Carlsen. Are those chess engines smarter than Magnus Carlsen? No.
This kind of nitpick is important precisely because the argument is so vague and intuitive. Its pushing on a fuzzy abstraction that intelligence is dangerous in a way that seems convincing only if you’ve already accepted a certain model of intelligence. The detailed arguments don’t seem to work.
The conclusion that AGI may be able to do things that seem like magic to us is probably right, but this post does not hold up to scrutiny as an intuition pump.
The only example that seems particularly dangerous (El Chapo) does not seem convincingly connected to intelligence
I’d say “being able to navigate a highly complex network of agents, a lot of which are adversaries” counts as “intelligence”. Well, one form of intelligence, at least.
This point suggests alternative models for risks and opportunities from “AI”. If deep learning applied to various narrow problems is a new source of various superhuman capabilities, that has a lot of implications for the future of the world, setting “AGI” aside.
Perhaps you think of intelligence as just raw IQ. I count persuasion as a part of intelligence. After all, if someone can’t put two coherent sentences together, they won’t be very persuasive. Obviously being able to solve math/logic problems and being persuasive are very different things, but again, I count both as “intelligence”.
Of course, El Chapo had money (to bribe prison guards), which a “boxed” AGI won’t have, that I agree with. I disagree that it will make a big difference.
I intentionally avoided as much as possible the implication that intelligence is “only” raw IQ. But if intelligence is not on some kind of real-valued scale, what does any part of this post mean?
If you want example particularly connected to prisons, you can take anarchist revolutionary Sergey Nechayev, who was able to propagandize prison guards enough to connect with outside terrorist cell. The only reason why Nechayev didn’t escape is because Narodnaya Volya planned assasination of Tsar and they didn’t want escape to interfere.
I think that high levels of intelligence make it easier to develop capabilities similar to the ones discussed in 1 and 3-5, up to a point. (I agree that El Chapo should be discounted due to the porosity of Mexican prisons) A being with an inherently high level of intelligence will be able to gather more information from events in their life and process that information more quickly, resulting in a faster rate of learning. Hence, a superintelligence will acquire capabilities similar to magic more quickly. Furthermore, the capability ceiling of a superintelligence will be higher than the capability ceiling of a human, so they will acquire magic-like capabilities impossible for humans to ever preform.
What makes you think that those people were able to do those things because of high levels of intelligence? It seems to me that in most cases, the reported feat is probably driven by some capability / context combination that stretches the definition of intelligence to varying degrees. For instance I would guess that El Chapo pulled that off because he already had a lot of connections and money when he got to prison. The other examples seem to demonstrate that it is possible for a person to develop impressive capabilities in a restricted domain given enough experience.
We are exactly worried about that though. It is not that AGI will be inteligent (that is the name), but that it can and probably will develop dangerous capabilities. Inteligence is the word we use to describe it, since it is associated with the ability to gain capability, but even if the AGI is sometimes kind of brute force or dumb does not mean that it cannot also have dangerous enough capabilities to beat us out.
The post is an intuition pump for the idea that intelligence enables capabilities that look like “magic.”
It seems to me that all it really demonstrates is that some people have capabilities that look like magic, within domains where they are highly specialized to succeed. The only example that seems particularly dangerous (El Chapo) does not seem convincingly connected to intelligence. I am also not sure what the chess example is supposed to prove—we already have chess engines that can defeat multiple people at once blindfolded, including (presumably) Magnus Carlsen. Are those chess engines smarter than Magnus Carlsen? No.
This kind of nitpick is important precisely because the argument is so vague and intuitive. Its pushing on a fuzzy abstraction that intelligence is dangerous in a way that seems convincing only if you’ve already accepted a certain model of intelligence. The detailed arguments don’t seem to work.
The conclusion that AGI may be able to do things that seem like magic to us is probably right, but this post does not hold up to scrutiny as an intuition pump.
I’d say “being able to navigate a highly complex network of agents, a lot of which are adversaries” counts as “intelligence”. Well, one form of intelligence, at least.
This point suggests alternative models for risks and opportunities from “AI”. If deep learning applied to various narrow problems is a new source of various superhuman capabilities, that has a lot of implications for the future of the world, setting “AGI” aside.
Perhaps you think of intelligence as just raw IQ. I count persuasion as a part of intelligence. After all, if someone can’t put two coherent sentences together, they won’t be very persuasive. Obviously being able to solve math/logic problems and being persuasive are very different things, but again, I count both as “intelligence”.
Of course, El Chapo had money (to bribe prison guards), which a “boxed” AGI won’t have, that I agree with. I disagree that it will make a big difference.
I intentionally avoided as much as possible the implication that intelligence is “only” raw IQ. But if intelligence is not on some kind of real-valued scale, what does any part of this post mean?
If you want example particularly connected to prisons, you can take anarchist revolutionary Sergey Nechayev, who was able to propagandize prison guards enough to connect with outside terrorist cell. The only reason why Nechayev didn’t escape is because Narodnaya Volya planned assasination of Tsar and they didn’t want escape to interfere.
I think that high levels of intelligence make it easier to develop capabilities similar to the ones discussed in 1 and 3-5, up to a point. (I agree that El Chapo should be discounted due to the porosity of Mexican prisons) A being with an inherently high level of intelligence will be able to gather more information from events in their life and process that information more quickly, resulting in a faster rate of learning. Hence, a superintelligence will acquire capabilities similar to magic more quickly. Furthermore, the capability ceiling of a superintelligence will be higher than the capability ceiling of a human, so they will acquire magic-like capabilities impossible for humans to ever preform.