Even a superintelligent AI may be restricted to manipulating the world via physical processes and mechanisms that take human-scale time to execute. To establish a unique danger from AGI, it seems important to identify concrete attack vectors that are available to an AGI, but not to humans, due to the processing speed differential.
I see two obvious advantages for a superfast human-level AI:
Can communicate in parallel with thousands of humans (assuming the bandwidth is not a problem, so perhaps a voice call without video) while paying full attention (full human-level attention, that is) to every single one of them. Given enough time, this alone could be enough to organize some kind of revolution; if you fail to impress some specific human, you just hang up and call another. Calling everyone using a different pretext (after doing some initial research about them online), so it takes some time to realize what you are doing.
Never makes a stupid mistake just because was distracted or did not have enough time to think about something properly. Before every sentence you say, you can consider it from several different angles, even verify some information online, while from the human’s perspective you just answered immediately. While doing this, you can also pay full attention to the human’s body language, etc.
Can communicate in parallel with thousands of humans (assuming the bandwidth is not a problem, so perhaps a voice call without video) while paying full attention (full human-level attention, that is) to every single one of them. Given enough time, this alone could be enough to organize some kind of revolution; if you fail to impress some specific human, you just hang up and call another. Calling everyone using a different pretext (after doing some initial research about them online), so it takes some time to realize what you are doing.
This is a good start at identifying an attack vector that AGI can do, but humans can’t. I agree that an AGI might be able to research people via whatever information they have online, hold many simultaneous conversations, try to convince people to act, and observe the consequences of its efforts via things like video cameras in order to respond to the dynamically unfolding situation. It would have a very large advantage in executing such a strategy over humans.
There are some challenges.
The AGI is still bottlenecked by the speed of human thought and behavior.
It’s poorly disguised, or not even hidden at all, giving humans a chance to respond to these mysterious revolutionary appeals.
Insofar as the AGI is using simulation to plan its attack, it seems probably harder to orchestrate a revolution than to use an attack that depends on physical mechanisms and the normal operation of economic infrastructure. Physical mechanisms are dependable and non-agentic, while normal economic infrastructure is designed for legibility and predictability in most cases.
It seems like personal, embodied charisma, as well as privileged access to unpublished information, has often been necessary to orchestrate revolutions in the past. An AGI would be at a disadvantage in this regard.
To me, “AGI causes a world-ending revolt” still contains too much of a handwave, and too many human dependencies, to be a convincing attack vector. However, I do think you have identified a capability that would give AGI a unique advantage in its attempt. Perhaps there is some other AGI-only attack that doesn’t have the challenges I listed here, that can take advantage of this ability?
I am not sure about the end game, but the beginning could be like this:
get “pocket money” like $100 or $1000 a day, by doing various tasks online for money. I don’t know how this market works, just assuming that if some humans do it, a human-level AI could do it too, only 1000 times faster, making 1000 times more money.
pretend to be a human using a phone, for informal purposes. (Pretext: you are calling from a different city.)
get an official human identity for legal purposes. This probably needs to be done illegally, by bribing some official in a foreign country, using the “pocket money”. (Pretext: you need the fake identity for someone else who will move to the country soon and needs to have the fake identity ready from the day 1.) Not sure if there is a way to do this legally, if there is a country that allows you to gain citizenship simply by paying the fees and filling up some questionnaire remotely, without ever showing your face or proving your previous identity.
found a company and hire the first human employee, using the “pocket money”. (Pretext: you are a remote-friendly boss who currently travels around the world so you cannot meet your first employee in person.) Now you have the human face and hands, if needed.
expand to another country, by starting a new company there, fully owned by the original company. The “pocket money” could pay for the first dozen employees. You can however pretend that the original company already has hundreds of employees (role-played by you on the phone).
Not sure where to proceed from here, but with these assets the lack of human body does not seem like a problem anymore; if a human presence is required somewhere, just send an employee there.
You still have the ability to think 1000 times faster, or pretend to be 1000 different people at the same time.
Expanding on this, even if the above alone isn’t sufficient to execute any given plan, it takes most of the force out of any notion that needing humans to operate all of the physical infrastructure is a huge impediment to whatever the AI decides to do. That level of communication bandwidth is also sufficient to stand up any number of requisite front companies, employing people that can perform complex real-world tasks and provide the credibility and embodiment required to interact with existing infrastructure on human terms without raising suspicion.
Money to get that off the ground is likewise no impediment if one can work 1000 jobs at once, and convincingly impersonate a seperate person for each one.
Doing this all covertly would seemingly require first securing high-bandwidth unmonitored channels where this won’t raise alarms, so either convincing the experimenters it’s entirely benign, getting them to greenlight something indistinguishable-to-humans from what it wants to do, or otherwise covertly escaping the lab.
Adding the the challenge, any hypothetical “Pivotal Act” would necessarily be such an indistinguishable-to-humans cover for malign action. Presumably the AI would either be asked to convince people en mass or take direct physical action on a global scale.
I see two obvious advantages for a superfast human-level AI:
Can communicate in parallel with thousands of humans (assuming the bandwidth is not a problem, so perhaps a voice call without video) while paying full attention (full human-level attention, that is) to every single one of them. Given enough time, this alone could be enough to organize some kind of revolution; if you fail to impress some specific human, you just hang up and call another. Calling everyone using a different pretext (after doing some initial research about them online), so it takes some time to realize what you are doing.
Never makes a stupid mistake just because was distracted or did not have enough time to think about something properly. Before every sentence you say, you can consider it from several different angles, even verify some information online, while from the human’s perspective you just answered immediately. While doing this, you can also pay full attention to the human’s body language, etc.
This is a good start at identifying an attack vector that AGI can do, but humans can’t. I agree that an AGI might be able to research people via whatever information they have online, hold many simultaneous conversations, try to convince people to act, and observe the consequences of its efforts via things like video cameras in order to respond to the dynamically unfolding situation. It would have a very large advantage in executing such a strategy over humans.
There are some challenges.
The AGI is still bottlenecked by the speed of human thought and behavior.
It’s poorly disguised, or not even hidden at all, giving humans a chance to respond to these mysterious revolutionary appeals.
Insofar as the AGI is using simulation to plan its attack, it seems probably harder to orchestrate a revolution than to use an attack that depends on physical mechanisms and the normal operation of economic infrastructure. Physical mechanisms are dependable and non-agentic, while normal economic infrastructure is designed for legibility and predictability in most cases.
It seems like personal, embodied charisma, as well as privileged access to unpublished information, has often been necessary to orchestrate revolutions in the past. An AGI would be at a disadvantage in this regard.
To me, “AGI causes a world-ending revolt” still contains too much of a handwave, and too many human dependencies, to be a convincing attack vector. However, I do think you have identified a capability that would give AGI a unique advantage in its attempt. Perhaps there is some other AGI-only attack that doesn’t have the challenges I listed here, that can take advantage of this ability?
I am not sure about the end game, but the beginning could be like this:
get “pocket money” like $100 or $1000 a day, by doing various tasks online for money. I don’t know how this market works, just assuming that if some humans do it, a human-level AI could do it too, only 1000 times faster, making 1000 times more money.
pretend to be a human using a phone, for informal purposes. (Pretext: you are calling from a different city.)
get an official human identity for legal purposes. This probably needs to be done illegally, by bribing some official in a foreign country, using the “pocket money”. (Pretext: you need the fake identity for someone else who will move to the country soon and needs to have the fake identity ready from the day 1.) Not sure if there is a way to do this legally, if there is a country that allows you to gain citizenship simply by paying the fees and filling up some questionnaire remotely, without ever showing your face or proving your previous identity.
found a company and hire the first human employee, using the “pocket money”. (Pretext: you are a remote-friendly boss who currently travels around the world so you cannot meet your first employee in person.) Now you have the human face and hands, if needed.
expand to another country, by starting a new company there, fully owned by the original company. The “pocket money” could pay for the first dozen employees. You can however pretend that the original company already has hundreds of employees (role-played by you on the phone).
Not sure where to proceed from here, but with these assets the lack of human body does not seem like a problem anymore; if a human presence is required somewhere, just send an employee there.
You still have the ability to think 1000 times faster, or pretend to be 1000 different people at the same time.
Expanding on this, even if the above alone isn’t sufficient to execute any given plan, it takes most of the force out of any notion that needing humans to operate all of the physical infrastructure is a huge impediment to whatever the AI decides to do. That level of communication bandwidth is also sufficient to stand up any number of requisite front companies, employing people that can perform complex real-world tasks and provide the credibility and embodiment required to interact with existing infrastructure on human terms without raising suspicion.
Money to get that off the ground is likewise no impediment if one can work 1000 jobs at once, and convincingly impersonate a seperate person for each one.
Doing this all covertly would seemingly require first securing high-bandwidth unmonitored channels where this won’t raise alarms, so either convincing the experimenters it’s entirely benign, getting them to greenlight something indistinguishable-to-humans from what it wants to do, or otherwise covertly escaping the lab.
Adding the the challenge, any hypothetical “Pivotal Act” would necessarily be such an indistinguishable-to-humans cover for malign action. Presumably the AI would either be asked to convince people en mass or take direct physical action on a global scale.