Recent math PhD focused on AI strategy and forecasting. Contributor at AI Futures Project.
Alex Kastner
Karma: 188
Here are some more thoughts on a superintelligence-run persuasion campaign:
Like Daniel wrote in a comment, it’s good to think of Agent-5 as distributed and able to nudge things all over the internet. The nudges could be highly personalized to demographics and individuals, so responsive to the kind of subtle emotional triggers the superintelligence learns about each individual.
It seems many people’s opinions today are already significantly shaped by social media and disinformation. So this makes me think a similar process that’s much more agentic, personalized, and superintelligence-optimized could be very potent.
There’s the possibility of mind-hacking too, though I chose to leave that out of the blogpost.
The CEO is probably well-positioned to take credit for a lot of the benefits Agent-5 seems to bring to the world (some of these benefits are genuine, some illusory).
In an earlier iteration of this scenario I had a military coup rather than this gradual political ascension via persuasion. But then I decided that a superintelligence capable of controlling the robots well enough to disempower the human military would probably also be powerful enough to do something less heavy-handed like what’s in the scenario.
In my scenario I was imagining Agent-5 to be superhuman at politics. But I tentatively think [just top-human at politics + being extremely well-informed about what every human actor wants and knows + being able to have a coordinated influence on all these human actors] already has a big shot at leading to the story’s events.
It’s also important that the CEO starts from a relative position of power and prestige, and is not some random guy. He’s the kind of person who could rise to power the way he does (at least at first) if he was lucky and had political acumen. Maybe I should have emphasized this point more in the scenario.