Can only Superintelligent AI be able to escape AI labs and self-replicate itself on other servers?
The RRS has rogue AIs become capable of self-replication on other servers far earlier than Agent-4. The author assumes that these AIs cause enough chaos to have mankind create an aligned ASI. \
why there is so much emphasis on “rogue AIs” plural
Rogue AIs are AIs who were assigned different tasks or outright different LLMs whose weights became public (e.g. DeepSeekV3.1 or KimiK2, but they aren’t YET capable of self-replication). Of course, these AIs find it hard to coordinate with each other.
As for “superintelligent AI being super enlightened and super capable and so fixing all of our complex problems” the superintelligent AI itself is to be aligned with human needs. Agent-4 from the Race Ending is NOT aligned to the humans. And that’s ignoring the possibility that the superintelligent AI who is super enlightened has a vision of mankind’s future which differs from the ideas of human hosts (e.g. Zuckerberg). I tried exploring the results of such a mismatch in my take at writing scenarios.
The RRS has rogue AIs become capable of self-replication on other servers far earlier than Agent-4. The author assumes that these AIs cause enough chaos to have mankind create an aligned ASI. \
Rogue AIs are AIs who were assigned different tasks or outright different LLMs whose weights became public (e.g. DeepSeekV3.1 or KimiK2, but they aren’t YET capable of self-replication). Of course, these AIs find it hard to coordinate with each other.
As for “superintelligent AI being super enlightened and super capable and so fixing all of our complex problems” the superintelligent AI itself is to be aligned with human needs. Agent-4 from the Race Ending is NOT aligned to the humans. And that’s ignoring the possibility that the superintelligent AI who is super enlightened has a vision of mankind’s future which differs from the ideas of human hosts (e.g. Zuckerberg). I tried exploring the results of such a mismatch in my take at writing scenarios.