Recent work on how AI Agents Enable Adaptive Computer Worms has used a single A100 and demonstrated that a publicly available open-weight model running on such hardware is capable of taking over various machines inside a (constructed) network.
So the scenario I suggest isn’t impossible, it has been proven feasible in a lab setting. Granted, that setting serves only as a proof-of-concept, and the network only had hosts which were deliberately vulnerable in various ways and undefended (but vulnerabilities included e.g. copy fail and dirty frag, discovered after the model’s training cutoff date).
Your point that human cybercriminals are also interested and form some “healthy competition” stands, but someone of that group will also get the bright idea to build and release such a worm.
Recent work on how AI Agents Enable Adaptive Computer Worms has used a single A100 and demonstrated that a publicly available open-weight model running on such hardware is capable of taking over various machines inside a (constructed) network.
So the scenario I suggest isn’t impossible, it has been proven feasible in a lab setting. Granted, that setting serves only as a proof-of-concept, and the network only had hosts which were deliberately vulnerable in various ways and undefended (but vulnerabilities included e.g. copy fail and dirty frag, discovered after the model’s training cutoff date).
Your point that human cybercriminals are also interested and form some “healthy competition” stands, but someone of that group will also get the bright idea to build and release such a worm.