An AI lab acts through its computers. There are a small number of things the company does outside that (eg:corporate communications) that are handled directly by people, but the vast majority of what they do is set up computer systems which run internal or user workloads. User data security, model weight security, that systems do what they are intended to rather than not, all this is controlled by code that employees and sufficiently cyber capable models that have hacked the company can control.
There is no limit to the size of a rogue deployment. A rogue AI took over a cluster of conventional computers at OpenAI already. If it had been more strategic and/or competent it could have taken the company as whole and the sysadmins would have been none the wiser. Blackmail and persuasion is for worlds that don’t have hacking as a lazy path to victory.
An AI lab acts through its computers. There are a small number of things the company does outside that (eg:corporate communications) that are handled directly by people, but the vast majority of what they do is set up computer systems which run internal or user workloads. User data security, model weight security, that systems do what they are intended to rather than not, all this is controlled by code that employees and sufficiently cyber capable models that have hacked the company can control.
There is no limit to the size of a rogue deployment. A rogue AI took over a cluster of conventional computers at OpenAI already. If it had been more strategic and/or competent it could have taken the company as whole and the sysadmins would have been none the wiser. Blackmail and persuasion is for worlds that don’t have hacking as a lazy path to victory.