Information security practice seems like it has a limited but real relevance to alignment. For a large fraction of alignment proposals, and frontier model practice, there is some practical aspect of the safety plan that literally depends on relevant ML infrastructure not having cybersecurity vulnerabilities. At minimum I think it’s worth doing for infosec people who are otherwise not going to be able to contribute to Agent Foundations or something, to raise the required IQ to discover and execute e.g. privilege escalation 0days on the kernels running RL loops.
My guess is that the most helpful way infosec practicioners could contribute would be to improve the sandboxing onion. I.E., hardening Linux, Kubernetes, Docker, Firecracker, etc., or coming up with new ways to run these models in controlled settings where it’d be harder to influence the outside world.
Information security practice seems like it has a limited but real relevance to alignment. For a large fraction of alignment proposals, and frontier model practice, there is some practical aspect of the safety plan that literally depends on relevant ML infrastructure not having cybersecurity vulnerabilities. At minimum I think it’s worth doing for infosec people who are otherwise not going to be able to contribute to Agent Foundations or something, to raise the required IQ to discover and execute e.g. privilege escalation 0days on the kernels running RL loops.
My guess is that the most helpful way infosec practicioners could contribute would be to improve the sandboxing onion. I.E., hardening Linux, Kubernetes, Docker, Firecracker, etc., or coming up with new ways to run these models in controlled settings where it’d be harder to influence the outside world.