I would be curious to know more exactly what you view as an existential threat. I agree that in the sense of AI capability ARA is likely not a game changer (my expertise is limited here). I differ in that I expect agents as described in the original article to cause considerable damage to IT systems.
I work “on the ground” in cyber security and I am shocked again and again at how bad the security level is across industries. I certainly have a considerable bias as our customers often call when they were just or are in the process of being hacked. If I make allowance for that effect I would still estimate that at least two out of three companies cannot withstand a targeted attack by a medium-skilled attacker (Ransomware group with a few thousands to burn) for more than a few weeks. This estimate is based on manufacturing, public services, hospitals (in my experience among the worst).
For all these, the attacker does not need frontier-level models to cause severe damage. Today, most of the cyber crime activity is based on static scripts (sometimes with known hashes). This is largely sufficient for many infrastructures.
The original article showed a graphic of the systems and the vulnerability used to compromise them. I would rank the listed issues as easy to exploit and in this combination it would be a very insecure infrastructure. However, those issues are by no means unheard of in standard productive infrastructures and ransomware attacks.
I am not certain that ARA will explode the cyber crime sector (data management, extorsion, etc. all takes time and money), but there are similarities: Using compromised infrastructure to launch further attacks is a standard and highly automated procedure. Assuming an attacker who just wants to see the world burn, there is a real risk of widespread attacks with severe real world impact on the same scale as AGI—the decision makers may be human but the result might still be similar.
On the defense side, I do not see the majority of companies adopting AI to secure their systems proactively. Again, based on my experience, you don’t need frontier models, static scripts are sufficient to identify the most severe issues (e.g. Pingcastle for Active Directory). Still, IT departments rarely use it to audit and fix their own systems. I worry less about a zero-day vulnerability identified by mythos than about productive WinXP systems.
Does anybody know of a project for creating an AI-sysadmin that can analyse and patch systems/ configurations automatically while keeping the infrastructure operational? I know of project Glasswing but, as I understand it, it is focused on identifying new issues, not fixing old ones.
First and foremost: Thank you for the article! I am convinced that there should be more focus on practical enterprise security (this may be a personal bias since I work in that field).
Based on the IT-infrastructures I see and the available labor and skill to maintain them, I estimate that more than half of small and mid-level companies are not capable of withstanding an AI automated attack based on current models. I would guess (not really my area of expertise) that it may even be cost effective for attackers to use many small open source instead of Mythos-level models. For attackers, there may be no point in using/developing frontier models when the entire IT environment accross industries crumbles beforehand. What are your views about this?
Many writeups I see, both offensive and defensive, focus on highly skilled APTs and crazy new exploits. Most of what I see in real life is based on bad passwords, lack of MFA, and missing patches from 5-10 years ago (potential anti-survivorship bias).
Do you think it is beneficial to extend the space of “Found vulnerabilities” by another dimension “Consideration” where attackers wheaponize the vulnerability (more than just identification and PoC) and defenders patch it? This is currently sparse(ish) for attackers as the skillset required for identifying code malpractices is lower than identifying real exploitability and creating the corresponding PoC. (You address these points but is there any benefit in including these in the sparse sampling analogy?)
You point at a bumpy transition in 2026⁄27 in a heading. From what I see and what you mention about legacy systems, I would put it at a bumpy phase between 2026 and 2030, at least. Do I overstress a catchy headline here or is your point that anything beyond 2027 is hopeless?