I think the point isn’t to demonstrate a security mindset, because it’s obviously not optimally secure, but rather to point out that this architecture is not trivially easy to break, and it’s likely reasonably hard for AIs to self-exfiltrate themselves such that the owner doesn’t control the AI anymore.
I don’t find that satisfying. Anyone can point out that a perimeter is “reasonably hard” to breach by pointing at a high wall topped with barbed wire, and naive observers will absolutely agree that the wall sure is very high and sure is made of reinforced concrete.
The perimeter is still trivially easy to breach if, say, the front desk is susceptible to social engineering tactics.
Claiming that an architecture is even reasonably secure still requires looking at it with an attacker’s mindset. If you just look at the parts of the security you like, you can make a very convincing-sounding case that still misses glaring flaws. I’m not definitely saying that’s what this article does, but it sure is giving me this vibe.
I think the point isn’t to demonstrate a security mindset, because it’s obviously not optimally secure, but rather to point out that this architecture is not trivially easy to break, and it’s likely reasonably hard for AIs to self-exfiltrate themselves such that the owner doesn’t control the AI anymore.
I don’t find that satisfying. Anyone can point out that a perimeter is “reasonably hard” to breach by pointing at a high wall topped with barbed wire, and naive observers will absolutely agree that the wall sure is very high and sure is made of reinforced concrete.
The perimeter is still trivially easy to breach if, say, the front desk is susceptible to social engineering tactics.
Claiming that an architecture is even reasonably secure still requires looking at it with an attacker’s mindset. If you just look at the parts of the security you like, you can make a very convincing-sounding case that still misses glaring flaws. I’m not definitely saying that’s what this article does, but it sure is giving me this vibe.