I think I’d expect it to play out as the last option you suggest, e.g. attacking important infrastructure at an opportune moment. I wrote a theory of how this could play out on here at the weekend if you check my post history, but I’d encourage you to read Forethought’s report, as they have the most developed thinking on this.
callumzc
Karma: 22
Two quick areas I’d love to get others thoughts on:
1) If we accept that Anthropic will decide who should access powerful models before their release – sidestepping whether they should – then what criteria should they use to inform their decision? Should a first criterion be to distribute access in ways that defends democratic regimes?
I wrote about Mythos/Glasswing and AI-enabled coups here a few days ago. The next step to that argument could be that, as Anthropic are now the effective arbiter of which actors can or cannot defend critical infrastructure, they have acquired a form of structural geopolitical power that would historically trigger obligations to uphold international governance norms. This is similar to UN principles that establish corporations have a responsibility to avoid contributing to human rights violations; contributing to coup risk could plausibly fall within that frame.
This is an incredibly dicey criterion to enact. The history of US government agencies and corporations picking winners in contested regimes is an infamous one, so I wonder if there is a way to frame the obligation in a sufficiently narrow sense that it doesn’t encourage Anthropic to do this. One option could be to not make access-control decisions a la Glasswing in ways that predictably disadvantage the defenders of democratic institutions.
I still find this pretty unsatisfying—I wonder if anyone has thoughts on how this could work. I’m setting aside arguments that Anthropic shouldn’t exercise this power, since this is what they are doing in practice. I would love to hear what criteria might make them exercise this power in the most sensible manner.
2) What characteristics does a state particularly vulnerable to an AI-enabled coup have? Some quick takes below but interested in other ideas.
1. Weak state capacity (state has fewer resources to respond to a coordinated attack).2. Concentrated digital infrastructure (eg reliance on a small number of critical nodes that are easier to target, possibly a small number of foreign systems).
3. Weak state control or civilian control of the military, so that pressure can exploit existing fragmentation.
4. High political polarisation and instability (existing tensions to exploit, more likely that a live political crisis can be exploited)
I’d really like to develop better-evidenced characteristics, as my initial sense it that it could be mid-tier modernised democracies that are more at risk to this threat than failed states. I’d also imagine these states would be excluded from involvement in any governance arrangement that resembles Glasswing.
This is what I’d expect to see happen. I do wonder exactly what variables you might use to determine who is most at risk. Intuitively, this might be a mix of
1) Weak state capacity.
2) Concentrated digital infrastructure (eg reliance on a small number of critical nodes that are easier to target, possibly a small number of foreign systems).
3) Weak state control or civilian control of the military, so that pressure can exploit existing fragmentation.
I can’t tell you exactly which countries fit the bill but I can guarantee their government has no relationship to frontier labs.
Good read. Anecdotally I’ve had a lot of traction from UK policymakers when discussing the broad argument that we need to be bullish on AI growth as a matter of national security, but I’ve struggled a lot more to get engagement on more practical things like building data centres.
I’d like to see our government frame the argument for building this sort of growth-enabling infrastructure in national security terms. For example, I thought that the ‘levelling up’ style comms push around AI Growth Zones in Wales contributed to the perception that we’re looking to build infrastructure there for some soft reason related to regional inequality, rather than those areas being well-suited to strategically important infrastructure.
(This is quite tangential to the piece but I have lots of practical thoughts about how you would push for this in the UK specific context that I’d love to scheme out if anyone else reading this works in the field).