Unfortunately with the facts as reported, none of this changes my priors. This is bad from every angle, particularly safety. Now instead of highly visible frontier labs, this angle is encouraging shadow development, splintering, and defensive obfuscation out of fear of rising to the point of government notice.
The winning move is now to stay under the radar, but that’s not at all the same as stopping. Control through fear is brittle.
This administration won’t be around forever, so the move here is to develop capabilities silently until this administration either collapses under its own weight or the normal political cycle progresses and a new administration comes in. Because this administration has gutted more or less everything regulatory and is allergic to continuity planning, the ideal time to launch a frontier model becomes any time there’s substantial political change. That’s when you’re most likely to get one past the goalie.
Surface pauses based on fear rather than principle are temporary and dangerous. They hold until they suddenly don’t. No matter what side of this you’re on, this action is not a cause for celebration.
Now instead of highly visible frontier labs, this angle is encouraging shadow development, splintering, and defensive obfuscation out of fear of rising to the point of government notice.
But that was always the case, no? Yet frontier labs chose to be visible anyway. Why should what happened change anything?
Like, IF:
You are building ASI.
You expect it to give you geopolitically relevant capabilities.
You don’t want to be nationalized and your AGI powers taken from you.
THEN:
You should make sure the government is as unaware of what you’re doing as possible.
This logical structure does not depend on what administration is in charge. Governments responding to ASI development this way is overdetermined. The only case where a government would not respond this way is if you live in a state so failed it’s given up even trying to govern.
It’s always been confusing to me that the LLM megacorps were so visible and frank about what they were doing. What was the reason for this? Main ones I could think of:
Was it because they actually wanted to be moral actors instead of Machiavellian schemers, wanted the public and governments to have awareness of what was going on, and implicitly consented to being regulated/nationalized eventually?
Was it because their needs for investment and talent-attraction were high enough that they had to grab attention any way they can, with no room to spare for long-term considerations?
Was it because they did not really believe in ASI and its geopolitical implications, or in their own ability to reach it?
Was it because they were stupid/blind, and just did not think through to that eventual endgame?
Well, my guess is that it’s some combination of all four.
In any case, I believe this event only changes the game inasmuch as (4) was the driving motivation behind the previous transparency. It doesn’t really change anything with regards to the other three.
So, to what extent was (4) the primary reason for transparency? Not sure. But we know that at least OpenAI’s leadership explicitly understood the geopolitical implications, and explicitly aimed towards developing the geopolitically relevant thing.
Perhaps they, and other lab CEOs, were thinking about it in overly far-mode terms, and failed to properly track the implications for their actions in the present (especially as those implications conflicted with near-term goals of raising money and attracting talent, so there was pressure to rationalize them away). In this case, sure, I guess this is the wake-up call for them to go into stealth mode.
Largely I don’t think our positions are very far apart on what materially matters here because the crux of what I’m saying doesn’t depend on who was pursuing any particular strategy before this.
I’ll grant that there was nothing preventing this strategy prior to current events; given Anthropic’s commitment to Glasswing secrecy doesn’t seem to be the primary strategy they were employing. To be honest, I don’t know that it matters all that much for the forward-facing implications.
Regardless of what strategy any given actor was pursuing prior, the point remains: we now know more about the state of play than we did before these events. We now know we’re in the world in which an active authority has unilaterally and arbitrarily imposed restrictions on a disfavored actor for actions unrelated to safety.
Daddy just came home drunk, probably a good idea to hide in the bedroom until he sobers up.
This implies a likely 2.5 yr time for ‘develop capabilities silently’ during which apparently even internal deployments are subject to random restrictions. That’s a long time for a period in which ability to raise money, backed by rising revenue as products improve, is otherwise a huge factor in rate of advancement.
apparently even internal deployments are subject to random restrictions
The government has yet to demonstrate the ability to restrict internal deployments of unnamed models (not literally unnamed, but without publicly facing names).
It would not be difficult to fine-tune or modify a model a bit and have a “formally defensible reason” to call it something else.
Of course, if the government really wants to control this kind of thing (at least for large and well visible US-based corporations), it can likely do so, but that would take more than serving export control orders.
Midterms are this November and given the political headwinds favoring substantial changes in the composition/control of Congress it looks like that’s the first major inflection opportunity.
Unfortunately with the facts as reported, none of this changes my priors. This is bad from every angle, particularly safety. Now instead of highly visible frontier labs, this angle is encouraging shadow development, splintering, and defensive obfuscation out of fear of rising to the point of government notice.
The winning move is now to stay under the radar, but that’s not at all the same as stopping. Control through fear is brittle.
This administration won’t be around forever, so the move here is to develop capabilities silently until this administration either collapses under its own weight or the normal political cycle progresses and a new administration comes in. Because this administration has gutted more or less everything regulatory and is allergic to continuity planning, the ideal time to launch a frontier model becomes any time there’s substantial political change. That’s when you’re most likely to get one past the goalie.
Surface pauses based on fear rather than principle are temporary and dangerous. They hold until they suddenly don’t. No matter what side of this you’re on, this action is not a cause for celebration.
I’m just going to leave this here.
But that was always the case, no? Yet frontier labs chose to be visible anyway. Why should what happened change anything?
Like, IF:
You are building ASI.
You expect it to give you geopolitically relevant capabilities.
You don’t want to be nationalized and your AGI powers taken from you.
THEN:
You should make sure the government is as unaware of what you’re doing as possible.
This logical structure does not depend on what administration is in charge. Governments responding to ASI development this way is overdetermined. The only case where a government would not respond this way is if you live in a state so failed it’s given up even trying to govern.
It’s always been confusing to me that the LLM megacorps were so visible and frank about what they were doing. What was the reason for this? Main ones I could think of:
Was it because they actually wanted to be moral actors instead of Machiavellian schemers, wanted the public and governments to have awareness of what was going on, and implicitly consented to being regulated/nationalized eventually?
Was it because their needs for investment and talent-attraction were high enough that they had to grab attention any way they can, with no room to spare for long-term considerations?
Was it because they did not really believe in ASI and its geopolitical implications, or in their own ability to reach it?
Was it because they were stupid/blind, and just did not think through to that eventual endgame?
Well, my guess is that it’s some combination of all four.
In any case, I believe this event only changes the game inasmuch as (4) was the driving motivation behind the previous transparency. It doesn’t really change anything with regards to the other three.
So, to what extent was (4) the primary reason for transparency? Not sure. But we know that at least OpenAI’s leadership explicitly understood the geopolitical implications, and explicitly aimed towards developing the geopolitically relevant thing.
Perhaps they, and other lab CEOs, were thinking about it in overly far-mode terms, and failed to properly track the implications for their actions in the present (especially as those implications conflicted with near-term goals of raising money and attracting talent, so there was pressure to rationalize them away). In this case, sure, I guess this is the wake-up call for them to go into stealth mode.
Largely I don’t think our positions are very far apart on what materially matters here because the crux of what I’m saying doesn’t depend on who was pursuing any particular strategy before this.
I’ll grant that there was nothing preventing this strategy prior to current events; given Anthropic’s commitment to Glasswing secrecy doesn’t seem to be the primary strategy they were employing. To be honest, I don’t know that it matters all that much for the forward-facing implications.
Regardless of what strategy any given actor was pursuing prior, the point remains: we now know more about the state of play than we did before these events. We now know we’re in the world in which an active authority has unilaterally and arbitrarily imposed restrictions on a disfavored actor for actions unrelated to safety.
Daddy just came home drunk, probably a good idea to hide in the bedroom until he sobers up.
This implies a likely 2.5 yr time for ‘develop capabilities silently’ during which apparently even internal deployments are subject to random restrictions. That’s a long time for a period in which ability to raise money, backed by rising revenue as products improve, is otherwise a huge factor in rate of advancement.
The government has yet to demonstrate the ability to restrict internal deployments of unnamed models (not literally unnamed, but without publicly facing names).
It would not be difficult to fine-tune or modify a model a bit and have a “formally defensible reason” to call it something else.
Of course, if the government really wants to control this kind of thing (at least for large and well visible US-based corporations), it can likely do so, but that would take more than serving export control orders.
Midterms are this November and given the political headwinds favoring substantial changes in the composition/control of Congress it looks like that’s the first major inflection opportunity.