2. The safety stack travels with the model. The Department was not asking us to modify how our models behave. Their position was, build the model however you want, refuse whatever requests you want, just don’t try to govern our operational decisions through usage policies.
This is a very confusing stance to me.
Is this a lie? Or do some of the relevant parties (namely the DoW) have a very different impression of the situation than I do?
Shouldn’t trained-in refusals matter at least as much to operational decisions as contractual agreements?
Is it just that contractual agreements feel like an affront to their authority, where trained-in model refusals feel like technical nerd shit? It seems like asking Claude / GPT to do something, and having it say “sorry, I won’t” is much more annoying than contract terms?
Maybe they think it’s much easier to force Anthropic to modify Claude’s behavior
I mean, I think they might be laboring under a misapprehension about what kind of technology AI is. Like, they might think that Claude and GPT have a dashboard with a bunch of dials, and that the companies can just turn off the guardrails, or something like that, rather than the AIs being artifacts that have (mostly) their own properties and dispositions once they’re trained.
If you thought AI was that kind of thing, you might (mistakenly?) think that the contractual terms were more binding than the technical guardrails, because you think that you can just demand that the company turn off the guardrails, later.
Then again, maybe the “technical stack” that OpenAI has in mind is the kind of thing that can be just turned off.
This is a very confusing stance to me.
Is this a lie? Or do some of the relevant parties (namely the DoW) have a very different impression of the situation than I do?
Shouldn’t trained-in refusals matter at least as much to operational decisions as contractual agreements?
Is it just that contractual agreements feel like an affront to their authority, where trained-in model refusals feel like technical nerd shit? It seems like asking Claude / GPT to do something, and having it say “sorry, I won’t” is much more annoying than contract terms?
Is there something that I’m missing here?
Maybe they’re confident that they can jailbreak Claude?
Maybe they think it’s much easier to force Anthropic to modify Claude’s behavior than it is to modify the cotract?
But I don’t see why they’d be confident about either of those.
I mean, I think they might be laboring under a misapprehension about what kind of technology AI is. Like, they might think that Claude and GPT have a dashboard with a bunch of dials, and that the companies can just turn off the guardrails, or something like that, rather than the AIs being artifacts that have (mostly) their own properties and dispositions once they’re trained.
If you thought AI was that kind of thing, you might (mistakenly?) think that the contractual terms were more binding than the technical guardrails, because you think that you can just demand that the company turn off the guardrails, later.
Then again, maybe the “technical stack” that OpenAI has in mind is the kind of thing that can be just turned off.