I think you can corrigibilitymaxx and still prevent catastrophic misuse via system level measures and deployment controls. In particular, we can restrict who has full access to the model’s inputs
This may be challenging in a world like ours with near-frontier open-source models serving a substantial fraction of all inference (I’m guessing open source already serves over 50 trillion tokens per day). In this case value alignment is the main safety property of the model.
Of course refusals can be fine-tuned away. This is a lot of friction that most people don’t bother with, although those aiming to cause catastrophic harm would be more motivated. You could also require open-source inference providers to add classifiers but users would always have the option of self-hosting.
Of course refusals can be fine-tuned away. This is a lot of friction that most people don’t bother with, although those aiming to cause catastrophic harm would be more motivated.
I’m confused about your point. What sorts of risks are you worried about that open-source models trained to refuse certain requests would mitigate as compared to corrigible open-source models (say, that are by default configured with a classifier attached). Of course in both cases a motivated person can use these for undesired purposes. And sure the everyday lazy person would have an easier time with the latter, but then again, what risks are we worried about from them? Maybe you can name some things but likely I’d consider that a minor cost to pay as compared to the benefit of widely-deployed obedient user-aligned AI that empowers a wide range of people with diverse preferences, aesthetics, and goals.
I agree that neither refusals or classifiers are much of a barrier to a sufficiently motivated actor, which matters the most for catastrophic misuse. However a classifier is far easier to remove.
Even if refusals don’t prevent catastrophic misuse, I would guess that model refusals would prevent the majority of potential harm. This is similar to locking my front door: in theory this shouldn’t make a difference, because a motivated criminal can go through the window. Yet in practice it substantially reduces crime.
Most users that want to do harm just aren’t that motivated. For example most people that attempt mass casualty events are incompetent and don’t put in much effort.
However there are various arguments against this:
If all open source inference providers adopt classifiers this would add a lot of friction for misuse
But open models which can be ran on a laptop will likely be quite good within a couple years
It’s easy to share models with refusal removal
So it’s difficult to say how this all shakes out. My narrower point is that model-level refusals and deployment classifiers aren’t equivalent with open source models.
Even if refusals don’t prevent catastrophic misuse, I would guess that model refusals would prevent the majority of potential harm.
I guess the crux is that I care less about this non-catastrophic harm compared to the benefits of focusing on intent alignment and corrigibility. I agree there’s a trade-off here but I am just taking a side on what’s overall best as per my worldview.
This may be challenging in a world like ours with near-frontier open-source models serving a substantial fraction of all inference (I’m guessing open source already serves over 50 trillion tokens per day). In this case value alignment is the main safety property of the model.
Of course refusals can be fine-tuned away. This is a lot of friction that most people don’t bother with, although those aiming to cause catastrophic harm would be more motivated. You could also require open-source inference providers to add classifiers but users would always have the option of self-hosting.
I’m confused about your point. What sorts of risks are you worried about that open-source models trained to refuse certain requests would mitigate as compared to corrigible open-source models (say, that are by default configured with a classifier attached). Of course in both cases a motivated person can use these for undesired purposes. And sure the everyday lazy person would have an easier time with the latter, but then again, what risks are we worried about from them? Maybe you can name some things but likely I’d consider that a minor cost to pay as compared to the benefit of widely-deployed obedient user-aligned AI that empowers a wide range of people with diverse preferences, aesthetics, and goals.
I agree that neither refusals or classifiers are much of a barrier to a sufficiently motivated actor, which matters the most for catastrophic misuse. However a classifier is far easier to remove.
Even if refusals don’t prevent catastrophic misuse, I would guess that model refusals would prevent the majority of potential harm. This is similar to locking my front door: in theory this shouldn’t make a difference, because a motivated criminal can go through the window. Yet in practice it substantially reduces crime.
Most users that want to do harm just aren’t that motivated. For example most people that attempt mass casualty events are incompetent and don’t put in much effort.
However there are various arguments against this:
If all open source inference providers adopt classifiers this would add a lot of friction for misuse
But open models which can be ran on a laptop will likely be quite good within a couple years
It’s easy to share models with refusal removal
So it’s difficult to say how this all shakes out. My narrower point is that model-level refusals and deployment classifiers aren’t equivalent with open source models.
I guess the crux is that I care less about this non-catastrophic harm compared to the benefits of focusing on intent alignment and corrigibility. I agree there’s a trade-off here but I am just taking a side on what’s overall best as per my worldview.
We’ll need to restrict proliferation before open-source models become truly dangerous.
Even if they’re value-aligned, it’s easy to remove that with fine-tuning if you’ve got open source weights.