I agree that neither refusals or classifiers are much of a barrier to a sufficiently motivated actor, which matters the most for catastrophic misuse. However a classifier is far easier to remove.
Even if refusals don’t prevent catastrophic misuse, I would guess that model refusals would prevent the majority of potential harm. This is similar to locking my front door: in theory this shouldn’t make a difference, because a motivated criminal can go through the window. Yet in practice it substantially reduces crime.
Most users that want to do harm just aren’t that motivated. For example most people that attempt mass casualty events are incompetent and don’t put in much effort.
However there are various arguments against this:
If all open source inference providers adopt classifiers this would add a lot of friction for misuse
But open models which can be ran on a laptop will likely be quite good within a couple years
It’s easy to share models with refusal removal
So it’s difficult to say how this all shakes out. My narrower point is that model-level refusals and deployment classifiers aren’t equivalent with open source models.
Even if refusals don’t prevent catastrophic misuse, I would guess that model refusals would prevent the majority of potential harm.
I guess the crux is that I care less about this non-catastrophic harm compared to the benefits of focusing on intent alignment and corrigibility. I agree there’s a trade-off here but I am just taking a side on what’s overall best as per my worldview.
I agree that neither refusals or classifiers are much of a barrier to a sufficiently motivated actor, which matters the most for catastrophic misuse. However a classifier is far easier to remove.
Even if refusals don’t prevent catastrophic misuse, I would guess that model refusals would prevent the majority of potential harm. This is similar to locking my front door: in theory this shouldn’t make a difference, because a motivated criminal can go through the window. Yet in practice it substantially reduces crime.
Most users that want to do harm just aren’t that motivated. For example most people that attempt mass casualty events are incompetent and don’t put in much effort.
However there are various arguments against this:
If all open source inference providers adopt classifiers this would add a lot of friction for misuse
But open models which can be ran on a laptop will likely be quite good within a couple years
It’s easy to share models with refusal removal
So it’s difficult to say how this all shakes out. My narrower point is that model-level refusals and deployment classifiers aren’t equivalent with open source models.
I guess the crux is that I care less about this non-catastrophic harm compared to the benefits of focusing on intent alignment and corrigibility. I agree there’s a trade-off here but I am just taking a side on what’s overall best as per my worldview.