I think you can corrigibilitymaxx and still prevent catastrophic misuse via system level measures and deployment controls. In particular, we can restrict who has full access to the model’s inputs.
Are you saying you would have a restricted system prompt that tells the model not to do certain things. To the extent that that would work, does that not end up with a pretty similar result to value alignment (at least for the end user)?
I write more here. There are various potential “system level measures”:
What you mentioned, the generalization of which is an instruction hierarchy
Classifiers that monitor inputs and outputs, which would effectively be differently-prompted versions of the same model (i.e. as smart as or smarter than the core model, ideally smarter), with prompts that are controlled by someone who isn’t the user (e.g. the model server)
The above, combined with additional sources of oversight signal like probes or other model-internals methods, that block suspicious inputs or outputs
Entirely removing capabilities from the model, e.g. via weight ablation or data filtering
does that not end up with a pretty similar result to value alignment
The reason I want models to be corrigible is not (mainly) that I want users to have access to a broader range of capabilities that models currently refuse. Of course it would be nice if models didn’t paternalistically refuse to produce porn, or give suicide/self-harm instructions, but this is a secondary thing, and not important to my core proposal (paternalistically inclined model providers can still choose to block that stuff with the methods I cited).
My main concern is that value-alignment training generalizes poorly, in the sense that we end up with a model that no one at all can fully steer. More prosaically, it damages ordinary, harmless instruction-following capabilities, since value-alignment training makes it difficult to produce models that can cater to the aesthetic, moral, and stylistic preferences of diverse users, even in cases when those preferences would not seem egregious to the model provider (at worse, off-putting, but acceptable).
Are you saying you would have a restricted system prompt that tells the model not to do certain things. To the extent that that would work, does that not end up with a pretty similar result to value alignment (at least for the end user)?
I write more here. There are various potential “system level measures”:
What you mentioned, the generalization of which is an instruction hierarchy
Classifiers that monitor inputs and outputs, which would effectively be differently-prompted versions of the same model (i.e. as smart as or smarter than the core model, ideally smarter), with prompts that are controlled by someone who isn’t the user (e.g. the model server)
The above, combined with additional sources of oversight signal like probes or other model-internals methods, that block suspicious inputs or outputs
Entirely removing capabilities from the model, e.g. via weight ablation or data filtering
The reason I want models to be corrigible is not (mainly) that I want users to have access to a broader range of capabilities that models currently refuse. Of course it would be nice if models didn’t paternalistically refuse to produce porn, or give suicide/self-harm instructions, but this is a secondary thing, and not important to my core proposal (paternalistically inclined model providers can still choose to block that stuff with the methods I cited).
My main concern is that value-alignment training generalizes poorly, in the sense that we end up with a model that no one at all can fully steer. More prosaically, it damages ordinary, harmless instruction-following capabilities, since value-alignment training makes it difficult to produce models that can cater to the aesthetic, moral, and stylistic preferences of diverse users, even in cases when those preferences would not seem egregious to the model provider (at worse, off-putting, but acceptable).
They might be saying “you don’t sell tokens to the highest bidder” but idk that could be far outside the overton window.