I don’t think adding a system prompt is an example of this strategy. Training the system prompt into the model probably produces better results than adding it after the fact, unlike the other methods you mentioned.
This isn’t necessarily true. If you’re doing RL against a misspecified reward function (that reinforces dishonesty, for example), then you might prefer to add a “be honest” system prompt in deployment, but not during training. If you add the prompt during training, its effect might be removed by RL.
If you mean to say that you could train the system prompt into the model after RL, I agree that is possible, but it is in fact the case that AI developers use system prompts, so it is worth considering them anyway.
(Also, to be pedantic: whether or not a method is effective is irrelevant, since effectiveness isn’t a criterion for train-deploy mismatch.)
I don’t think adding a system prompt is an example of this strategy. Training the system prompt into the model probably produces better results than adding it after the fact, unlike the other methods you mentioned.
This isn’t necessarily true. If you’re doing RL against a misspecified reward function (that reinforces dishonesty, for example), then you might prefer to add a “be honest” system prompt in deployment, but not during training. If you add the prompt during training, its effect might be removed by RL.
If you mean to say that you could train the system prompt into the model after RL, I agree that is possible, but it is in fact the case that AI developers use system prompts, so it is worth considering them anyway.
(Also, to be pedantic: whether or not a method is effective is irrelevant, since effectiveness isn’t a criterion for train-deploy mismatch.)