This isn’t necessarily true. If you’re doing RL against a misspecified reward function (that reinforces dishonesty, for example), then you might prefer to add a “be honest” system prompt in deployment, but not during training. If you add the prompt during training, its effect might be removed by RL.
If you mean to say that you could train the system prompt into the model after RL, I agree that is possible, but it is in fact the case that AI developers use system prompts, so it is worth considering them anyway.
(Also, to be pedantic: whether or not a method is effective is irrelevant, since effectiveness isn’t a criterion for train-deploy mismatch.)
This isn’t necessarily true. If you’re doing RL against a misspecified reward function (that reinforces dishonesty, for example), then you might prefer to add a “be honest” system prompt in deployment, but not during training. If you add the prompt during training, its effect might be removed by RL.
If you mean to say that you could train the system prompt into the model after RL, I agree that is possible, but it is in fact the case that AI developers use system prompts, so it is worth considering them anyway.
(Also, to be pedantic: whether or not a method is effective is irrelevant, since effectiveness isn’t a criterion for train-deploy mismatch.)