If we have CoTness detection, then it should not be very hard to have an additional layer that would validate if there are CoT-like regions inside the user message or the tool output and inform the user about a message rejection because of a security issue? The problem will persist for open models, though—as those might be easily stripped of the additional security feature, as this would be external to the model.
If we have CoTness detection, then it should not be very hard to have an additional layer that would validate if there are CoT-like regions inside the user message or the tool output and inform the user about a message rejection because of a security issue? The problem will persist for open models, though—as those might be easily stripped of the additional security feature, as this would be external to the model.