I think it’s a fair question. Filtering on the canary string is neither necessary nor sufficient for not training on evals, so it’s tempting to just ignore it. I would personally also filter out docs with the canary string, but I’m not sure why they aren’t.
I think it’s a fair question. Filtering on the canary string is neither necessary nor sufficient for not training on evals, so it’s tempting to just ignore it. I would personally also filter out docs with the canary string, but I’m not sure why they aren’t.