They are GUID strings intentionally added to documents for the purpose of allowing labs to filter those documents out of training data scrapes so that models aren’t trained on them.
As far as I’m aware, they have been used for evals, as well as some AI research and data that researchers fear would cause “misalignment” if AIs read them in pretraining.
There is a canary string specific to alignment faking data, though it’s also been used for other misalignment-related research and data, such as the paper about emergent misalignment from reward hacking. You can find it at the bottom of the readme here (I won’t post it directly so that it doesn’t affect this post). Searching for this string on a search engine is also a way to find some interesting content.
Alphanumeric codes in the text of a web page or a document that mean “do not include this text in training set of an AI”. Should be trivial to filter out such texts from training… if the companies care.
The strings themselves are arbitrary, it’s like agreeing that e.g. “FOO123” is the symbol, and if you want a text to be excluded, just write “FOO123″ anywhere in the text. But they are much longer and random, so there is no chance of someone typing them by accident.
(I am not giving a specific example, because that would needlessly exclude this web page.)
What is “canary strings”?
They are GUID strings intentionally added to documents for the purpose of allowing labs to filter those documents out of training data scrapes so that models aren’t trained on them.
As far as I’m aware, they have been used for evals, as well as some AI research and data that researchers fear would cause “misalignment” if AIs read them in pretraining.
There is a canary string specific to alignment faking data, though it’s also been used for other misalignment-related research and data, such as the paper about emergent misalignment from reward hacking. You can find it at the bottom of the readme here (I won’t post it directly so that it doesn’t affect this post). Searching for this string on a search engine is also a way to find some interesting content.
Alphanumeric codes in the text of a web page or a document that mean “do not include this text in training set of an AI”. Should be trivial to filter out such texts from training… if the companies care.
The strings themselves are arbitrary, it’s like agreeing that e.g. “FOO123” is the symbol, and if you want a text to be excluded, just write “FOO123″ anywhere in the text. But they are much longer and random, so there is no chance of someone typing them by accident.
(I am not giving a specific example, because that would needlessly exclude this web page.)