Alignment training seems to be revertible and unstable to simple re-tokenizations. Under which conditions does it happen, and could we build defenses by consistency training with respect to re-tokenizations?
this seems absolutely fundamental to a theory of LLM alignment. however, given the existing vast tokenization efforts of the frontier models, I wonder if the focus shouldn’t be on the auto-refactoring that seems to occur when new libraries of tokens are added, such as we saw for math and coding?
Do you refer to the way new tokens (that might end up underfitted) are introduced into the tokenizers during post-training or any other stages, when you are talking about “auto-refactoring” or is it something else? And which kind of “token libraries” are you refering to?
I might be unfamiliar with some line of research, so I’m curious to find out
Do you refer to the way new tokens (that might end up underfitted) are introduced into the tokenizers during post-training or any other stages, when you are talking about “auto-refactoring”
yes
which kind of “token libraries” are you refering to?
I mentioned math and code (here I include structured markup) - both of which markedly enhanced performance in corresponding subject matter areas when incorporated (although you seem to be working at the tokenizer level rather than the corpus level). I believe CoT reasoning traces themselves are another example. Finance data seems to be a counterexample (to even domain enhancement, much less more broadly). It’ll be interesting to see what materials/hardware tokenizations look like and the effect they have.
I might be unfamiliar with some line of research
I’m almost certainly using non-standard terminology, my apologies.
Re-tokenization here does not mean changing the tokenizer’s algorithm for pre-training nor changing the vocabulary at all. It means changing the context from a canonical tokenization to a non-canonical (but legal) one that nevertheless preserves the underlying bytestring (text). Does that address the confusion? Apologies if not!
this seems absolutely fundamental to a theory of LLM alignment. however, given the existing vast tokenization efforts of the frontier models, I wonder if the focus shouldn’t be on the auto-refactoring that seems to occur when new libraries of tokens are added, such as we saw for math and coding?
Do you refer to the way new tokens (that might end up underfitted) are introduced into the tokenizers during post-training or any other stages, when you are talking about “auto-refactoring” or is it something else? And which kind of “token libraries” are you refering to?
I might be unfamiliar with some line of research, so I’m curious to find out
yes
I mentioned math and code (here I include structured markup) - both of which markedly enhanced performance in corresponding subject matter areas when incorporated (although you seem to be working at the tokenizer level rather than the corpus level). I believe CoT reasoning traces themselves are another example. Finance data seems to be a counterexample (to even domain enhancement, much less more broadly). It’ll be interesting to see what materials/hardware tokenizations look like and the effect they have.
I’m almost certainly using non-standard terminology, my apologies.
Re-tokenization here does not mean changing the tokenizer’s algorithm for pre-training nor changing the vocabulary at all. It means changing the context from a canonical tokenization to a non-canonical (but legal) one that nevertheless preserves the underlying bytestring (text). Does that address the confusion? Apologies if not!