In handwavey terms, I’m not surprised that the model learns the underlying philology, given typos. I don’t bother correcting “thnig” in text, nor in convos with LLMs, because I know the reader can descramble it. I wonder if this prompts learning the underlying semantic chunks?
Interesting!
In handwavey terms, I’m not surprised that the model learns the underlying philology, given typos. I don’t bother correcting “thnig” in text, nor in convos with LLMs, because I know the reader can descramble it. I wonder if this prompts learning the underlying semantic chunks?