Is latching onto some ‘handrail’ phrase and then remixing it a common thing for LLMs to do when they are learning these kinds of tasks?
Neural machine translation is prone to some pretty strange hallucinated phrases and translations. (Google Translate, because it’s used at such scale, has offered many examples of translations where a single letter somehow turns into a sentence, or it keeps expanding a translation.)
I don’t know that these anomalies tend to come from anywhere in particular. (I took a quick look and aside from one instance of a magazine ad for a Morse code flashcard which used ‘fish’ as a mnemonic for ‘f’, and a suspicious level of Boy Scouts hits, didn’t find a candidate for this either.)
I was also thinking about Translate—another example from them is that in some languages, our first shot at using Transformers would sometimes translate queries as “wikipedia wikipedia wikipedia”, just a list of that word in some number, I guess because it’s a super common word that shows up in web text. It would get stuck where “wikipedia” was always the most likely next token.
I also haven’t heard a good theory about what exactly is going wrong there.
Neural machine translation is prone to some pretty strange hallucinated phrases and translations. (Google Translate, because it’s used at such scale, has offered many examples of translations where a single letter somehow turns into a sentence, or it keeps expanding a translation.)
I don’t know that these anomalies tend to come from anywhere in particular. (I took a quick look and aside from one instance of a magazine ad for a Morse code flashcard which used ‘fish’ as a mnemonic for ‘f’, and a suspicious level of Boy Scouts hits, didn’t find a candidate for this either.)
I was also thinking about Translate—another example from them is that in some languages, our first shot at using Transformers would sometimes translate queries as “wikipedia wikipedia wikipedia”, just a list of that word in some number, I guess because it’s a super common word that shows up in web text. It would get stuck where “wikipedia” was always the most likely next token.
I also haven’t heard a good theory about what exactly is going wrong there.