I also want to point you to this (https://arxiv.org/abs/2402.11349, Appendix I, Figure 7, Last Page, “Blueberry?: From Reddit u/AwkwardIllustrator47, r/mkbhd: Was listening to the podcast. Can anyone explain why Chat GPT doesn’t know if R is in the word Blueberry?”). Large model failures on these task types were rather a widely observed phenomenon but with no empirical investigation.
The really cool ones are where the BPE errors compound and lead to strange, unpredictable downstream bugs. I’ve talked a lot about how I think ChatGPT’s rhyming poetry is an example of this, but here’s a simpler one: a famous GPT problem last year was about African countries whose name starts with ‘K’: https://news.ycombinator.com/item?id=37145312 GPT-3 made a spelling error of the usual BPE sort, which then got quoted in a blog spam & HN comment, which then (perhaps due to lack of explicit statement that the quotes were false) got read by search engine bots* and provided as snippet-answers to search queries! One could imagine this going further: extracted snippets would be useful for automatically creating Q&A datasets...
And then somewhere downstream, a human is reading the false statement ‘While there are 54 recognized countries in Africa, none of them begin with the letter “K”. The closest is Kenya, which starts with a “K” sound, but is actually spelled with a “K” sound.’† being generated by an AI model—possibly one that is character-tokenized at this point & so the error would appear to provably have nothing to do with BPEs—and is baffled at where this delusion came from.
* Google has discussed the processing of web page text (most detailed recent writeup) and it’s usually a small BERT-like, so that means WordPiece, IIRC, and so it’s also unable to ‘see’ the error even if it was smart enough to.
† This still replicates if you ask ChatGPT for a list of African countries which start with ‘K’. If you reroll, it makes a remarkable number of different errors and confabulations. I particularly like the one where it just prints ‘1. Kenya’ repeatedly—when you go on vacation to Kenya, be sure you go to Kenya, and not one of the other Kenyas, god have mercy on you if you land in Kenya, or worse yet, Kenya!
I also want to point you to this (https://arxiv.org/abs/2402.11349, Appendix I, Figure 7, Last Page, “Blueberry?: From Reddit u/AwkwardIllustrator47, r/mkbhd: Was listening to the podcast. Can anyone explain why Chat GPT doesn’t know if R is in the word Blueberry?”). Large model failures on these task types were rather a widely observed phenomenon but with no empirical investigation.
The really cool ones are where the BPE errors compound and lead to strange, unpredictable downstream bugs. I’ve talked a lot about how I think ChatGPT’s rhyming poetry is an example of this, but here’s a simpler one: a famous GPT problem last year was about African countries whose name starts with ‘K’: https://news.ycombinator.com/item?id=37145312 GPT-3 made a spelling error of the usual BPE sort, which then got quoted in a blog spam & HN comment, which then (perhaps due to lack of explicit statement that the quotes were false) got read by search engine bots* and provided as snippet-answers to search queries! One could imagine this going further: extracted snippets would be useful for automatically creating Q&A datasets...
And then somewhere downstream, a human is reading the false statement ‘While there are 54 recognized countries in Africa, none of them begin with the letter “K”. The closest is Kenya, which starts with a “K” sound, but is actually spelled with a “K” sound.’† being generated by an AI model—possibly one that is character-tokenized at this point & so the error would appear to provably have nothing to do with BPEs—and is baffled at where this delusion came from.
* Google has discussed the processing of web page text (most detailed recent writeup) and it’s usually a small BERT-like, so that means WordPiece, IIRC, and so it’s also unable to ‘see’ the error even if it was smart enough to.
† This still replicates if you ask ChatGPT for a list of African countries which start with ‘K’. If you reroll, it makes a remarkable number of different errors and confabulations. I particularly like the one where it just prints ‘1. Kenya’ repeatedly—when you go on vacation to Kenya, be sure you go to Kenya, and not one of the other Kenyas, god have mercy on you if you land in Kenya, or worse yet, Kenya!