Every now and then I’ve asked AIs to “name as many characters as you can from [moderately obscure game/story]”. So far I’ve never had one fail to hallucinate extra characters, or fail to double down when I ask for more details about its creations.
Well spotted. I had a similar thought recently, that the implications or details of rarely read books are one of the remaining gaps in AI knowledge. This is because it’s not just spelt out in the text, you have to understand the details and think about them. Current training methods don’t process texts that deeply, and if it’s a rare book, there won’t be essays spelling out the lore anywhere in the training corpus.
Every now and then I’ve asked AIs to “name as many characters as you can from [moderately obscure game/story]”. So far I’ve never had one fail to hallucinate extra characters, or fail to double down when I ask for more details about its creations.
Without search right? I would bet you mean without search but probably best to spell it out.
Tried with search just now and ChatGPT at least no longer displayed this failure mode.
Well spotted. I had a similar thought recently, that the implications or details of rarely read books are one of the remaining gaps in AI knowledge. This is because it’s not just spelt out in the text, you have to understand the details and think about them. Current training methods don’t process texts that deeply, and if it’s a rare book, there won’t be essays spelling out the lore anywhere in the training corpus.