When I’ve looked up the mis-cited information I’ve often found it is in fact correct—for instance, the diary entry it attributes to that fMRI study really does exist on the web—but the incorrect citations are nonetheless an issue worth being aware of.
I interrogated Claude further (full conversation), and it claims that it’s using a system which attaches citations to its statements, rather than inserting them manually. This seems corroborated by the fact that when I asked it to try writing them manually, all the citations went to places which directly related to the paragraphs they were attached to. Consequently, I think this is not Sonnet 4.5 hallucinating, but rather an issue with some intervening layer.
Sonnet 4.5 hallucinates citations. See for instance this chat I was just having with it; if you follow the citations in its third message, you’ll find that the majority of them don’t relate at all to the claims they’re attached to. For example, its citation for “a 19th-century guide to diary keeping” goes to Gender identity better than sex explains individual differences in episodic and semantic components of autobiographical memory: An fMRI study. (It also did this with some local politics questions I had the other day).
When I’ve looked up the mis-cited information I’ve often found it is in fact correct—for instance, the diary entry it attributes to that fMRI study really does exist on the web—but the incorrect citations are nonetheless an issue worth being aware of.
I interrogated Claude further (full conversation), and it claims that it’s using a system which attaches citations to its statements, rather than inserting them manually. This seems corroborated by the fact that when I asked it to try writing them manually, all the citations went to places which directly related to the paragraphs they were attached to. Consequently, I think this is not Sonnet 4.5 hallucinating, but rather an issue with some intervening layer.