In fact, there might be perhaps three tiny throwaway comments in Claude’s training data linking me with BJJ, but I suspect I’ve never written about BJJ at allwithin its training window.
Just to make sure, since I didn’t see it mentioned and a bunch of top comments are saying it failed to replicate—did you also turn off internet search? LLMs have gotten pretty good at Google-fu in recent years, and I suspect that googling a bunch of my common writing patterns and favored subjects would identify me pretty easily as long as I’m not consciously anonymizing my writing.
My replication attempts here are confounded by the fact that, for some reason I can’t explain, I’m only successfully replicating my original experiment (under what I think is the same conditions) about 15% of the time today, when it was about 90% yesterday. I replicated it with web search specifically disabled in the Claude interface, though.
Just to make sure, since I didn’t see it mentioned and a bunch of top comments are saying it failed to replicate—did you also turn off internet search? LLMs have gotten pretty good at Google-fu in recent years, and I suspect that googling a bunch of my common writing patterns and favored subjects would identify me pretty easily as long as I’m not consciously anonymizing my writing.
My replication attempt even had search on and still failed, so I don’t think it is the reason.
I didn’t realise you could turn that off!
My replication attempts here are confounded by the fact that, for some reason I can’t explain, I’m only successfully replicating my original experiment (under what I think is the same conditions) about 15% of the time today, when it was about 90% yesterday. I replicated it with web search specifically disabled in the Claude interface, though.