It seems to me like both this post and discussion around Buck’s post are less about LLM generated content and more about lying.
Opus giving a verifiable mathematical counterexample is clearly not lying. Saying “I think” is on somewhat shakier but mostly fine ground. LLMs saying things like “When I was a teenager” when not editing a human’s account is clearly lying, and lying is bad no matter who does it, human or not. Extensively editing personal accounts indeed gets into very murky waters.
It seems to me like both this post and discussion around Buck’s post are less about LLM generated content and more about lying.
Opus giving a verifiable mathematical counterexample is clearly not lying. Saying “I think” is on somewhat shakier but mostly fine ground. LLMs saying things like “When I was a teenager” when not editing a human’s account is clearly lying, and lying is bad no matter who does it, human or not. Extensively editing personal accounts indeed gets into very murky waters.