But, a bit similarly to how high-level actions don’t screen off intent, text does not screen off thought. How you want to interpret and react to text, and how you want to interact with the person who published that text, depend on the process that produced the text.
While I am also irritated by AI-written text being sent to me unlabeled, I think this is just outright incorrect (or “proves too much”).
I think the “GAZP vs. GLUT” argument is exactly my complaint: it is not by random chance that those two texts were the same. Some process refined the two texts the same, and that process is what I care about.
I discuss something related in the post, and as I said, I agree that if in fact you check the LLM output really hard, in such a manner that you would actually change the text substantively on any of a dozen or a hundred points if the text was wrong, but you don’t change anything because it’s actually correct, then my objection is quantitatively lessened.
I do however think that there’s a bunch of really obvious ways that my argument does go through. People have given some examples in the comment, e.g. the LLM could tell a story that’s plausibly true, and happens to be actually true of some people, and some of those people generate that story with their LLM and post it. But I want to know who would generate that themselves without LLMs. (Also again in real life people would just present LLM’s testimony-lookalike text as though it is their testimony.) The issue with the GLUT is that it’s a huge amount of info, hence immensely improbable to generate randomly. An issue here is that text may have only a few bits of “relevant info”, so it’s not astronomically unlikely to generate a lookalike. Cf. Monty Hall problem; 1⁄3 or 2⁄3 or something of participants find themselves in a game-state where they actually need to know the algorithm that the host follows!
While I am also irritated by AI-written text being sent to me unlabeled, I think this is just outright incorrect (or “proves too much”).
I think the “GAZP vs. GLUT” argument is exactly my complaint: it is not by random chance that those two texts were the same. Some process refined the two texts the same, and that process is what I care about.
I discuss something related in the post, and as I said, I agree that if in fact you check the LLM output really hard, in such a manner that you would actually change the text substantively on any of a dozen or a hundred points if the text was wrong, but you don’t change anything because it’s actually correct, then my objection is quantitatively lessened.
I do however think that there’s a bunch of really obvious ways that my argument does go through. People have given some examples in the comment, e.g. the LLM could tell a story that’s plausibly true, and happens to be actually true of some people, and some of those people generate that story with their LLM and post it. But I want to know who would generate that themselves without LLMs. (Also again in real life people would just present LLM’s testimony-lookalike text as though it is their testimony.) The issue with the GLUT is that it’s a huge amount of info, hence immensely improbable to generate randomly. An issue here is that text may have only a few bits of “relevant info”, so it’s not astronomically unlikely to generate a lookalike. Cf. Monty Hall problem; 1⁄3 or 2⁄3 or something of participants find themselves in a game-state where they actually need to know the algorithm that the host follows!