I think that referring to LLMs at all in this post is a red herring. The post should simply say, “Don’t cite dubious sources without checking them out.” The end. Doesn’t matter whether the sources are humans or LLMs. I consider most recent LLMs more-reliable than most people. Not because they’re reliable; because human reliability is a very low bar to clear.
The main point of my 1998 post “Believable Stupidity” was that the worst failure modes of AI dialogue are also failure modes of human dialogue. This is even more true today. I think humans still produce more hallucinatory dialogue than LLMs. Some I dealt with last month:
the millionaire white male Ivy-league grad who accused me of disagreeing with his revolutionary anti-capitalist politics because I’m privileged and well-off, even though he knows I’ve been unemployed for years, while he just got his third start-up funded and was about to buy a $600K house
friends claiming that protestors who, on video, attacked a man from several sides before he turned on them, did not attack him, but were minding their own business when he attacked them
my fundamentalist Christian mother, who knows I think Christianity is completely false, keeps quoting the Psalms to me, and is always surprised when I don’t call them beautiful and wise
These are the same sort of hallucinations as those produced by LLMs when some keyword or over-trained belief spawns a train of thought which goes completely off the rails of reality.
Consider the notion of “performativity”, usually attributed to the Nazi activist Heidegger. This is the idea that the purpose of much speech is not to communicate information, but to perform an action, and especially to enact an identity such as a gender role or a political affiliation.
In 1930s Germany, this manifested as a set of political questions, each paired with a proper verbal response, which the populace was trained in behavioristically, via reward and punishment. Today in the US, this manifests as two opposing political programs, each consisting of a set of questions paired with their proper verbal responses, which are taught via reward and punishment.
One of these groups learned performativity from the Nazis via the feminist Judith Butler. The other had already learned it at the First Council of Nicaea in 325 AD, in which the orthodox Church declared that salvation (and not being exiled or beheaded) depended on using the word homoousios instead of homoiousios,even though no one could explain the difference between them.The purpose in all four cases was not to make an assertion which fit into a larger argument; it was to teach people to agree without thinking by punishing them if they failed to mouth logical absurdities.
So to say “We have to listen to each other’s utterances as assertions” is a very Aspie thing to say today. The things people argue about the most are not actually arguments, but are what the post-modern philosophers Derrida and Barthes called “the discourse”, and claimed was necessarily hallucinatory in exactly the same way LLMs are today (being nothing but mash-ups of earlier texts). Take a stand against hallucination as normative, but don’t point to LLMs when you do it.
For a similar reason, I have largely come around to using “AI” instead of “LLM” in most cases. What people mean by AI is the total package of a quasi-intelligent entity that you can communicate with using human natural language. The LLM is just the token predictor part, but you don’t use an LLM by itself, any more than you use a red blood cell by itself. You use the whole… artificial organism. “Inorganism”?
It seems like a goed starting point to simply start from how you would treat a human intelligence and then look for differences.
The term “LLM” remains important for the historical dividing line for when AIs took the leap into effective natural language interfaces. However, an AI is still an AI. They just got a lot better all of a sudden.
Likewise, it seems fine to consider the AI of a computer game to be a plain old AI and to not need any scare quotes. It’s acting like a intelligent being but is something that humans created; hence, an artificial intelligence.
I realize that I ignored most of the post in my comment above. I’m going to write a sloppy explanation here of why I ignored most of it, which I mean as an excuse for my omissions, rather than as a trustworthy or well-thought-out rebuttal of it.
To me, the post sounds like it was written based on reading Hubert Dreyfus’ What Computers Can’t Do, plus the continental philosophy that was based on, rather than on materialism, computationalism, and familiarity with LLMs. There are parts of it that I did not understand, which for all I know may overcome some of my objections.
I don’t buy the vitalist assertion that there aren’t live mental elements underlying the LLM text, nor the non-computationalist claim that there’s no mind that is carrying out investigations. These are metaphysical claims.
I very much don’t buy that LLM text is not influenced by local-contextual demands from “the thought” back to the more-global contexts. I would say that is precisely what deep neural networks were invented to do that 3-layer backprop networks don’t.
Just give someone the prompt? It wouldn’t work, because LLMs are non-deterministic. I might not be able to access that LLM. It might have been updated. I don’t want to take the time to do it. I just want to read the text.
“If the LLM text contains surprising stuff, and you DID thoroughly investigate for yourself, then you obviously can write something much better and more interesting.”
This is not obvious, and certainly not always efficient. Editing the LLM’s text, and saying you did so, is perfectly acceptable.
This would be plagiarism. Attribute the LLM’s ideas to the LLM. The fact that an LLM came up with a novel idea is an interesting fact.
The most-interesting thing about many LLM texts is the dialogue itself—ironically, for the same reasons Tsvi gives that it’s helpful to be able to have a dialogue with a human. I’ve read many transcripts of LLM dialogues which were so surprising and revelatory that I would not have believed them if I were just given summaries of them, or which were so complicated that I could not have understood them without the full dialogue. Also, it’s crucial to read a surprising dialogue yourself, verbatim, to get a feel for how much of the outcome was due to leading questions and obsequiousness.
But I don’t buy the argument that we shouldn’t quote LLMs because we can’t interrogate them, because
it also implies that we shouldn’t quote people or books, or anything except our own thoughts
it’s similar to the arguments Plato already made against writing, which have proved unconvincing for over 2000 years
we can interrogate LLMs, at least more-easily than we can interrogate books, famous people, or dead people
I think that referring to LLMs at all in this post is a red herring. The post should simply say, “Don’t cite dubious sources without checking them out.” The end. Doesn’t matter whether the sources are humans or LLMs. I consider most recent LLMs more-reliable than most people. Not because they’re reliable; because human reliability is a very low bar to clear.
The main point of my 1998 post “Believable Stupidity” was that the worst failure modes of AI dialogue are also failure modes of human dialogue. This is even more true today. I think humans still produce more hallucinatory dialogue than LLMs. Some I dealt with last month:
the millionaire white male Ivy-league grad who accused me of disagreeing with his revolutionary anti-capitalist politics because I’m privileged and well-off, even though he knows I’ve been unemployed for years, while he just got his third start-up funded and was about to buy a $600K house
friends claiming that protestors who, on video, attacked a man from several sides before he turned on them, did not attack him, but were minding their own business when he attacked them
my fundamentalist Christian mother, who knows I think Christianity is completely false, keeps quoting the Psalms to me, and is always surprised when I don’t call them beautiful and wise
These are the same sort of hallucinations as those produced by LLMs when some keyword or over-trained belief spawns a train of thought which goes completely off the rails of reality.
Consider the notion of “performativity”, usually attributed to the Nazi activist Heidegger. This is the idea that the purpose of much speech is not to communicate information, but to perform an action, and especially to enact an identity such as a gender role or a political affiliation.
In 1930s Germany, this manifested as a set of political questions, each paired with a proper verbal response, which the populace was trained in behavioristically, via reward and punishment. Today in the US, this manifests as two opposing political programs, each consisting of a set of questions paired with their proper verbal responses, which are taught via reward and punishment.
One of these groups learned performativity from the Nazis via the feminist Judith Butler. The other had already learned it at the First Council of Nicaea in 325 AD, in which the orthodox Church declared that salvation (and not being exiled or beheaded) depended on using the word homoousios instead of homoiousios, even though no one could explain the difference between them. The purpose in all four cases was not to make an assertion which fit into a larger argument; it was to teach people to agree without thinking by punishing them if they failed to mouth logical absurdities.
So to say “We have to listen to each other’s utterances as assertions” is a very Aspie thing to say today. The things people argue about the most are not actually arguments, but are what the post-modern philosophers Derrida and Barthes called “the discourse”, and claimed was necessarily hallucinatory in exactly the same way LLMs are today (being nothing but mash-ups of earlier texts). Take a stand against hallucination as normative, but don’t point to LLMs when you do it.
I sort of agree that LLMs are somewhat incidental to the point of the post ( https://www.lesswrong.com/posts/DDG2Tf2sqc8rTWRk3/llm-generated-text-is-not-testimony?commentId=d5gtpsRzESm4dNxBZ ). I also agree that utterances today are very often failing to be testimony in the way I discuss, and that this fact is very important. A main aim of this essay is to help us think about that phenomenon.
For a similar reason, I have largely come around to using “AI” instead of “LLM” in most cases. What people mean by AI is the total package of a quasi-intelligent entity that you can communicate with using human natural language. The LLM is just the token predictor part, but you don’t use an LLM by itself, any more than you use a red blood cell by itself. You use the whole… artificial organism. “Inorganism”?
It seems like a goed starting point to simply start from how you would treat a human intelligence and then look for differences.
The term “LLM” remains important for the historical dividing line for when AIs took the leap into effective natural language interfaces. However, an AI is still an AI. They just got a lot better all of a sudden.
Likewise, it seems fine to consider the AI of a computer game to be a plain old AI and to not need any scare quotes. It’s acting like a intelligent being but is something that humans created; hence, an artificial intelligence.
I realize that I ignored most of the post in my comment above. I’m going to write a sloppy explanation here of why I ignored most of it, which I mean as an excuse for my omissions, rather than as a trustworthy or well-thought-out rebuttal of it.
To me, the post sounds like it was written based on reading Hubert Dreyfus’ What Computers Can’t Do, plus the continental philosophy that was based on, rather than on materialism, computationalism, and familiarity with LLMs. There are parts of it that I did not understand, which for all I know may overcome some of my objections.
I don’t buy the vitalist assertion that there aren’t live mental elements underlying the LLM text, nor the non-computationalist claim that there’s no mind that is carrying out investigations. These are metaphysical claims.
I very much don’t buy that LLM text is not influenced by local-contextual demands from “the thought” back to the more-global contexts. I would say that is precisely what deep neural networks were invented to do that 3-layer backprop networks don’t.
Just give someone the prompt? It wouldn’t work, because LLMs are non-deterministic.
I might not be able to access that LLM. It might have been updated. I don’t want to take the time to do it. I just want to read the text.
“If the LLM text contains surprising stuff, and you DID thoroughly investigate for yourself, then you obviously can write something much better and more interesting.”
This is not obvious, and certainly not always efficient. Editing the LLM’s text, and saying you did so, is perfectly acceptable.
This would be plagiarism. Attribute the LLM’s ideas to the LLM. The fact that an LLM came up with a novel idea is an interesting fact.
The most-interesting thing about many LLM texts is the dialogue itself—ironically, for the same reasons Tsvi gives that it’s helpful to be able to have a dialogue with a human. I’ve read many transcripts of LLM dialogues which were so surprising and revelatory that I would not have believed them if I were just given summaries of them, or which were so complicated that I could not have understood them without the full dialogue. Also, it’s crucial to read a surprising dialogue yourself, verbatim, to get a feel for how much of the outcome was due to leading questions and obsequiousness.
But I don’t buy the argument that we shouldn’t quote LLMs because we can’t interrogate them, because
it also implies that we shouldn’t quote people or books, or anything except our own thoughts
it’s similar to the arguments Plato already made against writing, which have proved unconvincing for over 2000 years
we can interrogate LLMs, at least more-easily than we can interrogate books, famous people, or dead people