I realize that I ignored most of the post in my comment above. I’m going to write a sloppy explanation here of why I ignored most of it, which I mean as an excuse for my omissions, rather than as a trustworthy or well-thought-out rebuttal of it.
To me, the post sounds like it was written based on reading Hubert Dreyfus’ What Computers Can’t Do, plus the continental philosophy that was based on, rather than on materialism, computationalism, and familiarity with LLMs. There are parts of it that I did not understand, which for all I know may overcome some of my objections.
I don’t buy the vitalist assertion that there aren’t live mental elements underlying the LLM text, nor the non-computationalist claim that there’s no mind that is carrying out investigations. These are metaphysical claims.
I very much don’t buy that LLM text is not influenced by local-contextual demands from “the thought” back to the more-global contexts. I would say that is precisely what deep neural networks were invented to do that 3-layer backprop networks don’t.
Just give someone the prompt? It wouldn’t work, because LLMs are non-deterministic. I might not be able to access that LLM. It might have been updated. I don’t want to take the time to do it. I just want to read the text.
“If the LLM text contains surprising stuff, and you DID thoroughly investigate for yourself, then you obviously can write something much better and more interesting.”
This is not obvious, and certainly not always efficient. Editing the LLM’s text, and saying you did so, is perfectly acceptable.
This would be plagiarism. Attribute the LLM’s ideas to the LLM. The fact that an LLM came up with a novel idea is an interesting fact.
The most-interesting thing about many LLM texts is the dialogue itself—ironically, for the same reasons Tsvi gives that it’s helpful to be able to have a dialogue with a human. I’ve read many transcripts of LLM dialogues which were so surprising and revelatory that I would not have believed them if I were just given summaries of them, or which were so complicated that I could not have understood them without the full dialogue. Also, it’s crucial to read a surprising dialogue yourself, verbatim, to get a feel for how much of the outcome was due to leading questions and obsequiousness.
But I don’t buy the argument that we shouldn’t quote LLMs because we can’t interrogate them, because
it also implies that we shouldn’t quote people or books, or anything except our own thoughts
it’s similar to the arguments Plato already made against writing, which have proved unconvincing for over 2000 years
we can interrogate LLMs, at least more-easily than we can interrogate books, famous people, or dead people
I realize that I ignored most of the post in my comment above. I’m going to write a sloppy explanation here of why I ignored most of it, which I mean as an excuse for my omissions, rather than as a trustworthy or well-thought-out rebuttal of it.
To me, the post sounds like it was written based on reading Hubert Dreyfus’ What Computers Can’t Do, plus the continental philosophy that was based on, rather than on materialism, computationalism, and familiarity with LLMs. There are parts of it that I did not understand, which for all I know may overcome some of my objections.
I don’t buy the vitalist assertion that there aren’t live mental elements underlying the LLM text, nor the non-computationalist claim that there’s no mind that is carrying out investigations. These are metaphysical claims.
I very much don’t buy that LLM text is not influenced by local-contextual demands from “the thought” back to the more-global contexts. I would say that is precisely what deep neural networks were invented to do that 3-layer backprop networks don’t.
Just give someone the prompt? It wouldn’t work, because LLMs are non-deterministic.
I might not be able to access that LLM. It might have been updated. I don’t want to take the time to do it. I just want to read the text.
“If the LLM text contains surprising stuff, and you DID thoroughly investigate for yourself, then you obviously can write something much better and more interesting.”
This is not obvious, and certainly not always efficient. Editing the LLM’s text, and saying you did so, is perfectly acceptable.
This would be plagiarism. Attribute the LLM’s ideas to the LLM. The fact that an LLM came up with a novel idea is an interesting fact.
The most-interesting thing about many LLM texts is the dialogue itself—ironically, for the same reasons Tsvi gives that it’s helpful to be able to have a dialogue with a human. I’ve read many transcripts of LLM dialogues which were so surprising and revelatory that I would not have believed them if I were just given summaries of them, or which were so complicated that I could not have understood them without the full dialogue. Also, it’s crucial to read a surprising dialogue yourself, verbatim, to get a feel for how much of the outcome was due to leading questions and obsequiousness.
But I don’t buy the argument that we shouldn’t quote LLMs because we can’t interrogate them, because
it also implies that we shouldn’t quote people or books, or anything except our own thoughts
it’s similar to the arguments Plato already made against writing, which have proved unconvincing for over 2000 years
we can interrogate LLMs, at least more-easily than we can interrogate books, famous people, or dead people