Very cool!
I will say, it does seem like LessWrong is one of the worst places to unleash this. In part for reasons already mentioned by previous commenters, and in part because LessWrong is one of the places where people are actually doing cutting-edge stuff on a regular basis. Evaluating whether something is plausible when it is legitimately on the cutting edge of its field is one of the things I think most big llms are super bad at. It’s particularly frustrating when you’re trying to innovate, and people are always like “ChatGPT says that’s impossible!” lol. For example, I have a ChatGPT conversation from from last year where he says a surgery I had already succeeded in figuring out and getting was “probably not” possible. He then proceeded to say a bunch of mealy-mouthed, borderline incorrect stuff about its risks, which did not reflect a deep understanding of the tech involved at all. I think this is pretty par for the course when you’re doing unusual/innovative stuff—Claude and ChatGPT and their buddies are conservative, and biased against stuff that sounds weird.
With that said, the tool itself is very cool. I think I’m just hoping that people use it wisely and in a way that supports a culture of discovery on LessWrong, not in a pedantic way that just causes a lot more unnecessary annoying labor for people trying to do cutting edge stuff.
I’m working on an article for a magazine touching on pain management during post-surgical extubation, and once again I find myself freshly disturbed by how difficult it is to get good data about PAIN during extubation (and many other procedures) because the dominant culture in “pain management” cares so little about pain itself. They often care exclusively about preventing memories of pain, rather than preventing pain itself.
A paper I read today says: “Many of the patients exhibited severe bucking or similar bodily movements, raising concerns that they might retain memories of discomfort on extubation. However, given their low frequency, such memories were considered to be of negligible importance.”
This paper collects numbers on how many of the patients can recall their “discomfort” later. But they do not bother to collect numbers on how many patients exhibited signs of that “discomfort” in the first place. It would have been so easy to just give me a number on how many patients were “bucking”, but no. They only say “many.”
They treat the natural denominator—how many patients were, in fact, exhibiting signs of pain—as irrelevant. To them all that matters is whether you remember it.
The authors say, “Immediately before extubation, patients often exhibit signs that suggest that they are awake and experiencing discomfort. There is concern that patients may retain such memories of the extubation process.”
There is concern patients may retain such memories? How about some concern that the patients are in pain in the first place? Can we get a little concern for that?
In theory, of course, it is legitimate for an individual paper to focus on studying memory rather than studying pain.
But in fact, the concern demonstrated in this paper is the dominant one in the culture. In the anesthesia and pain management worlds, “I’m concerned they’ll remember this” is treated as the appropriate professional substitute for “I’m concerned they’re in pain” SO OFTEN. If you spend time with the anesthesia literature—it doesn’t even have to be a lot of time—you will start noticing this pattern everywhere. You’ll notice memory of pain being implicitly treated as synonymous with pain itself, pain itself ignored. You’ll notice that even in texts purporting to study/discuss pain itself, the tone and even sometimes structure of the research often betray an implicit philosophical assumption that pain, even extreme pain, is morally irrelevant so long as it is kept properly non-disruptive and liability-free and sequestered from patient satisfaction surveys. There is a refrain one often hears from critical care staff and anesthesiologists, “Pain is not an emergency,” and you will start to notice how the entire field, including the research side, is marinated in this philosophical assumption. And the fact that this philosophical assumption is so pervasive and relatively unquestioned makes it difficult to even document the existence of the problems it creates, since it prevents the development of would-be-relevant research.
Their conclusion begins: “Recall of discomfort during extubation appears to be rare, and the great majority of patients may not retain any memory of the extubation process. This information may be used to reassure patients”
You can make of that what you will. Me? I’m not reassured.
Here’s the paper in question: https://pubmed.ncbi.nlm.nih.gov/37638085/
Again to be clear, I’m not necessarily saying that THIS PAPER on its own is like, bad or evil, but rather pointing out that the field has been dragged to a very dark philosophical place, and this paper is but one tiny data point that I’m using merely as a representative example of where the field’s focus lies. Obv it is a bit difficult to prove “a vague but very bad cultural tendency to focus on the wrong thing is taking place”, so you’ll just have to wait until the bigger article comes out to see me take my best stab at actually proving that, heh.