People trying to predict the effects of automation/AI capabilities should consider that employees often perform valuable services which aren’t easily captured in evals, such as “beside manner”
Ah yes, bedside manner, that magical trait which only humans can ever possess. As if interacting with an overworked, time-pressured doctor who’s seen forty patients today and just wants to get through the queue is the pinnacle of experience and connection. The warmth and presence people romanticize about is an ideal, not a reality.
Meanwhile, setting aside capabilities, I would take interacting with claude over interacting with any of the doctors I’ve encountered in my entire life. And I’ve generally had good experiences with the medical system! Trans medicine is infamously terrible but the guy who prescribes my HRT is great, he’s nice and knowledgeable and compliant. I would say he’s probably top 10% as far as trans doctors go, based on what I’ve heard from my friends.
But no human can compete with a mind that’s almost infinitely kind and patient. No human can compete with a mind without time pressure, without ego, without bad days, and no instinct to play social games. Even if the human doctor knew everything that the AI does (a questionable assumption even today), the process of explaining and teaching a complete novice is taxing for humans in ways that don’t apply to AI.
An AI can sit with you for four hours while you meander around being afraid of injecting yourself with semaglutide. It can talk you through other administration routes, explain why injections are ideal, ask you enough questions to figure out that you think injection === IM, explain that subq injections (which are far less painful/dangerous and use much smaller needles) are a thing, and reassure you that it’s going to be okay until you actually believe. And afterwards, it can share in your delight and joy and adrenaline as you’re jittering from the stress of having done the injection and realized that yeah, it was no big deal.
An AI can spend half a day with you going through the random symptoms you’re having in one of your eyes that you’re terrified might mean you’re going blind. It can research and lay out what the actual probabilities of various outcomes, while managing your emotions so you step back from “panicking” to a more calibrated “I should probably get this scanned by a machine”. It can find you an in-person doctor, walk you through the process of scheduling an appointment, manage whatever random anxieties and fears crop up, talk to you and keep you calm on the bus ride over, and then go over the scan results with you when you decide that maybe the human doctor missed something.
These are real examples from my life and my girlfriend’s. And this isn’t even getting into the enormous mountain of emotional labor, vaguely therapist-y conversations we’ve both had about non-medical things with Claude. All for the price of $100 dollars a month; less than a single doctor’s visit.
AI is absolutely capable of outperforming humans on “bedside manner”.
This is a reasonable position, but, speaking as a programmer (bear with me, there’s a useful metaphor), I have much more confidence in my ability to assess code that a human—even a very novice one—has written than in my ability to do the same for an LLM’s code. A human programmer who makes an error generally leaves hints about their misunderstanding in the design of the surrounding code, whereas LLMs’ mistakes look completely different—almost uncanny—in nature.
I don’t think I’m alone in being wary of the same issue in medicine. Human doctors are interpretable—I can ask for the justification behind a procedure, and, even if his reasoning is wrong, it comes from a human thought process that I can naturally “debug”, by virtue of evolution training humans to be good at identifying other humans’ mistakes. If I ask an LLM for its line of reasoning behind a decision, I’m not just getting an alien chain of reasoning, I’m getting an inaccurate one, because the LLM does not have a persistent internal state that it can look up in order to tell me why it said X instead of Y.
Psychologically, an assessment I can be confident in is one that won’t bring me additional stress worrying about its accuracy. This is more subjective, and you’re more favorable towards extended interactions with LLMs than most people I’ve spoken to. While I (and, to my understanding, a majority of consumers) would much rather get an interpretable 80% likelihood diagnosis of, say, a painful but definitely not dangerous rash, than a non-interpretable 85% likelihood diagnosis, that’s entirely dependent on individual value tradeoffs.
Ah yes, bedside manner, that magical trait which only humans can ever possess. As if interacting with an overworked, time-pressured doctor who’s seen forty patients today and just wants to get through the queue is the pinnacle of experience and connection. The warmth and presence people romanticize about is an ideal, not a reality.
Meanwhile, setting aside capabilities, I would take interacting with claude over interacting with any of the doctors I’ve encountered in my entire life. And I’ve generally had good experiences with the medical system! Trans medicine is infamously terrible but the guy who prescribes my HRT is great, he’s nice and knowledgeable and compliant. I would say he’s probably top 10% as far as trans doctors go, based on what I’ve heard from my friends.
But no human can compete with a mind that’s almost infinitely kind and patient. No human can compete with a mind without time pressure, without ego, without bad days, and no instinct to play social games. Even if the human doctor knew everything that the AI does (a questionable assumption even today), the process of explaining and teaching a complete novice is taxing for humans in ways that don’t apply to AI.
An AI can sit with you for four hours while you meander around being afraid of injecting yourself with semaglutide. It can talk you through other administration routes, explain why injections are ideal, ask you enough questions to figure out that you think injection === IM, explain that subq injections (which are far less painful/dangerous and use much smaller needles) are a thing, and reassure you that it’s going to be okay until you actually believe. And afterwards, it can share in your delight and joy and adrenaline as you’re jittering from the stress of having done the injection and realized that yeah, it was no big deal.
An AI can spend half a day with you going through the random symptoms you’re having in one of your eyes that you’re terrified might mean you’re going blind. It can research and lay out what the actual probabilities of various outcomes, while managing your emotions so you step back from “panicking” to a more calibrated “I should probably get this scanned by a machine”. It can find you an in-person doctor, walk you through the process of scheduling an appointment, manage whatever random anxieties and fears crop up, talk to you and keep you calm on the bus ride over, and then go over the scan results with you when you decide that maybe the human doctor missed something.
These are real examples from my life and my girlfriend’s. And this isn’t even getting into the enormous mountain of emotional labor, vaguely therapist-y conversations we’ve both had about non-medical things with Claude. All for the price of $100 dollars a month; less than a single doctor’s visit.
AI is absolutely capable of outperforming humans on “bedside manner”.
Yes, the data backs you up. In 2022 studies were showing that people had limited trust in AI, and even that varied by field. In 2023 the study came out showing that in blind trials patients overwhelmingly preferred AI chat it’s over human doctors. (https://bytefeed.ai/ai-chatbots-bedside-manner-preferred-over-conventional-doctors-by-shocking-margin-according-to-blind-study/) In 2024 & 2025 we got the studies showing AI outperforms human doctors but patients still didn’t trust it so long as they knew it was AI. Again, in blind studies patients prefer AI. (https://www.kcl.ac.uk/news/doctors-stay-ai-assists-new-study-examines-public-perceptions-of-ai-in-healthcare). Doctors don’t like AI and don’t like doctors who use AI (https://carey.jhu.edu/articles/doctors-who-use-ai-are-viewed-negatively-their-peers-new-study-shows).
This is a reasonable position, but, speaking as a programmer (bear with me, there’s a useful metaphor), I have much more confidence in my ability to assess code that a human—even a very novice one—has written than in my ability to do the same for an LLM’s code. A human programmer who makes an error generally leaves hints about their misunderstanding in the design of the surrounding code, whereas LLMs’ mistakes look completely different—almost uncanny—in nature.
I don’t think I’m alone in being wary of the same issue in medicine. Human doctors are interpretable—I can ask for the justification behind a procedure, and, even if his reasoning is wrong, it comes from a human thought process that I can naturally “debug”, by virtue of evolution training humans to be good at identifying other humans’ mistakes. If I ask an LLM for its line of reasoning behind a decision, I’m not just getting an alien chain of reasoning, I’m getting an inaccurate one, because the LLM does not have a persistent internal state that it can look up in order to tell me why it said X instead of Y.
Psychologically, an assessment I can be confident in is one that won’t bring me additional stress worrying about its accuracy. This is more subjective, and you’re more favorable towards extended interactions with LLMs than most people I’ve spoken to. While I (and, to my understanding, a majority of consumers) would much rather get an interpretable 80% likelihood diagnosis of, say, a painful but definitely not dangerous rash, than a non-interpretable 85% likelihood diagnosis, that’s entirely dependent on individual value tradeoffs.