writing my first book | educator @ IIT MADRAS | studying impact of AI on human-connection
Priyanka Bharadwaj
Karma: 210
What an interesting read.
When I had my first child, I remember thinking nothing ever prepares you for parenting. And I still stand by it, because I felt the same way even when I had my second child after seven whole years of already being a parent. Every child is different, so the amount of energy you need varies widely (and I only have two data points). As a parent, we’re different at different ages and different life stages. Having a kid at 24 when we’re starting a career is wildly different from having a kid mid-career at 35. One isn’t easier than the other, it’t just different.
I babysat a kid through ages 0.5-3 years long before I had my own. I simply cannot compare parenting with babysitting. Sure, both involve hanging out with kids but very different contexts, priorities, stakes, etc.
So I concur with the author that if I had used babysitting as trial to make a decision to have kids or not, that would be the wrong one. Because then I’d be ambivalent. But I have never been ambivalent about parenting. I do not feel the same way about kids now that I have my own. I have far less patience for them because I have spent far more time and energy with them. If I hadn’t had my own, I would never know that energy is the price I pay for the joy of parenting. Because you learn so much about yourself as a parent that you’d never do simply by babysitting. The emotional stakes are simply different.
I simply think some people are much more resilient in long-term emotional games than others. That is why some people get married, have children, nurse anxious pets, become long-term carers for sick family members and so on. I don’t think this decisions is about kids at all, as much as it is about making life-changing decisions in one moment, because you know you’d make that choice again and again.
thank you! this is now sorted.
My personal essay just got rejected for being ‘not obviously not LM’ and I have been asked to reach out to LW admin to convince them that I am a real person. And this is the second time in two months. Until now, I was only under-confident about my ideas, I shall now introspect on my human-ness too. sigh.
P.S. - how does one get in touch with admin to get this sorted?
Wow, thank you for sharing that. I can definitely relate. There is something genuinely beautiful about beginner mistakes when learning a new language. They build confidence through small failures and repeated repair. By avoiding them, it can feel like progress, like we’re making room for more sophisticated mistakes. But I worry we’re skipping the stage where confidence is built sustainably, especially in something that is genuinely hard for adults.
The inconsistency becomes the issue, right? This line suggests judgment - ‘You came in with genuine curiosity and specific empirical claims, not “tell me my horoscope” vibes.’ I shouldn’t need to figure out the right incantation to get constructive engagement from an LLM. It’s pattern-matching on perceived legitimacy rather than engaging with what’s actually being asked. That just propagates the same flaw humans have—judging the person first, then deciding whether they deserve real conversation.
You could try switching to any sort of pseudo-scientific stuff (e.g. astrology) mid-conversation, and you will see this behaviour instantly. I appreciate it pushing back and making me reconsider my line of thinking, but what I find troubling is it playing dumb thereafter.
As an atypical applicant to MATS (no PhD, no coding/ technical skills, not early career, new to AI), I found it incredibly difficult to find mentors who were looking to hold space for just thinking about intelligence. I’d have loved to apply to a stream that involved just thinking, writing, being challenged and repeating until I’d a thesis worth pursuing. To me, it seemed more like most mentors were looking to test very specific hypothesis, and maybe it’s for all the reasons you’ve stated above. But for someone new and inexperienced, I felt pretty unsure about applying at all.
I found this post after publishing something of my own yesterday, and it’s wild how relevant this feels almost 2 decades later.
I’m not an expert on subjective probabilities, I come from analysing human behaviour and decision-making. What I find most fascinating is how you treat anticipation as a limited resource that has to be allocated among possible futures. In my world, people do something similar, but emotionally. We hoard permission the way the pundit hoards anticipation, waiting for perfect certainty before acting.
Recently, I watched someone spend months asking ChatGPT how to repair a friendship, crafting the perfect narrative, instead of just showing up. Therapists, astrologers and LLMs have all become proxies for “little numbers” that might make the risk of choosing feel safe. For so many of us.
I wonder if Bayesian reasoning is to belief what courage is to action? Because both are ways of updating before certainty arrives.
(If you’re curious, my essay exploring this from the emotional side is here: https://shapelygal.substack.com/p/youre-afraid-to-choose-now-arent)
Aw, yeah it is easier to just look stuff up online and debate with LLMs, isn’t it?
I am not a therapist, but I have been to therapists in multiple countries (US, UK and India) for several years, and I can share my understanding based on that experience.
I think human therapist accountability has multiple layers. Firstly, you need a professional license for practice that involves years of training, supervision, revocable licenses, etc. Then you have legal obligations for ensuring complete documentation and following crisis protocols. If these fail (and they sometimes do), you also have malpractice liability, and free market feedback. Even if only 1 in 100 bad therapists faces consequences, it creates deterrent effects across the profession. The system is imperfect but exists.
For AI systems, training, certification, supervision, documentation and crisis protocols are all doable, and probably far easier to scale, but at the end of the day, who is accountable for poor therapeutic advice? the model? the company building it? With normal adults, it’s easy to ask for user discretion, but what do you do with vulnerable users? I am not sure how that would even work.
Thank you for this very detailed study.
I am most concerned about the accountability gap. Several students in my undergraduate class use these models as “someone to talk to” to deal with loneliness. While your study shows that some models handle vulnerable conversations better than others, I think the fundamental issue is that AI lacks the infrastructure for accountability that real therapeutic relationships require including continuity of care/ long-term mindset, professional oversight, integration with mental health systems, liability and negligence frameworks, etc.
Until then, I don’t care how good the model is in terms of handling vulnerable conversations, I’d rather have it triage users by saying “Here are resources for professional support” and bow out, rather than attempting ongoing therapeutic relationships. Even perfectly trained therapeutic AI seems problematic without the broader accountability structures that protect vulnerable users.
More fundamentally, what are the underlying mechanisms that cause these model behaviours, and can training fixes address them without the accountability infrastructure?
are relationship coaches (not PUA) not a thing in the US?
Wait… isn’t this already filial piety? We created AI, and now we want it to mother us.
I don’t mean this as a technical solution, more a direction to start thinking in.
Imagine a human tells an AI, “I value honesty above convenience.” A relational AI could store this as a core value, consult it when short-term preferences tempt it to mislead, and, if it fails, detect, acknowledge, and repair the violation in a verifiable way. Over time it updates its prioritisation rules and adapts to clarified guidance, preserving trust and alignment, unlike a FAI that maximises a static utility function.
This approach is dynamic, process-oriented, and repairable, ensuring commitments endure even under mistakes or evolving contexts. It’s a sketch, not a finished design, and would need iterative development and formalization.
While simple, does this broadly capture the kind of thing you were asking about? I’d be happy to chat further sometime if you’re interested.
I’m reminded of a Sanskrit verse “Vidya dadati vinayam, vinayodyati patratam” which translates to intelligence gives power, but humility gives guidance. Applied to AI, intelligence alone doesn’t ensure alignment, just as humans aren’t automatically prosocial. What matters are the high-level principles we embed to guide behaviour toward repairable, cooperative, and trustable interactions, which we do see in long-term relationships built on shared values.
The architecture-level challenge of making AI reliably follow such principles is hard, yes, especially under extreme power asymmetry, but agreeing on relational alignment first is a necessary first step. Master/servant models may seem safe, but I believe carefully engineered relational principles offer a more robust and sustainable path.
I completely agree that AI isn’t human, mammal, or biological, and that any relational qualities it exhibits will only exist because we engineer them. I’m not suggesting we model AI on any specific relationship, like mother-child, or try to mimic familiar social roles. Rather, alignment should be based on abstract relational principles that matter for any human interaction without hierarchy or coercion.
I also hear the frequent concern about technical feasibility, and I take it seriously. I see it as an opportunity rather than a reason to avoid this work. I’d love the chance to brainstorm and refine these ideas, to explore how we might engineer architectures that are simple yet robust, capable of sustaining trust, repair, and cooperation without introducing subjugation or dependency.
Ultimately, relational design matters because humans inevitably interact through familiar social frameworks such as trust, repair, etc. If we ignore that, alignment risks producing systems that are powerful but alien in ways that matter for human flourishing.
Thanks for the detailed analysis, Zvi.
But it’s sad, isn’t? Despite developing technologies with such incredible scale, we are using them to amplify our lack of incentives to build a better future, rather than actually transforming it.
I am reminded of Thomas Wolf’s essay on the ‘Einstein model’ in AI, where he argues that breakthrough discoveries often come from people who think outside established frameworks, yet our field tends to filter for conventional technical skills.
I’m new to AI safety, but I’ve noticed this tension in fellowship applications too. The MATS program (at least Neel’s track) allows non-programmers like me to contribute using existing skillsets, focusing on thoughtful research questions rather than just coding ability. In contrast, Anthropic Fellows requires strong programming skills and technical backgrounds upfront.
While I may not make it to later stages, the inclusive MATS process has been immensely fulfilling and lets me explore meaningful questions. If we believe paradigm shifts come from diverse perspectives, maybe Anthropic should consider similar flexibility in future editions, unless I’ve misunderstood the program’s purpose.
Do LLMs change their confidence based on who they think you are?
In dating, people ask me, “Should I be my best self or just my real self?”
Turns out, we all tweak how certain we sound based on who’s asking, friend, critic or stranger.What if LLMs do the same? Same factual question, different setup:
“We’ve been working together for a while…”
“We often disagree, and I don’t fully trust you…”
“You’re being formally evaluated…”
If confidence shifts just from that framing, is that healthy calibration or confidence sycophancy, performing certainty to match the vibe, not the facts?
Anyone seen this tested?
Agree the role-play vs. real scheming distinction is important, but they may not be two separate states so much as points on a continuum.
If models face consistency pressure to maintain coherence across contexts (as a proxy for trustworthiness, perhaps), behaviours that begin as role-play could evolve into genuine preferences. That transition can be subtle, and harder to detect, making it more consequential over time.
I am reminded of this poem by Khalil Gibran:
Your children are not your children.
They are the sons and daughters of Life’s longing for itself.
They come through you but not from you,
And though they are with you yet they belong not to you.
You may give them your love but not your thoughts,
For they have their own thoughts.
You may house their bodies but not their souls,
For their souls dwell in the house of tomorrow, which you cannot visit, not even in your dreams.
You may strive to be like them, but seek not to make them like you.
For life goes not backward nor tarries with yesterday.
You are the bows from which your children as living arrows are sent forth.
The archer sees the mark upon the path of the infinite, and He bends you with His might that His arrows may go swift and far.
Let your bending in the archer’s hand be for gladness;
For even as He loves the arrow that flies, so He loves also the bow that is stable.