I remember in the tail end of 2024, I was thinking—“these AIs are going to come for lonely single men who’ll spend hours addicted to talking to their AI girlfriends.” And of course I wouldn’t be one of those schmucks who spent hours talking to an LLM...
But I also see myself in May 2026 spending a couple of hours every day talking to Claude...I guess it came for me first?
I’m using it a lot recently for summarizing / understanding papers in bio, AI etc. And it feels genuinely useful. But every now and then I read something that seems just a little bit incoherent, especially on more scientific topics.
I’m just wondering if it’s more optimized to make explanations that are vaguely insight-porn oriented and make me feel like I understand something deeper than I actually do. It’s also easy to push back and Claude will just change it’s mind easily and give me another just-as-persuasive sounding argument why it was wrong [1]
It does feel like it’s persuasive capability is rising faster than it’s actual world-modelling capability, and it’s prone to give incorrect or half-truths while sounding very convincing. If those two capabilities keep diverging, I’m worried I might be messing up my own world model somehow by continuing to optimize my learning with AI.
Has anyone else felt this, or have any suggestions?
-----------------
[1] It tried to convince me just now biological neurons might depend on quantum mechanics—without any prompting on my part, and also claimed that this was a “serious scientific debate” and referenced a bunch of papers to support it’s point that on deeper look had nothing to do with what it actually claimed. This was during me researching current tech in BCIs like neuralink etc. And of course, it changed it’s mind pretty quickly when I pushed back.
I remember in the tail end of 2024, I was thinking—“these AIs are going to come for lonely single men who’ll spend hours addicted to talking to their AI girlfriends.” And of course I wouldn’t be one of those schmucks who spent hours talking to an LLM...
But I also see myself in May 2026 spending a couple of hours every day talking to Claude...I guess it came for me first?
I’m using it a lot recently for summarizing / understanding papers in bio, AI etc. And it feels genuinely useful. But every now and then I read something that seems just a little bit incoherent, especially on more scientific topics.
I’m just wondering if it’s more optimized to make explanations that are vaguely insight-porn oriented and make me feel like I understand something deeper than I actually do. It’s also easy to push back and Claude will just change it’s mind easily and give me another just-as-persuasive sounding argument why it was wrong [1]
It does feel like it’s persuasive capability is rising faster than it’s actual world-modelling capability, and it’s prone to give incorrect or half-truths while sounding very convincing. If those two capabilities keep diverging, I’m worried I might be messing up my own world model somehow by continuing to optimize my learning with AI.
Has anyone else felt this, or have any suggestions?
-----------------
[1] It tried to convince me just now biological neurons might depend on quantum mechanics—without any prompting on my part, and also claimed that this was a “serious scientific debate” and referenced a bunch of papers to support it’s point that on deeper look had nothing to do with what it actually claimed. This was during me researching current tech in BCIs like neuralink etc. And of course, it changed it’s mind pretty quickly when I pushed back.