I wrote a strident version of this advice on LessWrong here in March, though it has become significantly more urgent since then as models improve (but not enough to be great substitutes for human writing), and more people notice they can use them.
JustisMills
Karma: 2,230
Your intuition is right; it doesn’t. It’s true whenever P is true/logically equivalent to just asserting P, which I think is the joke, but it’s not valid in general.
As an example “If ‘if it’s raining the streets are wet’ then it’s raining” is clearly false.
(Sorry if you are also participating in the joke in a way that is going over my head)
Yeah, it’s an interesting question how good human detection is. My guess is that people who are paying attention are getting better at sniffing out AI faster than AI is getting less distinctively scented, but “people who are paying attention” is a heck of a sleight of hand.
Overall, I suppose my main feeling is that I see AI generated stuff all the time in lots of different arenas, and I see other people judging it, and it sort of feels like an Eternal September where some people are freshly excited by some AI use case in their thinking, and don’t realize how it comes off (or do, but it hasn’t occurred to them that it comes off badly for good reasons as well as straightforward prejudice). There may also be lots of people using AI so skillfully that they don’t fall into any of these traps. It’s even possible they far outnumber the people who are (I think) bumbling. Perhaps my doubt for this last proposition is a stuck prior. But if so, it is well and truly stuck.
I infer you mean “the claim that LLM writing is a slog to get through.” Which yes, I wait for the day that most LLM writing I see is not usually a slog to get through. I hope it comes! It’s perfectly possible I see lots of LLM writing that was created by prompting wizards using state-of-the-art $200-a-month models in ways not dreamt of in my philosophy. If so, great. If you can fool me, we both win.
But I see so much LLM writing in the wild and professionally that just sucks. Whether it sucks because of some fundamental property of LLMs (doubtful), or because of path dependencies for the LLMs that are commercially available (getting warmer), or because of bog standard skill issues on the part of the relevant centaur (could be), one clear fact is that the authors don’t think there’s a problem.
In situations like this, where I see a lot of people doing something that seems like a pretty big mistake, I want to just say “don’t do that.” Because I think if I say “don’t do that, unless you’re good at it,” well, if the people making the mistake knew they weren’t good at it, they’d already not be making the mistake! I’d rather say “don’t go cave diving” and the tiny minority of expert, professional cave divers who know and relentlessly apply all the proper cave diving safety rules can smile knowingly and ignore me. Of course, the amateurs can ignore me too. But I am here advising otherwise!
I think there are two separate claims lurking here:
If you don’t have a clear idea, your expression of it will be a mess
If you do have a clear idea, your expression of it will be clear
I think the first is basically true, while the second isn’t necessarily true because writing can be hard. There are lots of ways for a clear idea to not make it into clear writing, like inferential distance as you mention.
The main thing to me is, if you start writing and it’s just not working, one hypothesis is that your thinking has still not really crystallized. And if you go straight to an LLM to “clarify this” you accidentally tend to throw out that hypothesis.
Many people around here were making pretty similar AI risk arguments 10+ years ago, before there were LLMs to point to and feel instinctively insecure about.
In general, I think different people and groups dislike AI for many different reasons, and think straightforward jealousy is currently not a very common motivation. Maybe in a few years!
I wrote a post in reply to this, which is here: https://justismills.substack.com/p/ordinary-people
Briefly, I think that my experience is dual-booting rationalist social instincts with ordinary social instincts, not in the sense of merely being able to model both but of kind of just genuinely feeling and identifying with both, and from that perspective I also feel the judgment on “the common man’s” reasoning, but feel a sort of symmetrical judgment of rationalists along the axes where the normie value system would find them absurd. To me this is pretty much a cure for the relevant misanthropy, like the normie orientation produces worse reasoning on rationalist terms but the rationalist orientation produces worse socialization on normie terms, and both feel like intrinsically lovable/sensible terms from the inside to me.
I think a better analogy is you as a local LLM having root access to the computer it’s running on. If you brick it, you’ve also broken the substrate that facilitates the process by which you’d naively unbrick—the terminal commands you’d use no longer work!
Horseshoe theory in action: the most and least woo people in the world might make a post with this title.
One thing is, babies very gradually get harder in exactly the way you describe! Like, at first by default they breastfeed, and don’t have teeth, which is at the very least highly instinctive to learn. Then they eat a tiiiny bit of solid food, like a bite or two once a day, to train you. So you have gotten way stronger at “baby eating challenges” by the time the baby can e.g. throw food. Likewise they’ll very rarely try to put stuff in their mouths early on, then really gradually more and more, so you hone that instinct too. Even diapers don’t smell bad the first couple of months! Hard to overestimate the effects of the extremely instinct compliant learning curve.
Sure, some of those. But also I just expected parenthood to change me a bunch to be better suited to it. Like, it’s a challenge such that rising to it transforms you. With babysitting you’re just skipping to a random bit pretty far into the process, not already having been transformed.
As a data point I was extremely confident I wanted kids, and didn’t especially vibe with most babies/never had changed a diaper before my kid was born, and my confident prediction, so far, was if anything an underestimate. I doubt a week of babysitting would have changed my intent whatsoever, but it would probably have been stressful and not that fun.
As an audience member, I often passively judge people for responding to criticism intensely or angrily, or judge both parties in a long and bitter back and forth, and basically never judge anyone for not responding.
When I’ve responded to criticism with “oh, thanks, hadn’t thought of that”, I haven’t really felt disapproved of or diminished. Sometimes the critic is just right, and sometimes they just are looking at the topic from another angle and it’s fine for readers to decide whether they like it better than mine. No big deal. I don’t really see evidence that anyone’s tracking my status that hard. I’d rather make sure nobody’s tracking me being unkind though, including myself.
(This comment is offered up as a data point from the peanut gallery. I have no idea if it’s representative! If you reply, it may make me happy, but if not I won’t mind.)
It’s grown on me as I’ve edited more technical stuff; growing up in the US literary tradition it always seemed elegant to put punctuation inside, though now that I read more stuff like LessWrong it does feel kinda imprecise/hand-wavey somehow. Like, wait a minute, that guy you’re quoting didn’t put a comma in there!
Yeah, I messed around with Typst for the first time recently. There’s a whole dang world out there!
Yeah, for me it solves itself instantly once I actually notice it in any given case. I mostly try not to think of myself as an expert at stuff (luckily for that, I rarely am!), but there are weird psychological incentives with being a professional at stuff, I think.
Having kids is a huge part of it for me. Came by it but absorbing a philosophy where it mattered a lot from my parents, which I liked and accepted; I’ve always aspired to be pretty similar to them. Now I’ve got a baby and yep, as I hoped or better.
I value many other things too, which have been pretty stable since high school. Helping the global poor somewhat, creating art, having friends, eventually getting married and being a good spouse, and some idiosyncratic imagination stuff. These things have all seemed obviously good to me for so long I don’t really know where they came from. Just following gradients as a kid and imprinting on (I think good) role models, maybe. I think I endorse it all, though.
I doubt this helps your particular quest, but that’s my answer, at least right now!
I haven’t noticed anyone else come out and say it here, and I may express this more rigorously later, but, like, GPT-5 is a pretty big bear signal, right? Not just in terms of benchmarks suggesting a non-superexponential trend but also, to name a few angles/anecdata:
It did slightly worse than o3 at the first thing I tried it on (with thinking mode on)
It failed to one-shot a practical coding problem that was entirely in javascript, which is generally among its strengths (and the problem was certainly in principle solvable)
It’s hallucinating pretty obviously when I give it a 100 page or so document to read and comment on (it references lots from the document, but gets many details wrong and overfixates on others)
It’s described as an automatic family of models that naturally picks the best for the situation, which seems like exactly what you’d do if nothing else was really giving you the sauce
The main reddit reaction seems to be that the demo graphs were atrocious, which is not exactly glowing praise
All the above paired with the fact that this is what they chose to call GPT-5, and with the fact that Claude’s latest release was a well-named and justified 0.1 increment
I’m largely with Zvi that even integrating this stuff as it already exists into the economy does some interesting stuff, and that we live in interesting times already. But other than what’s already priced in by integrations and efficiency optimizations, progress feels s-curvier to me today than it did a week ago.
30 people is the aim!
I think the main disanalogy with startup incubators is that we’re not necessarily betting for successful bloggers to increase their earning power; probably the modal respondent (given the community skews tech) can currently make as much money as a top 1% blogger already (though to be clear not necessarily a top .1% blogger).
I think the analogy that’s better is to a meditation retreat or athletic conditioning program, or something, which afaict do pretty often charge (or at least have a strongly suggested donation in the meditation case); part of the point of charging (other than the obvious reason of just paying for stuff) is to make sure people are really bought-in/personally motivated to get the most out of it possible.
(I am helping with this event, but I don’t speak for Ben here)
A lot of my wordsmithing advantage has eroded, and I willingly admit this, but am pretty sure having AI write for you remains a bad idea.
(Asserting without proof as I gotta go to brunch, and plus I am not sure I would be able to prove or even argue for it with a full belly, but seems worth flagging from a guy paid to edit stuff becoming noticeably more obsolete in that area year over year, and noticing that. I do consciously admit it, but still think AI writing is a bad substitute for human writing.)