I’m bumping into walls but hey now I know what the maze looks like.
Neil
Karma: 1,056
My own (tentative, rushed, improvised) system prompt is this one (long enough I put it in a Google doc; also allows for easy commenting if you have anything to say):
It’s the longest one I’ve seen but works pretty well! It’s been helpful for a few friends.
You can already do this! Just switch this from default to hidden according to the whim of the moment.
See also:
https://borretti.me/article/how-i-use-claude
I was planning on putting a whole list here but alas I am drawing a blank.
There’s a lot of dispersed wisdom in @Zvi’s stack too but I can’t remember any sufficiently discriminatory key words to find them.
Can confirm. Half the LessWrong posts I’ve read in my life were read in the shower.
no one’s getting a million dollars and an invitation the the beisutsukai
honored.
I like object-level posts that also aren’t about AI. They’re a minority on LW now, so they feel like high signals in a sea of noise. (That doesn’t mean they’re necessarily more signally, just that the rarity makes it seem that way to me.)
It felt odd to read that and think “this isn’t directed toward me, I could skip if I wanted to”. Like I don’t know how to articulate the feeling, but it’s an odd “woah text-not-for-humans is going to become more common isn’t it”. Just feels strange to be left behind.
Thank you for this. I feel like a general policy of “please at least disclose” would make me feel significantly less insane when reading certain posts.
Have you tried iterating on this? Like, the “I don’t care about the word prodrome’” sounds like the kind of thing you could include in your prompt and reiterate until everything you don’t like about the LLM’s responses is solved or you run out of ideas.
Also fyi ChatGPT Deep Research uses the “o3″ model, not 4o, even if it says 4o at the top left (you can try running Deep Research with any of the models selected in the top left and it will output the same kind of thing).
o3 was RLed (!) into being particularly good at web search (and tangential skills like avoiding suspicious links), and isn’t released in a way that lets you just chat with it. The output isn’t even raw o3, it’s the o3-mini model summarizing o3′s chain of thought (where o3 will think things, send a dozen tentacles out into the web, then continue thinking).
I learned this when I asked Deep Research to reverse engineer itself, and it linked the model card which in retrospect I should have done first and was foolish not to.
Anyway I mention this because afaik all the other deep research frameworks are a lot less specialized than OpenAI’s, and more like “we took an LLM and gave it access to the internet and let it think and search for a really long time”. I expect OpenAI to continue being SOTA here for a while.
Though I do enjoy using Grok’s “DeepSearch” and “DeeperSearch” function sometimes; it’s free and fun to watch (but terrible at understanding user intent, which I attribute to how little-flexible it is. It won’t listen to suggestions on where to look first or how to structure its research, relying on whatever system prompt it was given instead), you might want to check it out and update this post.
My highlight link didn’t work but in the second example, this is the particular passage that drove me crazy:
The punchline works precisely because we recognize that slightly sheepish feeling of being reflexively nice to inanimate objects. It transforms our “irrational” politeness into accidental foresight.
The joke hints at an important truth, even if it gets the mechanism wrong: our conversations with current artificial intelligences may not be as consequence-free as they seem.
That’s fair, I think I was being overconfident and frustrated, such that these don’t express my real preferences.
But I did make it clear these were preferences unrelated to my call, which was “you should warn people” not “you should avoid direct LLM output entirely”. I wouldn’t want such a policy, and wouldn’t know how to enforce it anyway.
I think I’m allowed to have an unreasonable opinion like “I will read no LLM output I don’t prompt myself, please stop shoving it into my face” and not get called on epistemic grounds except in the context of “wait this is self-destructive, you should stop for that reason”. (And not in the context of e.g. “you’re hurting the epistemic commons”.)
You can also ask Raemon or habykra why they, too, seem to systematically downvote content they believe to be LLM-generated. I don’t think they’re being too unreasonable either.That said, I agree with you there’s a strong selection effect with what writers choose to keep from the LLM, and that there’s also the danger of people writing exactly like LLMs and me calling them out on it unfairly. I tried hedging against this the first time, though maybe that was in a too-inflammatory manner. The second time, I decided to write this OP instead of addressing the local issue directly, because I don’t want to be writing something new each time and would rather not make “I hate LLM output on LW” become part of my identity, so I’ll keep it to a minimum after this.
Both these posts I found to have some value, though in the same sense my own LLM outputs have value, where I’ll usually quickly scan what’s said instead of reading thoroughly. LessWrong has always seemed to me to be among the most information-dense places out there, and I hate to see some users go this direction instead. If we can’t keep low density writing out of LessWrong, I don’t know where to go after that. (And I am talking about info density, not style. Though I do find style grating sometimes as well.)
I consider a text where I have to skip through entire paragraphs and ignore every 5th filler word (e.g. “fascinating”) to be bad writing, and not inherently enjoyable beyond the kernel of signal there may be in all that noise. And I don’t think I would be being unfair if I demanded this level of quality, because this site is a fragile garden with high standards and maintaining high standards is the same thing as not tolerating mediocrity.
Also everyone has access to the LLMs, and if I wanted an LLM output, I would ask it myself, and I don’t consider your taste in selection to bring me that much value.I also believe (though can’t back this up) that I spend nearly ~ an order of magnitude more time talking to LLMs than the average person on LW, and am a little skeptical of the claim that maybe I’ve been reading some direct LLM output on here without knowing it. Though that day will come.
It also doesn’t take much effort not to paste LLM output outright, so past a certain bar of quality I don’t think people are doing this. (Hypothetical people who are spending serious effort selecting LLM outputs to put under their own name would just be writing it directly in the real world.)
If it doesn’t clutter the UI too much, I think an explicit message near the submit button saying “please disclose if part of your post is copy-pasted from an LLM” would go a long way!
If this is the way the LW garden-keepers feel about LLM output, then why not make that stance more explicit? Can’t find a policy for this in the FAQ either!
I think some users here think LLM output can be high value reading and they don’t think a warning is necessary—that they’re acting in good faith and would follow a prompt to insert a warning if given.
Touching. Thank you for this.
When I was 11 I cut off some of my very-much-alive cat’s fur to ensure future cloning would be possible, and put it in a little plastic bag I hid from my parents. He died when I was 15, and the bag is still somewhere in my Trunk of Everything.I don’t imagine there’s much genetic content left but also I have a vague intuition that we severely underestimate how much information a superintelligence could extract from reality—so I’ll keep onto a lingering hope.
My past self would have wanted me to keep tabs on how the technology is going.
(For those hypothetically wondering, I also cut off some of my own hair that day in case someone would like to clone ME after death, and that bag is still in the Trunk as well.)
(And I couldn’t do this to my cat, but my past self also wrote over a million words about everything he experienced in a google docs in the hopes that the clone could know who his predecessor was and model himself accordingly. I was one of those terrified-of-death kids.)
Can we have a LessWrong official stance to LLM writing?
The last 2 posts I read contained what I’m ~95% sure is LLM writing, and both times I felt betrayed, annoyed, and desirous to skip ahead.
I would feel saner if there were a “this post was partially AI written” tag authors could add to as a warning. I think an informal standard of courteously warning people could work too, but that requires slow coordination-by-osmosis.
Unrelatedly to my call, and as a personal opinion, I don’t think you’re adding any value to me if you include even a single paragraph of copy-and-pasted Sonnet 3.7 or GPT 4o content. I will become the joker next time I hear a LessWrong user say “this is fascinating because it not only sheds light onto the profound metamorphosis of X, but also hints at a deeper truth”.
My calendar reminder didn’t go off, are submissions closed-closed?
Oh yeah no problem with writing with LLMs, only doing it without disclosing it. Though I guess this wasn’t the case here, sry for flagging this.
I’m not sure I want to change my approach next time though, bc I do feel like I should be on my toes. Beware of drifting too much toward the LLM’s stylebook I guess.
Maybe I’m going crazy, but the frequent use of qualifiers for almost every noun in your writing screams of “LLM” to me. Did you use LLM assistance? I don’t get that same feel from your comments, so I’m learning toward an AI having written only the Shortform itself.
If you did use AI, I’d be in favor of you disclosing that so that people like me don’t feel like they’re gradually going insane.If not, then I’m sorry and retract this. (Though not sure what to tell you—I think this writing style feels too formal and filled with fluff like “crucial” or “invaluable”, and I bet you’ll increasingly be taken for an AI in other contexts.)
Hi! I played around with your shortened Neil-prompt for an hour and feel like it definitely lost something relative to the original.
I do quite appreciate this kind of experimentation and so far have made no attempt whatsoever at shortening my prompt, but I should get to doing that at some point. This is directionally correct!
Thanks,