LW is probably not the place for “I asked this LLM (x) and here’s what it said!”, but where is?
I notice a nonzero amount of posts on LW, admittedly typically not overly high karma, that go something like, “I asked Bing to do x and its’ answer freaked me out!”, or “I talked to ChatGPT4 about itself and it told me weird stuff, here are some potential implications”, and although these posts are not excessive, and are sometimes interesting, I can’t help but feel like LW isn’t a great place for them.
I feel a bit like, at least in the current state of LLMs, it’s akin to dream interpretations. They can be valuable to discuss, but are also of primary interest and value to the dreamer (the prompter, in this case) and are of especially low value if the dreamer simply relates them verbatim with little additional commentary.
The reason I ask ‘but where is?’ is because I think a lot of interesting stuff shows up on Twitter, on here and occasionally on Reddit (which is the worst for people just posting the results of prompts verbatim as though Word from God) from folks doing really interesting ‘prompt based research’ into LLMs, and I do absolutely see value in this stuff and the discussions that result from it, and would love a place to collect anecdotal research into how LLMs think and respond besides individual scattered substack articles and tweets.
Just… perhaps LW isn’t the proper forum. I honestly wouldn’t be surprised if the moderation team eventually considers these posts to be inherently low quality and creates some sort of rule against them, but it’s not immediately obvious where the boundary of ‘low quality’ lies.
Site moderator here. This is a good question and one where I’m not that confident of the answer. One challenges is that there are going to be posts of this genre that are pretty interesting e.g. GPT can write Quines now (GPT-4). The issue is maybe that they’re low barrier to entry and easy to make, so you can get more low quality ones.
My guess is that for now users should feel free to attempt to post them to LessWrong, and we’ll handle them in the same way we handle all posts for quality/relevance/etc. We’re building new tools for moderation right now in general (see LW Team is adjusting moderation policy).
First posts will be subject to more scrutiny and might be rejected. “I spoke to GPT-4” will likely have to be very good to be approved as first post.
Automatically applied rate limits will be applied to users who post a lot of content that gets very low karma (I’m hesitant with this, but depending on volume, might be the right tool).
Isn’t the shortform feature perfect for this?
maybe just make a single post and post your stuff as comments to it, if you wanna do it here. it’s a perfectly fine place to do it. I share things when I think the writing has something to say.