I’d love to see occasional experiments where either completely LLM-generated or lightly edited LLM content is submitted to Less Wrong to see how people respond (with this fact being revealed after). It would degrade the site if this happened too often, but I think it would sense for moderators to occasionally grant permission for this.
I tried an experiment with Wittgenstein’s Language Games and the Critique of the Natural Abstraction Hypothesis back in March 2023 and it actually received (some) upvotes. I wonder how this would go with modern LLM’s, though I’ll leave it to someone else to ask for permission to run the experiment as folk would likely be more suspicious of anything I post due to already having run this experiment once.
If there are models that are that much better than SOTA models, would they be posting to LW? Seems unlikely—but if so, and they generate good enough content, that seems mostly fine, albeit deeply concerning on the secretly-more-capable-models front.
Such an experiment would be better conducted by making a post announcing it at the top and following with chunks of unlabelled human or AI text, like Scott Alexander did for art.
I’d love to see occasional experiments where either completely LLM-generated or lightly edited LLM content is submitted to Less Wrong to see how people respond (with this fact being revealed after). It would degrade the site if this happened too often, but I think it would sense for moderators to occasionally grant permission for this.
I tried an experiment with Wittgenstein’s Language Games and the Critique of the Natural Abstraction Hypothesis back in March 2023 and it actually received (some) upvotes. I wonder how this would go with modern LLM’s, though I’ll leave it to someone else to ask for permission to run the experiment as folk would likely be more suspicious of anything I post due to already having run this experiment once.
We get easily like 4-5 LLM-written post submissions a day these days. They are very evidently much worse than the non-LLM written submissions. We sometimes fail to catch one, and then people complain: https://www.lesswrong.com/posts/PHJ5NGKQwmAPEioZB/the-unearned-privilege-we-rarely-discuss-cognitive?commentId=tnFoenHqjGQw28FdY
Yeah, but how do you know that no one managed to sneak one past both you and the commentators?
Also, there’s an art to this.
If there are models that are that much better than SOTA models, would they be posting to LW? Seems unlikely—but if so, and they generate good enough content, that seems mostly fine, albeit deeply concerning on the secretly-more-capable-models front.
Such an experiment would be better conducted by making a post announcing it at the top and following with chunks of unlabelled human or AI text, like Scott Alexander did for art.
I think both approaches have advantages.