I did’t see that, but yeah I think all but the most blatant examples of AI writing are difficult to spot reliably, especially without substantial false-positive rates. The thing no one seems to get is the current policy doesn’t meaningfully change LLM use for anyone but the scrupulous.
Oliver Kuperman
Karma: 68
I think there are multiple things at play here. First of all, I never claimed that LLMs alone and without substantial human feedback could write on the level of top authors. The 2023 book is an example of a passable book written primarily by AI, which demonstrates how AI can provide a decent level of baseline writing quality.
I suspect AI writing skills have significantly improved since 2023, but (speculatively) several factors have limited it including:
It is likely harder to get AI to be better at creative writing using RL as opposed to math or coding as what makes a particular piece of creative writing “good“ is harder to formalize than whether or not a program works or a math proof is valid.
There has been less incentive for top AI labs to focus specifically on writing, as the market for AI writing tools was always going to be smaller than the market for AI programming tools, and being able to write a great blog post does not clearly move you closer to RSI in the same way that being able to code or do math does.
The people working for and running these AI companies have more expertise in math and coding and so are better designing targeted training for those fields.
Finally, much of writing is about relating real experiences and emotion. AI often lacks a human touch because it is simply not human, and while AI might be able to overcome this, this is a barrier that is simply less prevalent in other fields.
I think the first three bullet points apply decently well to “thinking” too.
Writing skill past a certain point is pretty useless in most people’s everyday lives, especially in our current age. It’s a fine skill to encourage, but I don’t feel like it should be used to gatekeep unless it’s obviously necessary, especially in the light of the long history of people using poor correlates to intellectual capacity and effort to judge others (Examples: Latin, Cursive).
On my use of claude to elucidate my point, I already provided justification for my view that writing skill correlates more with socio-economic status than intelligence or effort: writing is more of an acquired skill shaped by the taste of society than an innate property (like intelligence). I don’t wish to spend much time thinking about a point that is rather trivial.
How long you spend thinking about an idea isn’t based on societal preferences in the way that writing skill is.
Fair enough. I think you could probably create a carve-out for cross-posting, as I believe cross-posts are a small minority of posts. For GoogleDrive, I am not sure if this would be feasible, but allowing users to export a version history and having a tool to verify proof of work could be a solution.
I think your “prompt similar in length to post idea” is decent, but I think there are more sophisticated measures we could create that are less easily gamifiable, for instance, time spent on LessWrong as the primary browser, multiple edits on LessWrong across days.
I have no problem with moderators taking down posts they find to be low quality, I just don’t want a blanket ban on LLM output unless you put all your LLM-Assisted writing in distracting content blocks.
This all reads as a very elitist and privileged perspective. The ability to write well is not the same as the ability to think well, and what constitutes “high-quality writing” is highly subjective. There are numerous studies linking writing status to socio-economic status (example), and it is quite obvious to me that writing skill, which is largely an acquired skill, is correlated more with socio-economic status than raw intelligence (Opus 4.8 agrees with me, ask any chatbot, and it will probably agree too).
At the end of the day, I guess it is inevitable that we need some additional tools besides our own brains to decide if something is worth reading, as even without AI there is only so much writing we can read. However, the year is 2026, where there are more moderation tools than ever before, so why should we still use “writing quality” as the primary metric to judge effort? There are probably far better gauges of this that are available that would allow both human and cyborg writing to flourish.
On your other points, I do not think LLM writing is measurably worse stylistically (after a bit of prompting/editing) than the text produced by most people on this forum, and I do not think most people can reliably distinguish LLM text from human text after the most obvious tells are removed (overuse of em dashes, it’s not just x, it’s also y sentence structure, overly formal, impersonal writing, etc.)
I do not see why the enjoyability of writing is not important, but the enjoyability of reading is (you suggest that LLMs writing might be less “aesthetically” pleasing, but the only purpose of aesthetically pleasing writing is to make the reader). If writing is more enjoyable, better ideas will likely be produced, and it will drive more people to engage on the site than before.
Finally, I am not advocating for one-shot “LLM write me an essay on x” posts; I am advocating for allowing writers to use LLM assistance in a manner that still requires deep reflection and thought. How we can allow the latter without the former is a challenge, but not one that is insurmountable.
Not sure about the 3x bar, but there are numerous people who have either professed the considerable usefulness of using LLMs to assist their writing, or have used AI to produce highly reviewed works. Besides myself, there are Vauhini Vara (Pulitzer Prize finalist), Stephen LaMarche, and also politicians, such as the Lord Mayor of Cork, Fergal Dennehy.
However, I think this exercise is flawed because there are probably a lot of people who use AI to assist their writing without outright disclosing it.
LLM-assisted writing is a positive good, not a necessary evil, and the LessWrong LLM moderation policy should be changed to allow it without requiring the scarlet letter of AI content blocks.
Well crafted LLM-assisted writing is a good thing, and this community should embrace it instead of shunning it with unenforceable rules. I have already talked about the reasons elsewhere, but I don’t think I have emphasized the benefits of LLM-assisted writing enough.
First of all, LLM-assisted writing likely helps people with writing anxiety/ perfectionism. LLM writing allows you to focus on content instead of form by mostly taking care of grammar and flow for you, which is a significant part of writing, and gives you a “good enough” baseline that you would lack otherwise.
LLM-assisted writing helps with writing blocks, as if you are having trouble expressing something you can give an LLM a rough idea and see it iterate.
LLM assisted-writing facilitates instant feedback. If you have an unclear idea, LLMs will likely flag it and help you improve your argument.
Finally, LLM-assisted writing simply makes the writing process more enjoyable, as the aforementioned productivity gains as well as LLMs transforming writing from a purely solitary activity to something more resembling a conversation really make a positive difference.
All of these benefits far outstrip the downsides of allowing high-quality, LLM-assisted writing (which are mostly illusory) and I would be willing to debate this proposition with anyone. LLM-assisted writing has already been implemented elsewhere without complaint, and the harms of allowing it are largely hypothetical and can be counterbalanced by the upvote downvote system already used by LessWrong.
The poll wasn’t about “Do you Agree with Yudkowsky on AI Safety”, it was about whether or not one should take him seriously. Personally, if I think someone’s argument is very bad, I still might think they are worth taking seriously due to their credibility, and assume others believe the same (For instance, I take William Lane Craig decently seriously even if I disagree with his arguments for god). Thus, I still think this poll is the best evidence we have for whether or not Eliezer Yudkowsky is persuasive to a general audience.
I think we agree that low-effort, unreviewed LLM posts are bad, but this should be targeted instead of high effort cyborg writing. I don’t like it that posts which lightly use LLMs need to use these weird LLM boxes.
Blog posts are not public appearances, and LessWrong blog posts are not released to a general audience. Furthermore, I am not a public figure associated with AI safety like Eliezer Yudkowsky is, so even if these posts damage my reputation (unlikely), it would not be a big deal for the AI safety movement. I am not arguing Yudlowsky should stop writing on LessWrong, merely to stop making public appearances or at least take them more seriously and be more careful about his public image.
I don’t think the general populace really understands much about AI safety, so it makes the most sense to focus on appealing to the general public as opposed to casting out a specific net for… steam punk enthusiasts?
How do you explain the results in the Reddit poll I linked to from r/SamHarris then?
If Yudkowsky was not a public figure, I would not care, but empirically, adhering to social norms is a good idea when one is trying to influence society. Maybe Yudkowsky thinks we are already doomed, but I do not so I would prefer it if one of the most prominent contributors to AI safety discourse didn’t dress like a cartoon villain.
The policy should be changed to target what can actually be enforced then. Unenforceable policies on morally neutral behaviors only punish good actors and promote bad ones.
I think you could easily fix this by ensuring you adhere to the same non-LLM style guide and (if both sides are paranoid) having a third party to judge if sandbagging occurred.
I think the current LLM detection system is flawed, so I proposs a bet. In the proposal is accepted I will write 10 3 paragraph less-wrong styles post, 5 AI generated and 5 normal. If the LessWrong LLM detection system correctly assesses 8⁄10 or more, I lose the bet. If the detection system correctly fewer than 8⁄10, I will win the bet. The LLM posts will be fully AI generated, but I will be able to provide it a style sheet and instruct it to mimic my style. Additionally, I can give up to 2 prompts of feedback to modify the post. The human posts would all be generated by me, but would try to follow the same style sheet and use grammarly for spelling check. Finally, the LLM system must run without human intervention. Even if a human judge can “feel” something is LLM, this is not what the experiment is designed to measure. I would be willing to wager up to 500$ on this.
I think even Yudkowsky would agree he is not the most charismatic speaker. Yudkowsk’s advantages come from his intellect, but this advantage can be better deployed in a medium like writing or bf omeone else who is fanilliar with his arguments and a tad bit more charismatic (Nate Soares, for instance). I am not saying for Eliezer Yudkowsky to stop working on AI Safety, just public appearances. Would you like to run a poll on this question and bet on the results?
Nate Soares
What is your background? I feel like if you are living in the bay area, you may have different assumptions than the general population. I am from Florida, and while many people I know wouldn’t be judgemental towards more casual attire in a debate, they would be distracted by Yudkowsky‘s outfit which is both ostentatious and absurd.
This conclusion does not follow from the premises I laid out.
I am not claiming LLM writing hasn’t improved since 2023, just that it’s improvement has not tracked with the improvements in math and coding.
Very few people (using LLMs or not) are going to match the writing quality of the very top writers. That doesn’t mean their writing is worthless or should be banned.
Pure LLM writing might not be great at capturing emotion/first person perspective, but there is nothing preventing LLM-assisted writing from capturing this.