...this is from the “Existential Risks Observatory”, with a detailed website and lots of prestige-signaling like links to news coverage, and which claims to publish research. But a bunch of this reads to me as LLM written? And when I run it through various online LLM-checkers, tons of it is marked as LLM-written. I don’t know why you’d put out a research report like that, and it probably doesn’t meet our standards for LLM content on the site.
Hi Ben, thanks for your comment. You’re right that we have used AI for this research. Since we’re actually arguing that AI can, and sometimes should, be used for defensive actions (such as AI safety research), trying this out ourselves (or working with researchers who do) is not off.
We believe our post does meet the lesswrong AI standards, specifically we have added significant value beyond what the AI produced (for a start, ~50% is not AI-assisted at all), we think our post contains important insights that could reduce existential risk and are not yet widely adopted by the AI safety community, and it therefore does meet a high quality standard. Also, we have spent more than the recommended time editing (some edits where quite heavy and we added many of our own insights), and we vouch for what’s written. AI did help us in writing parts of this post and in coming up with some good ideas we would otherwise not have had.
Still, it’s precisely the goal of our post to discuss how AI can and should be used against xrisk, so if you think this usage is inappropriate or counterproductive, that would also be a good discussion to have, and perhaps useful for others!
The problems with LLM-aided writing are discussed in depth in the linked standards for LLM content on the site.
Whether or not future AI will be useful for protecting against takeover isn’t all that relevant to whether current LLMs should be helping you write and think.
Having said that, I think the LessWrong standards are both fairly strict and fairly vague, so this post probably does meet them, and I think it does contribute to the discussion, because the ideas and claims are human-reviewed in depth. Because it contributes and this is a vital and IMO neglected area, I’ve given it a big upvote.
However, I do find it a bit too LLM-styled for my liking. It’s pretty long and vague relative to the actual ideas and claims. This is the hallmark of “AI slop”. I think you’ve put real thought into this, but I would be more comfortable with a more thoroughly human-written version.
...this is from the “Existential Risks Observatory”, with a detailed website and lots of prestige-signaling like links to news coverage, and which claims to publish research. But a bunch of this reads to me as LLM written? And when I run it through various online LLM-checkers, tons of it is marked as LLM-written. I don’t know why you’d put out a research report like that, and it probably doesn’t meet our standards for LLM content on the site.
Hi Ben, thanks for your comment. You’re right that we have used AI for this research. Since we’re actually arguing that AI can, and sometimes should, be used for defensive actions (such as AI safety research), trying this out ourselves (or working with researchers who do) is not off.
We believe our post does meet the lesswrong AI standards, specifically we have added significant value beyond what the AI produced (for a start, ~50% is not AI-assisted at all), we think our post contains important insights that could reduce existential risk and are not yet widely adopted by the AI safety community, and it therefore does meet a high quality standard. Also, we have spent more than the recommended time editing (some edits where quite heavy and we added many of our own insights), and we vouch for what’s written. AI did help us in writing parts of this post and in coming up with some good ideas we would otherwise not have had.
Still, it’s precisely the goal of our post to discuss how AI can and should be used against xrisk, so if you think this usage is inappropriate or counterproductive, that would also be a good discussion to have, and perhaps useful for others!
The problems with LLM-aided writing are discussed in depth in the linked standards for LLM content on the site.
Whether or not future AI will be useful for protecting against takeover isn’t all that relevant to whether current LLMs should be helping you write and think.
Having said that, I think the LessWrong standards are both fairly strict and fairly vague, so this post probably does meet them, and I think it does contribute to the discussion, because the ideas and claims are human-reviewed in depth. Because it contributes and this is a vital and IMO neglected area, I’ve given it a big upvote.
However, I do find it a bit too LLM-styled for my liking. It’s pretty long and vague relative to the actual ideas and claims. This is the hallmark of “AI slop”. I think you’ve put real thought into this, but I would be more comfortable with a more thoroughly human-written version.
Substanctive comments provided separately