People seem to use the terms ‘slop’ or ‘AI sloppiness’ to refer to a wide variety of different issues with pretty different causes, consequences, and solutions. It seems good to typically be more precise, at least in long-form discussion.
These all differ:
AIs have poor capabilties at conceptual / very hard-to-check tasks
AIs aren’t trying on conceptual / very hard-to-check tasks but could do better
AIs “intentionally” produce outputs that are low-quality but appear good
AIs try to make their (low-quality) outputs look good
Many people specifically like AI outputs with a low-quality vibe so the internet fills up with this
AIs make it cheap to produce (mediocre) output so it will floods many places
I agree, though I think the category “conceptual tasks” is similarly broad! Really a lot of tasks involve using/modifying/inventing concepts as needed, some of which current AI can do well; I’m not sure how best to describe the skill that seems missing, but I’m skeptical that “conceptual” or “hard to check” carves it well.
Here’s yet another, quite different, meaning, which I heard used by a researcher at an AI company:
AIs produce output with recurring tics (“it’s not A — it’s B”, “genuinely”, etc.) that people notice and that start to grate on them
Or perhaps more abstractly:
AIs produce output that many people learn to tell at a glance is from AI
I don’t like that meaning because it’s not really about quality at all, or at most it’s about one narrow aspect of the output’s quality. But it seemed clear that those verbal tics are what the person had in mind when they explained they were working on a new “anti-slop” initiative. And it certainly fits that that’s an aspect which an AI company might want to prioritize.
Personally, when I say “slop” I normally mean something that overlaps with some of the meanings you list but is a bit distinct:
An AI produced something low-quality, and a human chose to post that without bothering to bring it up to reasonable quality.
So it’s about the human’s behavior as much as the AI’s performance. Partly that’s because the contexts where I use the term tend to be something to the effect of “don’t post slop here, it’s a waste of our time and yours”. (And because I know the term is ambiguous, I include a few words defining what I mean by it.)
Indexing as 1-6, I’ve never heard of the intent-based 3⁄4 before, even within anti spaces. Not that I’m doubting, but still wondering where.
If I axe those, I feel there are two natural categories:
stylistic slop, the kind of AI slop which the layman is concerned of. “Not X, But Y”, and the targets of Pangram and the like. This kind of slop is essentially a user-laziness + defaults problem, not a capabilities issue. Although it was capabilities gapped a few years ago, at this point it’s trivially easy to apply a bit of prompting creativity to bypass 99% of people’s slop detectors on various social media platforms. The failure of content creators to do so is, as you describe in 6, primarily downstream of the low barriers to entry on this technology.
conceptual slop, which can be sampled via certain crackpot circles or the lesswrong moderation graveyard. These correspond to points 1⁄2 better, and I haven’t spent much time considering the issue.
Although it was capabilities gapped a few years ago, at this point it’s trivially easy to apply a bit of prompting creativity to bypass 99% of people’s slop detectors on various social media platforms.
Can you elaborate? My impression is that at least Claude models struggle immensely to avoid their typical way of speaking (which I find annoying as hell), and I never managed to find a prompt that works to avoid that.
Also when used as more of an aesthetic judgement, AI sloppishness is quite orthogonal to the usual sense of “sloppy”—AI generated content has a certain polish and qualities that would be considered signs of skill and competence when produced by humans, esp. before they became easy to generate with AI and thus less valued. So in this context the problem isn’t that it’s “sloppy” as in looking careless, messy or full of mistakes but that it’s generic, bland, soulless etc. And a lot of creative, interesting and original (opposite of slop) content can be “sloppy” in the sense of being unpolished and imperfect.
People seem to use the terms ‘slop’ or ‘AI sloppiness’ to refer to a wide variety of different issues with pretty different causes, consequences, and solutions. It seems good to typically be more precise, at least in long-form discussion.
These all differ:
AIs have poor capabilties at conceptual / very hard-to-check tasks
AIs aren’t trying on conceptual / very hard-to-check tasks but could do better
AIs “intentionally” produce outputs that are low-quality but appear good
AIs try to make their (low-quality) outputs look good
Many people specifically like AI outputs with a low-quality vibe so the internet fills up with this
AIs make it cheap to produce (mediocre) output so it will floods many places
I agree, though I think the category “conceptual tasks” is similarly broad! Really a lot of tasks involve using/modifying/inventing concepts as needed, some of which current AI can do well; I’m not sure how best to describe the skill that seems missing, but I’m skeptical that “conceptual” or “hard to check” carves it well.
Here’s yet another, quite different, meaning, which I heard used by a researcher at an AI company:
AIs produce output with recurring tics (“it’s not A — it’s B”, “genuinely”, etc.) that people notice and that start to grate on them
Or perhaps more abstractly:
AIs produce output that many people learn to tell at a glance is from AI
I don’t like that meaning because it’s not really about quality at all, or at most it’s about one narrow aspect of the output’s quality. But it seemed clear that those verbal tics are what the person had in mind when they explained they were working on a new “anti-slop” initiative. And it certainly fits that that’s an aspect which an AI company might want to prioritize.
Personally, when I say “slop” I normally mean something that overlaps with some of the meanings you list but is a bit distinct:
An AI produced something low-quality, and a human chose to post that without bothering to bring it up to reasonable quality.
So it’s about the human’s behavior as much as the AI’s performance. Partly that’s because the contexts where I use the term tend to be something to the effect of “don’t post slop here, it’s a waste of our time and yours”. (And because I know the term is ambiguous, I include a few words defining what I mean by it.)
Indexing as 1-6, I’ve never heard of the intent-based 3⁄4 before, even within anti spaces. Not that I’m doubting, but still wondering where.
If I axe those, I feel there are two natural categories:
stylistic slop, the kind of AI slop which the layman is concerned of. “Not X, But Y”, and the targets of Pangram and the like.
This kind of slop is essentially a user-laziness + defaults problem, not a capabilities issue. Although it was capabilities gapped a few years ago, at this point it’s trivially easy to apply a bit of prompting creativity to bypass 99% of people’s slop detectors on various social media platforms. The failure of content creators to do so is, as you describe in 6, primarily downstream of the low barriers to entry on this technology.
conceptual slop, which can be sampled via certain crackpot circles or the lesswrong moderation graveyard. These correspond to points 1⁄2 better, and I haven’t spent much time considering the issue.
Can you elaborate? My impression is that at least Claude models struggle immensely to avoid their typical way of speaking (which I find annoying as hell), and I never managed to find a prompt that works to avoid that.
Just treat it as a combination of low standards and paranoia.
Also when used as more of an aesthetic judgement, AI sloppishness is quite orthogonal to the usual sense of “sloppy”—AI generated content has a certain polish and qualities that would be considered signs of skill and competence when produced by humans, esp. before they became easy to generate with AI and thus less valued. So in this context the problem isn’t that it’s “sloppy” as in looking careless, messy or full of mistakes but that it’s generic, bland, soulless etc. And a lot of creative, interesting and original (opposite of slop) content can be “sloppy” in the sense of being unpolished and imperfect.