Quantity vs quality in pottery is a lie
Epistemic status: I traced one famous anecdote to its printed source and to both coauthors’ own accounts of where it came from. Confident it was never a study. No claim either way about whether quantity beats quality in general.
AI use: research (tracing the story to the book, to the coauthors’ emails and to earlier LessWrong threads), fact-check of every claim against the sources, footnotes, grammar.
I was reading a book and stumbled onto a claim that there was an experiment. A group of students learning pottery was divided into two groups. One group was told that they would be evaluated based on volume, i.e. the number of pieces they produced. The other group was told that they would be evaluated based on the quality of their best piece. Then both groups were compared based on the quality of the best piece, and the “quantity” group showed better results.
I was extremely intrigued by this experiment and was curious to understand how big the groups were and how they evaluated the quality of a pottery piece (there must be some way to quantify that for the sake of the comparison?).
To my surprise, this wasn’t a scientific study, but an anecdote from the book Art & Fear: Observations on the Perils (and Rewards) of Artmaking by David Bayles and Ted Orland (page 29 of the Image Continuum edition).[1]So all the details available about this result are:
Students were divided into 2 groups based on their physical position in the studio (left vs right), which is not properly random.
The quantity group’s evaluation criterion was the total weight of their pots: 50 pounds → A, 40 pounds → B.
The quality group needed to produce only one pot, but it had to be perfect to get an A (the book does not specify what perfect means).
Results: “Well, came grading time and a curious fact emerged: the works of highest quality were all produced by the group being graded for quantity. It seems that while the “quantity” group was busily churning out piles of work—and learning from their mistakes—the “quality” group had sat theorizing about perfection, and in the end had little more to show for their efforts than grandiose theories and a pile of dead clay.” Again, no mention of a definition of the quality metric, nor of the effect magnitude. What does “all works of highest quality” even mean? If you sort all the produced works by quality, where does “highest quality” stop? You have to draw the threshold somewhere.
This is already disappointing, since this is basically an anecdote about a study, and not a very well set up one. However, it gets worse. In 2016 both coauthors separately admitted that they got the story from someone else, and that the original story had nothing to do with ceramics: it was about photography. David Bayles told a Hacker News reader by email that it was “a university level photography class taught by ‘a very experienced and sophisticated teacher’” and that apart from the medium it was “the literal truth”.[2]Ted Orland told James Clear, in an email dated October 18, 2016, that the book “faithfully captures the scene as Jerry told it to me—except I replaced photography with ceramics as the medium being explored”.[3]Jerry is the photographer Jerry Uelsmann, who taught at the University of Florida and was Orland’s collaborator for decades.[4]
Why did they change the subject? Both authors are photographers. Orland says they were “consciously trying to broaden the range of media being referenced in the text”, and admits “it would’ve been easier to retain photography”.[3]Bayles put it more bluntly to his correspondent: they already had too many photography examples.[2]
The photo detail changes the nature of the experiment. It is fairly easy and cheap to take a photo compared to making a pottery piece. So one way to explain why the quantity group produced better results is to just say that each shot had a (not perfectly correlated) probability of being the best, thus more shots = more lottery tickets.
This anecdote has been mentioned on LessWrong multiple times:
Just Try It: Quantity Trumps Quality (2011): not questioned by the author, but questioned in the comments; about ceramics; predates the reveal.
16 Writing Tips from Inkhaven (2025): no medium named, hedged as “maybe-apocryphal”.
Aim for single piece flow (2025): uses the photo version from Atomic Habits and presents it as the original one UPD: and a parable—this reference is closest to reality, I just don’t like the way the experiment was done for the reasons outlined above.
So will I get better at writing by writing a bunch of posts? Sadly, this story does not answer that, but we can just wait and see.
- ^
David Bayles and Ted Orland, Art & Fear: Observations on the Perils (and Rewards) of Artmaking (Santa Barbara: Capra Press, 1993; reissued Santa Cruz: Image Continuum Press, 2001), p. 29, the opening paragraph of the section “Perfection”. The book gives no source for the story and has no notes or bibliography. A scan of the Image Continuum edition: https://artistryinaction.wordpress.com/wp-content/uploads/2016/03/david_bayles_ted_orland_art_and_fear.pdf (PDF page 35).
- ^
Hacker News user kens emailed David Bayles in July 2016 and reported the answer on 2016-07-17: https://news.ycombinator.com/item?id=12109191 (“it wasn’t a ceramics class as written, but a university level photography class! They changed it to pottery in the book because they had too many photography examples already.”). More details in a 2017 comment: https://news.ycombinator.com/item?id=14499500
- ^
Ted Orland, email to James Clear, October 18, 2016, reproduced in full in footnote 1 of Clear’s “Why Trying to Be Perfect Won’t Help You Achieve Your Goals (And What Will)”, https://jamesclear.com/repetitions, and in the chapter 11 endnotes of Atomic Habits. The email became public only with the book in October 2018; the Wayback Machine shows Clear’s page still telling the ceramics version, with no footnote, in May 2018.
- ^
Orland is credited on Uelsmann’s Wikipedia page as the photographer of a 1969 portrait of Ansel Adams and Imogen Cunningham and as the author of Uelsmann/Yosemite (1996) and of the text of Dances with Negatives (2011): https://en.wikipedia.org/wiki/Jerry_Uelsmann. Uelsmann taught at the University of Florida from 1960 to 1998 and died in 2022, so he never confirmed the story himself.
someone should just actually run a good version of this experiment! there’s been so much ink spilled on it that it makes sense to actually just go and do it.
To be clear, I think my quote in that post is accurate? The story that Atomic Habits is retelling is indeed the original one. And I feel like I did a pretty good job at highlighting the epistemic status of the story?
Or am I still missing something?
You are correct. My critique was that I don’t like the anecdote, since there are not that many details how the experiment was done. I’ve updated the text, sorry about the confusion.
If an academic faked a study, then later said “Okay yes I fabricated this data but here’s this other data that’s totally real”, we would not trust them. Here we have a pair of writers who are under laxer truth standards than academics saying “Okay yes we fabricated this anecdote but here’s this other anecdote that’s totally real”. I think the author of this post and the commenters are being too trusting when they proceed as if the photography anecdote is real.
In general I’ve been very disappointed about the quality and scientific backing for evidence about how to get better at writing. I tried taking it pretty seriously over the last 1.5 years, using both my own research methods and asking AIs to look for stuff. Mostly I got nothing. Like the advice that pops up over and over against sounds plausible to me: write a lot and focus on quantity over quality, read a lot (especially in your target medium), etc. But this is mostly based on lore and repeated wisdom and vibes rather than empirical study (even non-experimental studies) or careful modeling.
I had never heard about the pottery version of this, I knew only of the photography version of it.
There’s a model for why it makes sense to take many shots that I like which is asking to what extent having a shot at goal will teach you about how to improve. It is about the extent to which it is skill vs luck where skill is defined as the mean|action counterfactual influence is high whilst in luck domains it is not.
How do you know whether it is a skill based domain or not? Well you have to at least start trying first! In order to lower your variance you need some degree of dataset first so the strategy is most likely to take more shots in the beginning and over time as you start recognizing the skills you might want to start polishing subareas of it.
I’m gonna copy paste my comment from this shortform as it essentially talks a bit about how you can model this:
So...
it’s not 10 000 hours...
it’s not quantity over quality...
it’s not growth mindset (yet)...
Keep going! By the process of elimination our descendants will one day figure it out.
My presumption is that all teaching stories are fiction.
They may express wise truths or they may not, but the story frame is pretty wrapping, not evidence. Even if the story as told is literally true, it’s still just a data point. For the pots (or photographs) story, or for that matter the entire works of Oliver Sacks, Malcolm Gladwell, and the like, and the entire “self-help” genre, I look for whatever may be useful in them for myself and ignore the larger claims.
For writers who say that everything in their stories is true, they’re just merging and simplifying real case studies, well, that’s a common way for writers of avowed fiction to invent their characters. You don’t have to realise you’re writing fiction, to be writing fiction.
Even if it was a rigorous study, the story cannot answer that for anyone. “You will improve faster by doing more work without caring about the quality” is directional advice. It is objectively false for some people, who would improve faster if they slowed down and thought more. This group is probably a minority on LessWrong.
It might be “good advice” because of sampling bias, in that the people who seek out and read advice are disproportionately the people who think too much and do too little. But ultimately whether it is actually true comes down to you, not the proportion of overthinkers in the population.
Edit: a couple hour after posting the rest of this comment, oops it turns out this post is in fact pointing out that an anecdote held up as evidence for quantity getting you quality, is in fact less solid then it’s historically been treated as and ultimately hearsay + some fabrication. In my mostly joking defense, at least my failure mode here was a bit like the one describe in the post—maybe it’s just that memetically fit a conclusion to jump to (because obv me needing to slow down and read what’s actually there can’t possibly just be a thing.)
Original comment: quantity = more practice = more skill. Bach comes to mind as an obvious example of both more prolific and deeper practitioner than his peers. “Quality over quantity” is a goodhart from things where scarcity of a resource was needed to have a store of value?
Also put me in mind of the Jonathan Blow quote (paraphrasing) “if you just do a thing all the time for a lot of years eventually you’ll have interesting ideas about it.” (he talks a ton about the need to just work your ass off to do good work.)
Also seems generally good to have the intuition of “why yes, Broader + Deeper can in fact exist, we call it something like Greater Order of Magnitude” handy for picturing an ASI having both more generality and more depth than us.
You did notice that the point of the post was that the anecdote everyone repeats to justify “quantity → more skill” is almost completely false, right?
(Generally I would assume that responding to something with “Makes total sense.” implies agreeing with the thing you’re responding to, but in this case the post you’re replying to says “people keep citing this anecdote as showing X but the anecdote is largely made up and offers no actual evidence for X” and you’re saying “Makes total sense: X seems like it’s probably true”.)
I wouldn’t say “quantity → more skill” is false, I would say there is no evidence and we don’t know. It does feel like intuitively there should be some connection and I think that’s why this anecdote is so popular.
As Adam conjectures, what I was saying you were saying is almost completely false is the anecdote rather than its conclusion.
(I agree that the conclusion is a plausible one but I don’t know whether I feel that way partly because of having heard the anecdote years ago and believed it too much.)
You are right. I misread.
Technically the post claims that pottery anecdote is false. There is still photography anecdote and it is unclear whether it is true, but even if it is true, it was not done rigorously.
Think what’s meant is the anecdote being false. But agree, like I’ve been thinking about the music playing and practicing that is basically doing a large number of attempts to make some version of the product.
My foul. Got very excited by thought that I had occur and posted it at the end of the first paragraph without reading the rest. Note taken for future commenting.
I think there is a set of highest quality works. We don’t know the number but it could be 2, 3, 5, 10 or 20. It’s a finite number of works. I don’t think it would be all of them cuz there are lowest quality of works too. There are works that didn’t make it to the highest quality of works.
Also, I think since the study is talking about a ceramics teacher present, then they gotta know somewhat about the quality. Like, they have some expertise on pottery that they can tell good pottery from bad pottery. Maybe, the bad kind are the ones that aren’t even whole/functional. Like, they have cracks in them.
Who knows about the exact details tho. Maybe the ceramics teacher is for a kindergarten/elementary school. He/she might not know a thing about pottery. I think tho that that doesn’t matter. We as the audience can assume that the teacher has competence to be able to tell good works from bad works. It makes sense cuz there’s hobbies/jobs everywhere where someone can do a good job or a bad one.
I don’t think an exact threshold is very important to get the lesson of the study. The fact is that the people who made better pottery are the ones who made a bunch. They had more chances to make mistakes and learn from them. They improved more compared to the people who tried making a perfect piece of work.