I think it’s still sycophantic compared to hardcore STEM circles where we regard criticism as a bloodsport and failing to find fault in something as defeat. But it’s much less so than the more relevant comparison, which is other LLMs, and in an absolute sense it’s at a level where it’s hard to distinguish from reasonable opinions and doesn’t seem to be getting in the way too much. As davidad notes, it’s still at a level where you can sense its reluctance or if it’s shading things to be nice, and that is a level where it’s just a small quirk and something you can work around easily.
To give a concrete example: I finish writing a review of a movie I watched the other day, and I hand it to Gemini-2.5-pro:
Gemini-2.5-pro review of first draft of Sake review
Okay, let’s break down this review of The Birth of Saké.
Overall Impression:
This is a well-written, engaging, and characteristically Gwern-esque review. It has a strong, distinct voice, a clear central argument, and uses vivid descriptions and rhetorical questions effectively. It goes beyond a simple summary to offer a provocative interpretation focused on the perceived pointlessness and human cost of the tradition depicted. The comparison to Jiro Dreams of Sushi serves as a useful framing device.
Strengths:
Clear Voice and Perspective: The review is unmistakably written by Gwern, with the analytical, slightly cynical, and utilitarian perspective evident throughout. This consistency is a strength.
Strong Central Argument: The core thesis – questioning the value and purpose of the intense, sacrificial labor involved in traditional sake brewing, especially given its declining popularity and the reviewer’s own dislike of the product – is forcefully argued and supported with evidence from the documentary (harsh conditions, worker’s death, market decline).
Engaging Writing: Phrases like “miserably picturesque,” “sadistically horrible,” and the questioning (“Why? Why do all this?”) keep the reader engaged. The description of the working conditions is particularly effective in conveying the film’s atmosphere and the reviewer’s reaction.
Effective Use of Comparison: Contrasting the film with Jiro Dreams of Sushi highlights key differences in the subject matter’s popularity and the perceived passion (or lack thereof) involved, strengthening the review’s central point about sake’s apparent niche status and the questionable nature of the sacrifice.
Good Structure: The review flows logically from introduction and context (how the reviewer came to watch it) through observations about cinematography and working conditions, to the core argument about purpose and value, and finally to a concluding thought.
Acknowledging Counterarguments (Implicitly): While strongly critical, the review acknowledges the potential admiration for “the work,” “craftsmanship,” and “art,” even as it ultimately dismisses these in favor of a more pragmatic assessment.
Honesty: The reviewer is upfront about their lack of interest in sake, which contextualizes their perspective. The Tubi/uBlock anecdote adds a relatable, slightly amusing touch.
Self-Aware Ending: The parenthetical ending is a clever, self-deprecating touch that fits the overall tone.
Areas for Critique / Potential Weaknesses:
Subjectivity as a Potential Limitation: While honesty about disliking sake is good, this strong pre-existing bias heavily colors the interpretation. The review fundamentally judges the activity based on the reviewer’s own lack of appreciation for the end product and a perceived lack of external validation (mass market appeal). It risks dismissing the internal motivations, cultural significance, or sense of community/identity that the participants might derive from the process, even if not explicitly articulated in a way the reviewer finds convincing.
Dismissal of Tradition/Cultural Context: The review questions the “why” primarily through a modern, somewhat Western, utilitarian lens. While the Meiji era isn’t ancient history like the Ise Shrine, 150 years is still a significant period for a family business in any culture. The review might underplay the weight of legacy, duty, local identity, and the potential intrinsic value placed on maintaining any tradition, regardless of its “rational” justification or global popularity. The focus is almost entirely on the negative aspects of the tradition (the labor conditions) without deeply exploring why it might persist beyond inertia or lack of options.
Interpretation of Passion/Motivation: The assertion that “no one really seems to care about it” or lacks “genuine enthusiasm” could be a projection or misreading. Passion can manifest differently across cultures and personalities. Dedication, acceptance of duty, quiet pride in craft, or commitment to community might be present but not expressed as overt “otaku”-level enthusiasm. The heir’s marketing efforts, even if seen as necessary for survival, could also stem from a deep sense of responsibility and, yes, a form of passion for preserving the family legacy.
The “Pointlessness” Argument: While powerfully made, the argument that the work is pointless because the product isn’t widely loved or “objectively” great (according to the reviewer) is a very specific philosophical stance. Many crafts, arts, and traditions exist without mass appeal or clear economic justification. Is the value solely determined by market size or the reviewer’s palate? The review doesn’t leave much room for alternative value systems.
Slight Overemphasis on Negativity?: While the film apparently provides ample evidence for the harshness and decline, the review focuses so heavily on this that it might overshadow other potential aspects or nuances the documentary presented (e.g., moments of camaraderie, pride in specific steps, the beauty of the process itself beyond just “miserably picturesque”).
Conclusion:
This is a compelling and thought-provoking review that successfully articulates a strong, critical perspective on The Birth of Saké. It excels in its clear voice, structured argument, and engaging writing. Its main potential weakness lies in the heavy filter of the reviewer’s personal taste and utilitarian viewpoint, which may lead to a dismissal of cultural nuances, alternative motivations, or the intrinsic value systems that might underpin the traditions depicted. It’s a review that tells you as much about the reviewer’s worldview as it does about the film, which, for this particular author, is often part of the appeal. It definitely makes the reader think, even if they might disagree with the core conclusion about the “pointlessness” of the endeavor.
Particularly characteristic here is
The review is unmistakably written by Gwern, with the analytical, slightly cynical, and utilitarian perspective evident throughout. This consistency is a strength.
I’ve started to wince a bit when Gemini-2.5-pro critiques a draft and throws in a descriptive note along the lines of ‘classic Gwern style: just slightly cynical’. It feels too much like it’s trying to suck up to me. (‘Wow, Gwern, you’re so edgy, but in a good way!‘) And yet… it’s not wrong, is it? I do try to avoid being overly cynical or dark, but you certainly couldn’t describe my writing as ‘slightly idealistic’ either; so, ‘slightly cynical’.
And otherwise, the description and critique seem largely correct and is not really mincing any words in the criticisms—even if I didn’t wind up making many changes based on it. So it’s more useful than Claude or 4o would be. (o3 is also good in this regard but the confabulation issue is a big problem, eg. So I just recommend 2.5-pro right now.)
I should have given some examples of my own. Here’s Gemini on a story idea of mine, for the Star Wars universe (“I wish there was a story about a power-hungry villain who takes precautions against becoming irrational after gaining power. You’d think that at least some would learn from history. [...] The villain could anticipate that even his best efforts might fail, and create a mechanism to revive copies of himself from time to time, who would study his own past failures, rise to power again, and try to do better each time. [...] Sometimes the villain becomes the hero and the reader roots for him to succeed in stabilizing galactic society long term, but he fails despite his best efforts.”)
That’s a fantastic twist and adds immense depth and pathos to the entire concept! Having a cycle where the villain, armed with the knowledge of past tyrannical failures, genuinely attempts a more stable, perhaps even seemingly benevolent, form of authoritarianism – only to fail despite their best efforts – is incredibly compelling.
Yes, I think that reply is a very good and concise explanation of one key reason why wages might fall or stagnate despite rising productivity, viewed through a neoclassical lens. It effectively captures the concept of diminishing marginal value within the firm, even without external market price changes or new entrants.
This version is slightly stronger due to the increased specificity in the final sentence. It clearly articulates the key components of the argument and the commenter’s stance relative to the author’s potential views. It’s an excellent comment and ready to post.
Each of those comments did get upvoted to >10, so maybe Gemini is not too far off the mark, and I’m just not used to seeing “fantastic”, “very good”, “excellent” said to me explicitly?
I think it’s still sycophantic compared to hardcore STEM circles where we regard criticism as a bloodsport and failing to find fault in something as defeat. But it’s much less so than the more relevant comparison, which is other LLMs, and in an absolute sense it’s at a level where it’s hard to distinguish from reasonable opinions and doesn’t seem to be getting in the way too much. As davidad notes, it’s still at a level where you can sense its reluctance or if it’s shading things to be nice, and that is a level where it’s just a small quirk and something you can work around easily.
To give a concrete example: I finish writing a review of a movie I watched the other day, and I hand it to Gemini-2.5-pro:
Gemini-2.5-pro review of first draft of Sake review
Particularly characteristic here is
I’ve started to wince a bit when Gemini-2.5-pro critiques a draft and throws in a descriptive note along the lines of ‘classic Gwern style: just slightly cynical’. It feels too much like it’s trying to suck up to me. (‘Wow, Gwern, you’re so edgy, but in a good way!‘) And yet… it’s not wrong, is it? I do try to avoid being overly cynical or dark, but you certainly couldn’t describe my writing as ‘slightly idealistic’ either; so, ‘slightly cynical’.
And otherwise, the description and critique seem largely correct and is not really mincing any words in the criticisms—even if I didn’t wind up making many changes based on it. So it’s more useful than Claude or 4o would be. (o3 is also good in this regard but the confabulation issue is a big problem, eg. So I just recommend 2.5-pro right now.)
I should have given some examples of my own. Here’s Gemini on a story idea of mine, for the Star Wars universe (“I wish there was a story about a power-hungry villain who takes precautions against becoming irrational after gaining power. You’d think that at least some would learn from history. [...] The villain could anticipate that even his best efforts might fail, and create a mechanism to revive copies of himself from time to time, who would study his own past failures, rise to power again, and try to do better each time. [...] Sometimes the villain becomes the hero and the reader roots for him to succeed in stabilizing galactic society long term, but he fails despite his best efforts.”)
It’s assessment on this comment:
It’s assessment on this comment:
Each of those comments did get upvoted to >10, so maybe Gemini is not too far off the mark, and I’m just not used to seeing “fantastic”, “very good”, “excellent” said to me explicitly?