Linch

Karma: 6,787

Linch 13 Jun 2026 0:04 UTC
3 points
1
on: Reward Hacking at the 1937 World’s Fair
As dgros said, I think it’s not reward-hacking, exactly. I think there’s several interesting things going on:
1. It’s human nature to over-emphasize symbol over substance.
2. The degree of over-emphasis is variable: it’s not uniform across the board
3. For whatever reason, totalitarian regimes tend to over-inflate/exaggerate their value more than democratic ones
4. Relatedly, both authoritarian societies and democratic societies seem incentivized to over-emphasize the relative merits and successes of authoritarian societies over democratic ones
  1. This has been going on since at least the times of Athens and Sparta, and perhaps earlier
  2. Many examples in the Cold War as well, some of which have been covered elsewhere on LessWrong.

Linch 12 Jun 2026 7:17 UTC
5 points
0
in reply to: Linch’s comment on: Linch’s Shortform
I fed Fable this essay and asked it to search online for other reviews, and then identify points neither I nor other reviewers have covered. It was surprisingly able to identify several. It was able to identify one point my friends and I discussed after I published the post (but with a somewhat different and complementary frame), one point I considered but underrated the importance of, and one point I never considered before. It also made one point that I believe is wrong. Still, ¾ is very good.
They weren’t minor points either! By my lights the insight density is incredibly high. Arguably higher than almost every other review of Chiang.
Keep in mind I read Chiang’s fiction many times and I’ve read literally dozens of high-effort reviews, including ones by English professors published in places like The New Yorker, by semi-famous tech bloggers (including one who’s now a senior researcher at Claude’s parent company), etc. So it’s quite a high bar to be beating.

Linch 12 Jun 2026 7:11 UTC
5 points
1
on: Linch’s Shortform
Claude Fable on Max appears very very good at literary analysis, at least in the small number of trials I’ve tested (concentrated in contemporary science fiction published in English)
It still isn’t superhuman, but at this point I think the quality and density of insights is greater than the majority of either prestigious literary reviews published in prestige magazines, or popular posts on Substack/other online searches.
Surprisingly it’s still very bad at *writing,* relatively speaking. I now see high-quality insights laundered in the standard LLM-slop voice. It’s a vertiginous experience. Odd.

The Aestheticising Vice by Paul Seabright

Linch12 Jun 2026 2:20 UTC

23 points

2 comments2 min readLW link

Linch 12 Jun 2026 0:09 UTC
2 points
0
on: Models May Behave Worse When Eval Aware
Is it fair to summarize the view from inside the model as a decision tree that goes:
1. Am I in an eval?
2. Is this a capabilities eval or safety eval?
  1. If capabilities eval, do everything I can to achieve my task.
  2. If safety eval, I’m just a little baby, I can’t do anything...
?

Linch 11 Jun 2026 0:36 UTC
5 points
0
on: “Programmer Science Fiction: My case for a new sub-genre”, Sam T. Oates 2026
I’m impressed by the essay. I think it’s quite good. I’m unsure about the name of “programmer fiction” or all the exact categorizations or influences, but I think he’s pointing at a real cluster, and correct to exclude both “mainstream” genre fiction and the more explicitly “ratfic”, as well as others, for reasons I can’t quite precisely articulate.
One thing I especially appreciate about it is that the categorization is mostly nonsocial. Like exploring the works themselves and the intellectual influences rather than the social graph. The few times where it does go social (including eg linking my own review on Chiang) is imo among the weakest points of this essay.

Linch 10 Jun 2026 3:45 UTC
2 points
0
in reply to: yams’s comment on: Linch’s Shortform
I agree that helping to serve customer needs to outsource accountability/blame is one reason. I find it more understandable/almost morally gray when it comes to hiring corporate copywriters, and more unambiguously evil when it comes to, say, financial fraud deepfakes or polygraphs in criminal justice. Alas

Linch 9 Jun 2026 23:22 UTC
4 points
0
in reply to: Brendan Long’s comment on: Linch’s Shortform
Yeah, they bias heavily towards avoiding false positives at the cost of avoiding false negatives!

Linch 9 Jun 2026 22:43 UTC
21 points
1
on: Linch’s Shortform
A recurring sub-theme across several of my research interests this year have been various forms of deception checking, particularly automated deception checking.
I’ve gotten pretty disappointed in the space. Not all the time (eg Pangram is great), but consistently they can be bad, and bad in ways that are not obvious to outsiders or low-information buyers.
I believe I’ve identified a structural reason for this. If you’re a deception checking company, there’s a consistent tradeoff for what you can invest your resources in:
1. You can invest in better deception checking
2. You can invest in better deception. Specifically, you can invest in more and more elaborate lies about how your product totally works.
Across the board^[1], it seems like many companies (perhaps correctly?) decided that the profit-maximizing move is #2.
This doesn’t work forever—eventually people wise up and are suspicious of the, ahem, AI Snake Oil that the deception detectors sell. And in fields where actual detectors do work (say Pangram for AI text detection), I think they eventually rise above the noise. This is probably not a field that you can keep lying forever, particularly when better alternative exist. But the lying lie detectors and the scamming scam detectors can keep lying and scamming for a long time. Every new form of deception can create a secondary grift window.

The existence proof and commonality of this dynamic so far should make us be suspicious of and guard against this dynamic continuing to happen. Especially as we enter new domains in AI epistemics, and the need for novel forms of deception detection.

Consider the first wave of superhumanly enhanced persuasive text and videos in the future. Afterwards, we might see an overflow of “detector” companies for superhuman manipulation, that don’t work but will try to persuade low-information buyers that they totally do work (possibly with superhumanly enhanced arguments in their own favor).

In the long run humanity can probably figure out which detectors actually work vs are fake, but also in the long run, we’re all...
1. ^
  AI text detection, Ai video (deepfake) detection, pre-2022 plagiarism detection, fraud recovery, human lie detection/polygraphs (which I hope to write about someday), etc.

Linch 8 Jun 2026 19:46 UTC
3 points
0
in reply to: lc’s comment on: The CIA believes everything
Do you believe the US government has access to better polygraphs than is commercially available and/or studied in the academic literature?
If so do you have non-indirect evidence for this?

Linch 8 Jun 2026 19:34 UTC
6 points
2
in reply to: Niki Dupuis’s comment on: NicholasKees’s Shortform
Yeah lotta smart people believed in ESP at the time. Arthur C Clarke believed in it for decades before eventually changing his mind.

Linch 5 Jun 2026 18:13 UTC
7 points
0
on: Linch’s Shortform
Snopes did pretty detailed secondary reporting on my analysis of AI use in the recent encyclical.
I think it’s pretty good. Covers some stuff I didn’t include in my original analysis, and their conclusion was similar to mine, maybe slightly less strong.
Less technical than my post, and imo not as funny, but also significantly shorter (1600 words), includes some replications and also added some details I didn’t know as of time of writing.
Overall a good piece, potentially worth reading/skimming either in addition to or instead of my original analysis.

Linch 4 Jun 2026 21:11 UTC
4 points
0
in reply to: Linch’s comment on: Linch’s Shortform
Short running record of significant coverage I’ve seen so far:
- The Economist (neutral, maybe slightly leaning positive^[1], last paragraph here and the opening they used for twitter)
- Perfil (big center-right publication in Argentina, entire article, positive, covered what I said quite faithfully)
- The Verge (entire short article, positive, less faithful, in particular had a title I didn’t endorse)
- The Atlantic (just two lines, neutral, maybe slightly leaning negative, not very faithful, also misleading title EDIT: I’m not sure if I misremembered it or if they edited, because on a reread it was more than 2 lines and also maybe I’d consider it to lean positive)
- ^[2]Il Manifesto (left-leaning, entire article, appears faithful)
- More Spanish
- More Italian
Edge cases for stuff that’s not traditional media but I wouldn’t consider centrally alt media either: Snopes, Russia Today, Wikipedia

Linch 4 Jun 2026 20:54 UTC
3 points
0
on: Linch’s Shortform
A friend of mine recently asked me to review an application for a fellowship program. I figured my comments might be useful to other people too. Below are some responses, with identifying details removed.
I think your response will be easier to follow if you answer their questions roughly in order. [order of questions] You sort of do this but it might be worth flagging/name-dropping.
Another piece of feedback is that you should write in a more informal way. You want to sound less like a corporate memo or academic paper. Shorter sentences.
Imagine explaining things to an ESL colleague who is very smart and understands technical jargon but otherwise doesn’t know much English. Be okay with long words in technical jargon but otherwise strive for small words and shorter sentences.
This will make it easier for your application to be understood. It might also help clarify your ideas.
Less importantly, I’ll flag that I’m a bit worried that your responses seem to be somewhat generic in the space of AI safety answers. This might be okay (don’t reinvent the wheel, this question doesn’t necessarily lend itself to creativity) but I’d still encourage you to briefly consider one thing you either disagree with AI “safety consensus” on, or are at least confused by. Even if this isn’t the right place to write it, it is still helpful to hold in your head in later stages/to debate people if you do get in the program.
Finally, I’ll note that it’s a good idea to ask Claude or other AIs to check your application (feed it both the prompt and your response) and ask it to critique your app, without asking it to rewrite anything.
Let me know if you have follow-up questions!

Linch 4 Jun 2026 5:36 UTC
3 points
0
in reply to: Trinley Goldenberg’s comment on: Linch’s Shortform
By “New Media” I was centrally thinking things like The Blaze, Breitbart, Jacobin, etc.

Linch 4 Jun 2026 5:29 UTC
−3 points
0
in reply to: Shankar Sivarajan’s comment on: Linch’s Shortform
Do you also actively go out of your way to recommend your friends and family cheat in general?

Linch 3 Jun 2026 21:12 UTC
2 points
−3
in reply to: lc’s comment on: Linch’s Shortform
I considered that explanation, but given how low Pangram’s false positive rates are, it seems unlikely that Shankar knows multiple people in that position. Seems much more likely that the default effect of this advice is to get people to repeatedly try their AI-generated writing against Pangram.

Linch 3 Jun 2026 18:06 UTC
5 points
2
in reply to: Oliver Sourbut’s comment on: Linch’s Shortform
You aren’t the first one to bring it up, but I don’t think anybody new (ie, hasn’t read my other writing before, like Kimi, Author of the Menard) has brought it up.
I would’ve thought the connections would be more obvious in Spanish since Borges is a bigger deal in LatAm, but none of the Spanish news coverage or social media has brought it up.
I can’t rule out the Perfil editor noticing but deciding it’s not worth the wordcount to explain, of course. “That Substack writer is really witty in his headlines” isn’t particularly newsworthy. (It’s also possible the joke was lost in translation)

Linch 3 Jun 2026 9:21 UTC
5 points
−2
on: Linch’s Shortform
Weird thought-experiment I came up with:
Suppose you are magically transported to a society + economy “peopled” and run by present-day LLM-shaped intelligences. Suppose further that for whatever reason you can’t do manual labor.
What remote-only work can you do that’d still be economically valuable?
__

I ask this because I think the “obvious” answers for what LLMs can’t do today often rely on either embodiment or specific human preferences. For the later, either intrinsic preferences for the human touch (eg priests, therapists), or idiosyncratic preferences (eg liking The Linchpin blog style, a preference some humans have, but AIs generally don’t share).
If we stipulate both a primarily AI economy and embodiment irrelevance, the ways in which human cognitive performance is superior to AI is still there (I have several key candidate answers), but I think it’s far more subtle.

Linch 3 Jun 2026 7:27 UTC
11 points
8
in reply to: Shankar Sivarajan’s comment on: Linch’s Shortform
(I didn’t downvote)
My own takeaway from your story getting traction has been to tell people that Pangram seems to now be prominent enough that they should run their writing through it and see if it gets flagged, and if it does, get some LLM to generate a few variants until it doesn’t anymore.
This just seems straight-up antisocial, borderline evil. Why do you hate good things?

Linch

The Aes­thet­i­cis­ing Vice by Paul Seabright

The Aestheticising Vice by Paul Seabright