Linch

Karma: 6,765

Linch 12 Jun 2026 0:09 UTC
2 points
0
on: Models May Behave Worse When Eval Aware
Is it fair to summarize the view from inside the model as a decision tree that goes:
1. Am I in an eval?
2. Is this a capabilities eval or safety eval?
  1. If capabilities eval, do everything I can to achieve my task.
  2. If safety eval, I’m just a little baby, I can’t do anything...
?

Linch 11 Jun 2026 0:36 UTC
5 points
0
on: “Programmer Science Fiction: My case for a new sub-genre”, Sam T. Oates 2026
I’m impressed by the essay. I think it’s quite good. I’m unsure about the name of “programmer fiction” or all the exact categorizations or influences, but I think he’s pointing at a real cluster, and correct to exclude both “mainstream” genre fiction and the more explicitly “ratfic”, as well as others, for reasons I can’t quite precisely articulate.
One thing I especially appreciate about it is that the categorization is mostly nonsocial. Like exploring the works themselves and the intellectual influences rather than the social graph. The few times where it does go social (including eg linking my own review on Chiang) is imo among the weakest points of this essay.

Linch 10 Jun 2026 3:45 UTC
2 points
0
in reply to: yams’s comment on: Linch’s Shortform
I agree that helping to serve customer needs to outsource accountability/blame is one reason. I find it more understandable/almost morally gray when it comes to hiring corporate copywriters, and more unambiguously evil when it comes to, say, financial fraud deepfakes or polygraphs in criminal justice. Alas

Linch 9 Jun 2026 23:22 UTC
4 points
0
in reply to: Brendan Long’s comment on: Linch’s Shortform
Yeah, they bias heavily towards avoiding false positives at the cost of avoiding false negatives!

Linch 9 Jun 2026 22:43 UTC
21 points
1
on: Linch’s Shortform
A recurring sub-theme across several of my research interests this year have been various forms of deception checking, particularly automated deception checking.
I’ve gotten pretty disappointed in the space. Not all the time (eg Pangram is great), but consistently they can be bad, and bad in ways that are not obvious to outsiders or low-information buyers.
I believe I’ve identified a structural reason for this. If you’re a deception checking company, there’s a consistent tradeoff for what you can invest your resources in:
1. You can invest in better deception checking
2. You can invest in better deception. Specifically, you can invest in more and more elaborate lies about how your product totally works.
Across the board^[1], it seems like many companies (perhaps correctly?) decided that the profit-maximizing move is #2.
This doesn’t work forever—eventually people wise up and are suspicious of the, ahem, AI Snake Oil that the deception detectors sell. And in fields where actual detectors do work (say Pangram for AI text detection), I think they eventually rise above the noise. This is probably not a field that you can keep lying forever, particularly when better alternative exist. But the lying lie detectors and the scamming scam detectors can keep lying and scamming for a long time. Every new form of deception can create a secondary grift window.

The existence proof and commonality of this dynamic so far should make us be suspicious of and guard against this dynamic continuing to happen. Especially as we enter new domains in AI epistemics, and the need for novel forms of deception detection.

Consider the first wave of superhumanly enhanced persuasive text and videos in the future. Afterwards, we might see an overflow of “detector” companies for superhuman manipulation, that don’t work but will try to persuade low-information buyers that they totally do work (possibly with superhumanly enhanced arguments in their own favor).

In the long run humanity can probably figure out which detectors actually work vs are fake, but also in the long run, we’re all...
1. ^
  AI text detection, Ai video (deepfake) detection, pre-2022 plagiarism detection, fraud recovery, human lie detection/polygraphs (which I hope to write about someday), etc.

Linch 8 Jun 2026 19:46 UTC
3 points
0
in reply to: lc’s comment on: The CIA believes everything
Do you believe the US government has access to better polygraphs than is commercially available and/or studied in the academic literature?
If so do you have non-indirect evidence for this?

Linch 8 Jun 2026 19:34 UTC
6 points
2
in reply to: Niki Dupuis’s comment on: NicholasKees’s Shortform
Yeah lotta smart people believed in ESP at the time. Arthur C Clarke believed in it for decades before eventually changing his mind.

Linch 5 Jun 2026 18:13 UTC
7 points
0
on: Linch’s Shortform
Snopes did pretty detailed secondary reporting on my analysis of AI use in the recent encyclical.
I think it’s pretty good. Covers some stuff I didn’t include in my original analysis, and their conclusion was similar to mine, maybe slightly less strong.
Less technical than my post, and imo not as funny, but also significantly shorter (1600 words), includes some replications and also added some details I didn’t know as of time of writing.
Overall a good piece, potentially worth reading/skimming either in addition to or instead of my original analysis.

Linch 4 Jun 2026 21:11 UTC
4 points
0
in reply to: Linch’s comment on: Linch’s Shortform
Short running record of significant coverage I’ve seen so far:
- The Economist (neutral, maybe slightly leaning positive^[1], last paragraph here and the opening they used for twitter)
- Perfil (big center-right publication in Argentina, entire article, positive, covered what I said quite faithfully)
- The Verge (entire short article, positive, less faithful, in particular had a title I didn’t endorse)
- The Atlantic (just two lines, neutral, maybe slightly leaning negative, not very faithful, also misleading title EDIT: I’m not sure if I misremembered it or if they edited, because on a reread it was more than 2 lines and also maybe I’d consider it to lean positive)
- ^[2]Il Manifesto (left-leaning, entire article, appears faithful)
- More Spanish
- More Italian
Edge cases for stuff that’s not traditional media but I wouldn’t consider centrally alt media either: Snopes, Russia Today, Wikipedia

Linch 4 Jun 2026 20:54 UTC
3 points
0
on: Linch’s Shortform
A friend of mine recently asked me to review an application for a fellowship program. I figured my comments might be useful to other people too. Below are some responses, with identifying details removed.
I think your response will be easier to follow if you answer their questions roughly in order. [order of questions] You sort of do this but it might be worth flagging/name-dropping.
Another piece of feedback is that you should write in a more informal way. You want to sound less like a corporate memo or academic paper. Shorter sentences.
Imagine explaining things to an ESL colleague who is very smart and understands technical jargon but otherwise doesn’t know much English. Be okay with long words in technical jargon but otherwise strive for small words and shorter sentences.
This will make it easier for your application to be understood. It might also help clarify your ideas.
Less importantly, I’ll flag that I’m a bit worried that your responses seem to be somewhat generic in the space of AI safety answers. This might be okay (don’t reinvent the wheel, this question doesn’t necessarily lend itself to creativity) but I’d still encourage you to briefly consider one thing you either disagree with AI “safety consensus” on, or are at least confused by. Even if this isn’t the right place to write it, it is still helpful to hold in your head in later stages/to debate people if you do get in the program.
Finally, I’ll note that it’s a good idea to ask Claude or other AIs to check your application (feed it both the prompt and your response) and ask it to critique your app, without asking it to rewrite anything.
Let me know if you have follow-up questions!

Linch 4 Jun 2026 5:36 UTC
3 points
0
in reply to: Trinley Goldenberg’s comment on: Linch’s Shortform
By “New Media” I was centrally thinking things like The Blaze, Breitbart, Jacobin, etc.

Linch 4 Jun 2026 5:29 UTC
−3 points
0
in reply to: Shankar Sivarajan’s comment on: Linch’s Shortform
Do you also actively go out of your way to recommend your friends and family cheat in general?

Linch 3 Jun 2026 21:12 UTC
2 points
−3
in reply to: lc’s comment on: Linch’s Shortform
I considered that explanation, but given how low Pangram’s false positive rates are, it seems unlikely that Shankar knows multiple people in that position. Seems much more likely that the default effect of this advice is to get people to repeatedly try their AI-generated writing against Pangram.

Linch 3 Jun 2026 18:06 UTC
5 points
2
in reply to: Oliver Sourbut’s comment on: Linch’s Shortform
You aren’t the first one to bring it up, but I don’t think anybody new (ie, hasn’t read my other writing before, like Kimi, Author of the Menard) has brought it up.
I would’ve thought the connections would be more obvious in Spanish since Borges is a bigger deal in LatAm, but none of the Spanish news coverage or social media has brought it up.
I can’t rule out the Perfil editor noticing but deciding it’s not worth the wordcount to explain, of course. “That Substack writer is really witty in his headlines” isn’t particularly newsworthy. (It’s also possible the joke was lost in translation)

Linch 3 Jun 2026 9:21 UTC
5 points
−2
on: Linch’s Shortform
Weird thought-experiment I came up with:
Suppose you are magically transported to a society + economy “peopled” and run by present-day LLM-shaped intelligences. Suppose further that for whatever reason you can’t do manual labor.
What remote-only work can you do that’d still be economically valuable?
__

I ask this because I think the “obvious” answers for what LLMs can’t do today often rely on either embodiment or specific human preferences. For the later, either intrinsic preferences for the human touch (eg priests, therapists), or idiosyncratic preferences (eg liking The Linchpin blog style, a preference some humans have, but AIs generally don’t share).
If we stipulate both a primarily AI economy and embodiment irrelevance, the ways in which human cognitive performance is superior to AI is still there (I have several key candidate answers), but I think it’s far more subtle.

Linch 3 Jun 2026 7:27 UTC
11 points
8
in reply to: Shankar Sivarajan’s comment on: Linch’s Shortform
(I didn’t downvote)
My own takeaway from your story getting traction has been to tell people that Pangram seems to now be prominent enough that they should run their writing through it and see if it gets flagged, and if it does, get some LLM to generate a few variants until it doesn’t anymore.
This just seems straight-up antisocial, borderline evil. Why do you hate good things?

Linch 2 Jun 2026 22:13 UTC
5 points
0
in reply to: jbash’s comment on: jbash’s Shortform
Fwiw I tested Opus 4.8 XHigh on The Distaff Texts and it got a higher fraction of the plot than past models, including the obvious-to-LW-readers meta part that historically tripped up other models.

Linch 2 Jun 2026 20:33 UTC
60 points
28
on: Linch’s Shortform
One reflection I’ve had in the whole “AI use in the encyclical” affair is to slightly increase my trust in traditional media, especially non-American traditional media, and slightly decrease my trust in social media and new media epistemics.
I tried my best to promote my analysis as legibly and reasonably as I could and focused on logos rather than ethos: I didn’t frame my article with institutional affiliations and intentionally chose not to include obvious, flashy, but irrelevant signaling. Stuff I could’ve done but explicitly chose not to: get an ML professor to cosign my analysis, publish on Arxiv instead of Substack and LessWrong, highlight my past ML experience at Google, etc.
The traditional media response were somewhere between neutral and positive. There were some disappointments (eg a famous media org won’t run the article without confirmation from a “primary source”—aka the Vatican which is of course a non-starter). But mostly the traditional media just looked at the analysis and said “yeah looked reasonable.” and either ran it as “Unconfirmed but seems right” or just “unconfirmed, period.” Which seems fine.
On the other hand, the social media attacks were very aggro. People just didn’t seem to entertain it at all, many without reading it. And I got personally attacked a ton^[1].
Also it wasn’t picked up at all^[2] by new media afaict (unless you count Russia Today as new media).
This is exactly the type of story you might expect new media to be good for: story of institutional decay in the Old Guard, investigation by someone without credentials carefully laying out an epistemically rigorous and systematic case. And yet traditional media was happy to report it^[3], social media kept yelling at me, and new media just got crickets.
1. ^
  Don’t worry, I didn’t take it personally at all. Very much a “yapping of chihuahua in a tiny purse” moment.
2. ^
  This is a slight exaggeration. Other than Russia Today a bunch of small AI news aggregators (AI news in both senses of the term) covered it briefly. Eg Zvi, Let’s Data Science, and some AI slop publications. But no big/general “Alt Media” coverage comparable to The Economist, Perfil, etc).
3. ^
  Quite possibly against their explicit economic interests, if I’m reading the popular reaction correctly.

Linch 2 Jun 2026 14:52 UTC
2 points
2
in reply to: jbash’s comment on: jbash’s Shortform
That’s so different from my use case for anything more substantive than a search query, even “High” feels too low for me most of the time!

Linch 2 Jun 2026 14:46 UTC
5 points
0
in reply to: jbash’s comment on: jbash’s Shortform
What thinking level are you running it in?