Reuben Adams

Karma: 14

Reuben Adams 4 Sep 2025 22:55 UTC
1 point
0
in reply to: faul_sname’s comment on: Reuben Adams’s Shortform
Amazing, thank you! Given Yudkowsky’s frequent re-use of metaphors, it wouldn’t surprise me if there exists a version with the exact wording. But this suffices to demonstrate this thought of his was not prompted by the announcement of GPT-3. Thanks again!

Reuben Adams’s Shortform

Reuben Adams4 Sep 2025 13:00 UTC

2 points

4 comments LW link

Reuben Adams 4 Sep 2025 13:00 UTC
2 points
0
on: Reuben Adams’s Shortform
What is the source of this Yudkowsky quote?

”The purest case of an intelligence explosion would be an Artificial Intelligence rewriting its own source code. The key idea is that if you can improve intelligence even a little, the process accelerates. It’s a tipping point. Like trying to balance a pen on one end—as soon as it tilts even a little, it quickly falls the rest of the way.
It’s quoted in “Artificial Intelligence in Byte-Sized Chunks” as evidence that “Both fears and foreseeing fabulous futures were back in fashion...” after GPT-3. I want to verify the timing, because Yudkowsky could easily have written this years before.
Neither I nor Claude Opus 4.1 can find it.

Reuben Adams 11 Aug 2025 13:13 UTC
8 points
14
in reply to: Buck’s comment on: Re: recent Anthropic safety research
You could have simply asked whether Eliezer was referring to that particular paper, no?
I’m not sure it’s a good norm to often respond in an unkind way to one particular poster. I get that you’ve apologised, but this is a pattern, and one that would drive me away from the platform were I on the receiving end (and if I actually posted).

Reuben Adams 26 Jun 2025 19:32 UTC
3 points
0
on: My pitch for the AI Village
I’ve learned a lot from following the Agent Village, and I found this post quite persuasive. But I’m still uncertain whether it will have a positive impact, partly for the reasons you mentioned, but also for these reasons:
1. While agents are bad, the Agent village may simply convince people that agents are crappy and therefore harmless.
2. Once agents are good, AI companies will be falling over themselves to do demonstrations, making the Agent village obsolete, at least for raising awareness of agent capabilities.
On the first point, many people don’t seem to project capabilities forward. Possibly because they only pay enough attention to see snapshots, and so don’t appreciate the pace of progress. For many (most?) people, seeing a snapshot of Claude getting lost in excessive Google docs, or o3 forgetting it can’t talk, may simply make them believe agents are rubbish or even a dead end. We live in an AI bubble, and I suspect most people will only update if they see AIs successfully doing things, rather than failing to do things.
On the second point, would the Agent Village significantly change its messaging and strategy once AI agents are economically viable? Would they focus more on uncovering misalignment risks than evaluating capabilities?

Reuben Adams 31 Dec 2024 17:12 UTC
1 point
0
on: Why I’m Moving from Mechanistic to Prosaic Interpretability
How about “Cognitive Interpretability”, or “AI Cognitive Psychology” (AI Cog Psych for short) rather than “Prosaic Interpretability”?

“Prosaic” conjures only some of the correct associations, and then only if you’ve heard of “Prosaic Alignment”, which was a pretty bad name imho. If you had told me to guess what you meant by the term PI, I would not have guessed what you have described.

I think MI, and what you call PI, are analogous to Cognitive Neuroscience and Cognitive Psychology, respectively, which is why I think AI Cog Psych will lead to more correct inferences on first hearing.

I also suspect that Cognitive Psychology, especially the linguistics part, already has a wealth of methods that could transfer very nicely onto LLMs. For example, in The Language Instinct, Steven Pinker describes how it is possible to discover many things about how we parse sentences without any brain scans—solely through natural language experiments. He mentioned a bunch of other experiments I think could work quite well, or could at least help build intuitions for how to discover mental mechanisms solely through input output behaviour on carefully constructed sentences.

It also sounds cooler to say you work on AI Cognitive Psychology rather than Prosaic Interpretability ;)

By the way, the analogy with genes is fantastic. I think it nicely points to the fact that even if some features are relatively straightforward to find, circuits may nevertheless be fiendishly difficult to uncover. Thanks for writing such an excellent post and being honest about some of your hard work that didn’t pan out how you hoped!

Reuben Adams 30 Aug 2023 16:03 UTC
5 points
1
on: Dating Roundup #1: This is Why You’re Single
Minor maths error:
I think you’re overestimating how significant those numbers say political compatibility is. The study said that 4% of marriages are between a Republican and a Democrat, which *sounds* low, but given that something like 30% of people are Republicans and 30% Democrats and 40% Independents, you would only expect 9% from pure mixing. There are 17% between Independents and non-Independents, but from random mixing you would only expect 24%.
Given those numbers of Republicans, Democrats and Independents, I’m pretty sure from pure mixing you would expect 18% (not 9%) of marriages to be between a Republican and a Democrat and 48% (not 24%) between an Independent and non-Independent.

Explanation: simplifying to heterosexual relationships and assuming both Republicans and Democrats are 50-50 male-female, you would expect 9% of marriages to be between a male Republican and a female Democrat, and 9% to be between a male Democrat and a female Republican, making 18% Republican-Democrat marriages in total.

Reuben Adams 28 Apr 2022 8:47 UTC
1 point
0
on: Basic Inframeasure Theory
Very much enjoying reading these! In Definition 3 (an a-measure), it might be worth making it more conspicuous that the measure should not have a negative part as I somehow only spotted that on the third or fourth reading.