Transformer Circuits

Tag

Finding Neurons in a Haystack: Case Studies with Sparse Probing

wesg and Neel Nanda

3 May 2023 13:30 UTC

33 points

6 comments2 min readLW link 1 review

(arxiv.org)

Paper Walkthrough: Automated Circuit Discovery with Arthur Conmy

Neel Nanda29 Aug 2023 22:07 UTC

36 points

1 comment1 min readLW link

(www.youtube.com)

How to Think About Activation Patching

Neel Nanda4 Jun 2023 14:17 UTC

50 points

5 comments20 min readLW link

(www.neelnanda.io)

Sleep peacefully: no hidden reasoning detected in LLMs. Well, at least in small ones.

Ilia Shirokov and Ilya Nachevsky

4 Apr 2025 20:49 UTC

17 points

4 comments7 min readLW link

200 COP in MI: Interpreting Algorithmic Problems

Neel Nanda31 Dec 2022 19:55 UTC

33 points

2 comments10 min readLW link

200 COP in MI: Techniques, Tooling and Automation

Neel Nanda6 Jan 2023 15:08 UTC

13 points

0 comments15 min readLW link

A Walkthrough of Interpretability in the Wild (w/ authors Kevin Wang, Arthur Conmy & Alexandre Variengien)

Neel Nanda7 Nov 2022 22:39 UTC

30 points

15 comments3 min readLW link

(youtu.be)

200 COP in MI: Analysing Training Dynamics

Neel Nanda4 Jan 2023 16:08 UTC

16 points

0 comments14 min readLW link

Explaining the Transformer Circuits Framework by Example

Felix Hofstätter25 Apr 2023 13:45 UTC

9 points

1 comment15 min readLW link

Understanding the tensor product formulation in Transformer Circuits

Tom Lieberum24 Dec 2021 18:05 UTC

16 points

2 comments3 min readLW link

A Walkthrough of In-Context Learning and Induction Heads (w/ Charles Frye) Part 1 of 2

Neel Nanda22 Nov 2022 17:12 UTC

20 points

0 comments1 min readLW link

(www.youtube.com)

An Extremely Opinionated Annotated List of My Favourite Mechanistic Interpretability Papers v2

Neel Nanda7 Jul 2024 17:39 UTC

146 points

18 comments25 min readLW link 1 review

200 COP in MI: Exploring Polysemanticity and Superposition

Neel Nanda3 Jan 2023 1:52 UTC

34 points

6 comments16 min readLW link

Does Circuit Analysis Interpretability Scale? Evidence from Multiple Choice Capabilities in Chinchilla

Neel Nanda, Tom Lieberum, Matthew Rahtz, János Kramár, Geoffrey Irving, Rohin Shah and Vlad Mikulik

20 Jul 2023 10:50 UTC

44 points

3 comments2 min readLW link

(arxiv.org)

Finding Sparse Linear Connections between Features in LLMs

Logan Riggs, Sam Mitchell and Adam Kaufman

9 Dec 2023 2:27 UTC

70 points

5 comments10 min readLW link

200 COP in MI: Looking for Circuits in the Wild

Neel Nanda29 Dec 2022 20:59 UTC

16 points

5 comments13 min readLW link

200 Concrete Open Problems in Mechanistic Interpretability: Introduction

Neel Nanda28 Dec 2022 21:06 UTC

108 points

0 comments10 min readLW link

Interpreting OpenAI’s Whisper

EllenaR24 Sep 2023 17:53 UTC

116 points

13 comments7 min readLW link

Graphical tensor notation for interpretability

Jordan Taylor4 Oct 2023 8:04 UTC

146 points

11 comments19 min readLW link

Arrakis—A toolkit to conduct, track and visualize mechanistic interpretability experiments.

Yash Srivastava17 Jul 2024 2:02 UTC

3 points

2 comments5 min readLW link

An Interpretability Illusion for Activation Patching of Arbitrary Subspaces

Georg Lange, Alex Makelov and Neel Nanda

29 Aug 2023 1:04 UTC

77 points

4 comments1 min readLW link

Sparse Autoencoders Work on Attention Layer Outputs

Connor Kissane, robertzk, Arthur Conmy and Neel Nanda

16 Jan 2024 0:26 UTC

85 points

9 comments18 min readLW link

Logit Prisms: Decomposing Transformer Outputs for Mechanistic Interpretability

ntt12317 Jun 2024 11:46 UTC

5 points

4 comments6 min readLW link

(neuralblog.github.io)

Automatically finding feature vectors in the OV circuits of Transformers without using probing

Jacob Dunefsky12 Sep 2023 17:38 UTC

16 points

2 comments29 min readLW link

When Does the Local Learning Coefficient Track Circuit Formation?

[deleted]8 May 2026 17:50 UTC

1 point

0 comments2 min readLW link

Finding Backward Chaining Circuits in Transformers Trained on Tree Search

abhayesian, Jannik Brinkmann and Victor Levoso

28 May 2024 5:29 UTC

53 points

1 comment9 min readLW link

(arxiv.org)

“What the hell is a representation, anyway?” | Clarifying AI interpretability with tools from philosophy of cognitive science | Part 1: Vehicles vs. contents

IwanWilliams9 Jun 2024 14:19 UTC

9 points

1 comment4 min readLW link

No Really, Attention is ALL You Need—Attention can do feedforward networks

Robert_AIZI31 Jan 2023 18:48 UTC

29 points

7 comments6 min readLW link

(aizi.substack.com)

AISC project: TinyEvals

Jett Janiak22 Nov 2023 20:47 UTC

26 points

0 comments4 min readLW link

SAEs (usually) Transfer Between Base and Chat Models

Connor Kissane, robertzk, Arthur Conmy and Neel Nanda

18 Jul 2024 10:29 UTC

67 points

0 comments10 min readLW link

Polysemantic Attention Head in a 4-Layer Transformer

Jett Janiak, cmathw and StefanHex

9 Nov 2023 16:16 UTC

51 points

0 comments6 min readLW link

Open Source Replication of Anthropic’s Crosscoder paper for model-diffing

Connor Kissane, robertzk, Arthur Conmy and Neel Nanda

27 Oct 2024 18:46 UTC

48 points

4 comments5 min readLW link

Anthropic’s SoLU (Softmax Linear Unit)

Joel Burget4 Jul 2022 18:38 UTC

21 points

1 comment4 min readLW link

(transformer-circuits.pub)

Addendum: More Efficient FFNs via Attention

Robert_AIZI6 Feb 2023 18:55 UTC

10 points

2 comments5 min readLW link

(aizi.substack.com)

What is the functional role of SAE errors?

Taras Kutsyk, Tim Hua, woog and Andre Assis

20 Jun 2025 18:11 UTC

12 points

6 comments38 min readLW link

Cross-Layer Transcoders are incentivized to learn Unfaithful Circuits

Georg Lange, RGRGRG, Kat Dearstyne and Kamal Maher

2 Feb 2026 21:32 UTC

47 points

6 comments18 min readLW link

Steganography via internal activations is already possible in small language models — a potential first step toward persistent hidden reasoning.

Ilia Shirokov and Ilya Nachevsky

9 Aug 2025 11:44 UTC

7 points

7 comments12 min readLW link

An Analogy for Understanding Transformers

CallumMcDougall13 May 2023 12:20 UTC

92 points

6 comments9 min readLW link

Concrete Methods for Heuristic Estimation on Neural Networks

Oliver Daniels14 Nov 2024 5:07 UTC

35 points

0 comments27 min readLW link

An adversarial example for Direct Logit Attribution: memory management in gelu-4l

Can, Yeu-Tong Lau, James Dao and Jett Janiak

30 Aug 2023 17:36 UTC

17 points

0 comments8 min readLW link

(arxiv.org)

Scaling Sparse Feature Circuit Finding to Gemma 9B

Diego Caples, Jatin Nainani, CallumMcDougall and rrenaud

10 Jan 2025 11:08 UTC

88 points

11 comments17 min readLW link

Are SAE features from the Base Model still meaningful to LLaVA?

Shan23Chen5 Dec 2024 19:24 UTC

5 points

2 comments10 min readLW link

Can quantised autoencoders find and interpret circuits in language models?

charlieoneill24 Mar 2024 20:05 UTC

30 points

4 comments24 min readLW link

Trans­former Circuits

Transformer Circuits