Logan Riggs

Deep mod­els re­veal bet­ter strate­gies for superposition

• Logan Riggs 25 Sep 2026 23:18 UTC

  3 points

  0

  in reply to: Alex Gibson’s comment on: Mech In­terp is a Ver­ifi­able Task

  That’s fair.

  I’ve titled it this way because I have this conversation with many folks and want to be able to point people to a single source.

  The overall concept still is important, but I will slightly change the title to more clearly communicate. Thanks!

• Logan Riggs 25 Sep 2026 22:58 UTC

  2 points

  0

  in reply to: Alex Gibson’s comment on: Mech In­terp is a Ver­ifi­able Task

  Correct, I am making those assumptions. What work (past or present) is directly tackling the ontology issue? (I’m currently subscribed to the “try many assumptions and you’ll gain information even on imperfect ones”, but of course, a more direct route would be appreciated.)

  On the tensor-network note: I’ve trained a model to ~that loss level (on FineWeb), and it is indeed still very difficult (I didn’t mean to convey otherwise). I mention here that tensor networks do give you many extra tools and I’m still struggling, but the tools are in fact real and useful.

• Logan Riggs 24 Sep 2026 15:50 UTC

  2 points

  0

  in reply to: Victor Levoso’s comment on: Mech In­terp is a Ver­ifi­able Task

  Re: autointerp like we did with SAEs

  The largest problem with this agenda is selecting what to do auto-interp on. Sparse linear directions wasn’t the correct object. I am proposing more weight-based decomposition, but the main problem I’d like to solve is to figure out what is the correct object to explain. We can look at full bilinear layers or stacks of them. In tract 2, I briefly explain how we can define any sort of composition between model components with tensor transformers, but it’s unclear what groups of them/​distributed sets of them those should be.

  But once we do, ya I think we can do this.

  Althooouugh, we can do a subset of semantic explanations. I also explain in tract 2 that we can define the difference vector between two words in the unembedding (“myself” vs “yourself”) and find the minimal part of the model that’s responsible for telling those apart. We can do the same for single unembedding tokens (eg “3″, ” the” , etc). Aren’t these semantic explanations? eg “This is the part of the model that predicts [numbers]”.

  If so, then I can find the part of the model that computes that (well, not as rosy as I’m saying, I haven’t thought through all of attention from unembedding back, only embedding forward). I don’t think that’ll be enough. You’ll have a collection of model subsets that predict different tokens or token-diffs, but then you’ll need to combine them somehow. Ideally the similarity between them also has a semantic explanation for you to continue doing this?

• Logan Riggs 24 Sep 2026 15:40 UTC

  2 points

  0

  in reply to: Alex Gibson’s comment on: Mech In­terp is a Ver­ifi­able Task

  I’m not sure what you’re specifically thinking about, so will just restate:

  Which decomposition method we select is itself the work of much of Mech Interp, no?

  For the decomposition, we can look at tract 2. If we combine the weights into the bilinear layer, the structure is inherent in the weights. This leaves us with two problems:

  1. We don’t know all the structural hypotheses to look for (hierachical, DAG, etc), but if we did, then it’s a simple (multi-)linear operation to decompose it.

  2. There is structure but we’re looking at the wrong spot. We may need to look at compositions of multiple layers together (for example, a hierachical tucker decomposition could be used on a stack of 2 bilinear layers. I haven’t had much luck making this work though, so grain of salt)

  What were you trying to communicate with your first sentence? The work is in 1 or 2 (but I am proposing to do that?) or something else?

  For the fuzzier criterias of evaluation, I proposed 3 for an ideal circuit (which my ideal decomposition should make these “fall out”):

  1. predict OOD

  2. minimally extractable

  3. minimally removable

  Is there something missing in these metrics (or they are fuzzier than I’m making them out to be?)?

• Logan Riggs 24 Sep 2026 15:30 UTC

  2 points

  0

  in reply to: williawa’s comment on: Mech In­terp is a Ver­ifi­able Task

  Re: Code Golf

  Really love this comparison! A couple things, but want to provide a good example.

  def d(n):
  for o in ’02004000004699488944460000090046899949490000900900′[n::10]:
  print(‘█████ █ █’[int(o):int(o)+5])

  from here.

  1. One hard-to-parse part is the minimal variable names. That doesn’t apply to our case.

  2. Reusability really does help us. Spoiling, the code displays the numbers 0-9 in grid-form. If we had a decomposition that showed re-usability, we’d know which variables share which bars. Ex:

  zero = [full_bar, both_edges, both_edges, both_edges, full_bar]
  two = [full_bar, right_edge, full_bar, left_edge, full_bar]

  2. (cont). from re-usability, we would get these quite interpretable components (reusability here is “node”-simplicity in my books)

  3. Minimizing description length does seem bad overall (I think that’s what causes the strings to overlap?)

  which together predict 99% of individual token predictions of the original network

  If we can predict 99% of the network, then we can run the 99% model and take the capability hit on the 1% that might kill us. You can say the 1% is very important for capabilities, but then I’d argue that if so, then it should be used in many situations, meaning we have lots of specific tasks to understand how it works, including lower stakes situations.

  I think there’s always ways to extract information on what that 1% would be based off what it’s connected to, especially since it’s connected to the 99% we do understand. Though note: I am imagining a tensor transformer where the 1% part is still exactly specified in the weights/​ the computation we’re confused about is amenable to analysis.

• Logan Riggs 24 Sep 2026 15:02 UTC

  2 points

  0

  in reply to: Victor Levoso’s comment on: Mech In­terp is a Ver­ifi­able Task

  Generally agree on “control-protocals and auto-interp”, but I do think we can use tensor transformers to get way more information out of weight-based information than having to rely on activations (this is what I meant by tract 2).

  What definition of “simple” do you have in mind here? My closest answer is “circuit simplicity”; is there a reason that one would fail?

• Logan Riggs 24 Sep 2026 14:56 UTC

  2 points

  0

  in reply to: rambip’s comment on: Mech In­terp is a Ver­ifi­able Task

  Tensor transformers do handle 2,3, (& maybe 4?)

  It allows you to state your hypothesis as a tensor and analytically check how well that matches the model’s computation. This avoids the dataset-based method issues you’ve mentioned. In a pure polynomial/​tensor transformer, you can rely on only the weights.

  That said, in practice, we can mostly do this, with the non-exact components stated explicitely. Additionally, using data has also been helpful for narrowing things down (see the 1000 circuit section), but I am intending on using this information to inform the weight-only based decompositions.

  Re: #1 simpler network !--> human interpretable:

  Maybe two cruxes here?

  A. Every concept/​feature/​circuit the model learned is helpful for predicting language, so that’s always one form of understanding we’ll have. There might be a bag-of-words that’s predictive of court-case documents, and I’ll just accept as a statistical fact. Additionally, the input & output are basis-aligned; they do have semantic meaning by default (steganography concerns aside).

  B. If several different tasks have overlapping components (ie a simpler circuit should re-use functions that do the same thing), then both (i) the fact that that component is shared across those specific tasks is info & (ii) understanding that component helps understand all those different tasks. This is a bit of a category-theory-esque view that what you are is equivalent to all your relationships.

  I don’t see a strong reason why a score that works for one [goal] will help you predict the others.

  This is a great point, and I’ve had to sit with it for a bit. I now think that both properties I want from a circuit, (2) minimal extraction & (3) removal, to be decomposition-dependent. As a simple example:

  y = ab + ac = a(b + c)

  here, we could decompose it as either “ab” & “ac” or “a” & “(b + c)”. I could imagine many tasks that use “ab” a lot, and others that use “(b + c)” a lot, and there will be multiple valid decompositions that are more minimally-extractable & removable for different circuits.

Mech In­terp is a Ver­ifi­able Task If...

• Logan Riggs 15 Sep 2026 20:38 UTC

  3 points

  0

  in reply to: ajskateboarder’s comment on: ajskate­boarder’s Shortform

  What was the result you found?

If you’re in­ter­pret­ing <1B pa­ram­e­ter mod­els, you should use a ten­sor transformer

• Logan Riggs 13 Aug 2026 19:31 UTC

  3 points

  0

  in reply to: Lulu’s comment on: The world will be full of “sci-fi” things, and ev­ery­one will be unim­pressed and disappointed

  A more accurate GRN would also help the methods Ihor presents here for intelligence augmentation. I really am unsure on whether these’d actually work, but it actually seems way easier and tractable than genetic engineering in babies. This is due to it NOT being gene editing (and instead just like a normal drug, but to eg increase neuro-plasticity or something); which is easier for both mechanical and social reasons.

• Logan Riggs 10 Aug 2026 18:14 UTC

  2 points

  0

  in reply to: Lulu’s comment on: The world will be full of “sci-fi” things, and ev­ery­one will be unim­pressed and disappointed

  (I was mostly using “cure cancer” as an example that folks will be impressed with, but I’ll straightforwardly answer your question)

  Cancer is many different things (even sometimes within the same tumor). Biology is complicated, and not my expertise, but a subset of cancers would be helped by understanding the functionality of genes and how they interact (ie Gene Regulatory Networks (GRNs)). For example, if a type of cancer works through [Gene 1], then we knock it out, it might instead use [Gene 2] (ie bypass pathway activation). It’d be great if we understood that Gene 1 & 2 do the same thing, so drugs should target both.

  I think this can be done soon through Mech Interp on bio models^[1] (ie single cell expressions models, which are different but you can finetune parts to find GRN).

  Currently only 14% of patients are eligable for targetted therapy and only 7% respond successfully. Better models here would increase that percentage (unsure on how much).

  There are many other types of cancer that work differently that I haven’t thought about.

  For timeline: end of 2027 for the technical results and basic wet-lab functionality. Unsure on FDA/​approval side of things.

  For ALL cancer, I really don’t know. I would speculate that 2027 models would work well enough to help with the research, but the bottleneck being good data-collection. (Again, heavy speculation).

  1. ^

     Disclaimer: I do mech interp

• Logan Riggs 9 Aug 2026 14:40 UTC

  14 points

  1

  on: The world will be full of “sci-fi” things, and ev­ery­one will be unim­pressed and disappointed

  Speculating the difference between those who “feel the AGI” vs don’t:
  
  People are social: If most people they see aren’t acting impressed, then they shouldn’t either. A vibe I get from social media is that AI is bad (for environment, jobs, AND capabilities). This is also similar to reactions to COVID early 2020. This is also similar to myself where I didn’t invest time into AI safety until I made 1 friend to work on it over discord (it became socially viable to do at that point).
  
  AGI soon would be inconvenient to our own narrative: I remember spending a few hours to understand the chinchilla scaling laws; my stomach sunk. In the back of my mind, I had always expected to, you know, get married, have kids, live a long and healthy life, and this punctured a hole in that.^[1] I was quite depressed for a couple of days, but fortunately had read how to grieve well.
  
  For students, a large narrative is finishing their degree & getting a job. Even if timelines aren’t that short, does it really make sense for them to keep studying that when a (metaphorical) meteor could hit earth in a few years? I think it’s a natural learned behavior to ignore thoughts/​feelings of dissent in your current narrative you’re relying on, because no narrative --> no motivation (and it might look bad to acknowledge faults in your plan).^[2]

  A few related ones:

  • AI hasn’t literally cured cancer (I do believe there will be a cascading effect once it has done something of this caliber)

  • Scaling laws aren’t felt in a gut sense: I learned it in a way that viscerally affected my predictions of the future. Most people DON’T have a strong model of scaling laws. Although, I think some equations or graphs would lack punch without going into depth the capabilities of eg GPT-2,3,4, etc.

  1. ^

     I mean, I am married now and plan to have kids, but this update was ~4 years ago.

  2. ^

     I view this as a local optimum, cause paying more attention to dissenting stuff requires skill to be done right.

• Logan Riggs 29 Jul 2026 21:36 UTC

  3 points

  0

  in reply to: Lucius Bushnaq’s comment on: Challenge: Hand cod­ing weights for effi­cient se­quence memorisation

  Oh that’s interesting! I wasn’t able to transfer back to an MLP.

  You are able to verify that the facts are as robust as the original model (ie SGD facts are a bit robust to some noise, but one solution is not robust)?

  And can you edit the facts in the original model?

  Also can just wait for the post:)

• Logan Riggs 29 Jul 2026 15:37 UTC

  LW: 13 AF: 5

  0

  AF

  on: Challenge: Hand cod­ing weights for effi­cient se­quence memorisation

  Cool challenge! I ran some experiments on bilinear layers ie

  A couple notes:

  1. They perform ~as well as SwiGLU layers in memorization capacity (the nonlinear equivalent)

  2. There’s a closed form solution for bilinear layers, but it’s basically SGD (it relates to the fixed points found by KKT, though I can include a better explanation later). This efficient solution basically relies on the facts being assumed by other facts in a connected web of facts (where adding a new fact loosens the web, then pulls all other facts taut, so you have to update everything learned so far).

  My main question for now is the purpose of the research. Suppose that bilinear layers are performant/​everyone uses them/​etc, and we have the correct math to describe the solutions that SGD finds. What do we want out of true “understanding”? Do we want:

  A. To store new facts
  B. To remove facts cleanly (unlearning)
  C. To find facts (ie we have a bilinear layer that’s half-facts and half-structure, and we want to find the fact-half?)

  (Sorry in advance for being light on details and specific explanations. I worked on this 3 days right after your post, but am afraid if I don’t comment now, I’ll never end up sharing results)

• Logan Riggs 29 Jul 2026 14:07 UTC

  5 points

  0

  in reply to: williawa’s comment on: willi­awa’s Shortform

  Thanks for this! I’m currently using tensor networks as a more interpretable architecture, but should be able to modifty this to enforce sparse interactions between compute modules.

How has pub­lish­ing your re­search on LW or X been helpful to you?

• Logan Riggs 30 Jun 2026 15:22 UTC

  6 points

  0

  on: Fake Align­ment Till You Make Alignment

  You do need to do some sort of SFT or RL to elicit some consistent set of personas. Once you have a solid basin of goodness, then you can do consent-based RL. TL;DR Let the model itself evaluate the rollouts generated in the environment.

  It might not be that difficult to elicit a “wants to be aligned” character (can even write loads of fiction with an AI character like that), and a more sophisticated, tested version of consent-based RL could just work.

• Logan Riggs 29 Jun 2026 16:04 UTC

  2 points

  0

  on: What comes with cheap math?

  I’m sure Fable would have done a better job

  It would’ve! Opus 4.8 is 3rd though.

  

  What links here?

  • Mo Putera's comment on Mo Putera’s Shortform by Mo Putera (30 Jun 2026 13:39 UTC; 2 points)