Logan Riggs

If you’re in­ter­pret­ing <1B pa­ram­e­ter mod­els, you should use a ten­sor transformer

• Logan Riggs 13 Aug 2026 19:31 UTC

  3 points

  0

  in reply to: Lulu’s comment on: The world will be full of “sci-fi” things, and ev­ery­one will be unim­pressed and disappointed

  A more accurate GRN would also help the methods Ihor presents here for intelligence augmentation. I really am unsure on whether these’d actually work, but it actually seems way easier and tractable than genetic engineering in babies. This is due to it NOT being gene editing (and instead just like a normal drug, but to eg increase neuro-plasticity or something); which is easier for both mechanical and social reasons.

• Logan Riggs 10 Aug 2026 18:14 UTC

  2 points

  0

  in reply to: Lulu’s comment on: The world will be full of “sci-fi” things, and ev­ery­one will be unim­pressed and disappointed

  (I was mostly using “cure cancer” as an example that folks will be impressed with, but I’ll straightforwardly answer your question)

  Cancer is many different things (even sometimes within the same tumor). Biology is complicated, and not my expertise, but a subset of cancers would be helped by understanding the functionality of genes and how they interact (ie Gene Regulatory Networks (GRNs)). For example, if a type of cancer works through [Gene 1], then we knock it out, it might instead use [Gene 2] (ie bypass pathway activation). It’d be great if we understood that Gene 1 & 2 do the same thing, so drugs should target both.

  I think this can be done soon through Mech Interp on bio models^[1] (ie single cell expressions models, which are different but you can finetune parts to find GRN).

  Currently only 14% of patients are eligable for targetted therapy and only 7% respond successfully. Better models here would increase that percentage (unsure on how much).

  There are many other types of cancer that work differently that I haven’t thought about.

  For timeline: end of 2027 for the technical results and basic wet-lab functionality. Unsure on FDA/​approval side of things.

  For ALL cancer, I really don’t know. I would speculate that 2027 models would work well enough to help with the research, but the bottleneck being good data-collection. (Again, heavy speculation).

  1. ^

     Disclaimer: I do mech interp

• Logan Riggs 9 Aug 2026 14:40 UTC

  14 points

  1

  on: The world will be full of “sci-fi” things, and ev­ery­one will be unim­pressed and disappointed

  Speculating the difference between those who “feel the AGI” vs don’t:
  
  People are social: If most people they see aren’t acting impressed, then they shouldn’t either. A vibe I get from social media is that AI is bad (for environment, jobs, AND capabilities). This is also similar to reactions to COVID early 2020. This is also similar to myself where I didn’t invest time into AI safety until I made 1 friend to work on it over discord (it became socially viable to do at that point).
  
  AGI soon would be inconvenient to our own narrative: I remember spending a few hours to understand the chinchilla scaling laws; my stomach sunk. In the back of my mind, I had always expected to, you know, get married, have kids, live a long and healthy life, and this punctured a hole in that.^[1] I was quite depressed for a couple of days, but fortunately had read how to grieve well.
  
  For students, a large narrative is finishing their degree & getting a job. Even if timelines aren’t that short, does it really make sense for them to keep studying that when a (metaphorical) meteor could hit earth in a few years? I think it’s a natural learned behavior to ignore thoughts/​feelings of dissent in your current narrative you’re relying on, because no narrative --> no motivation (and it might look bad to acknowledge faults in your plan).^[2]

  A few related ones:

  • AI hasn’t literally cured cancer (I do believe there will be a cascading effect once it has done something of this caliber)

  • Scaling laws aren’t felt in a gut sense: I learned it in a way that viscerally affected my predictions of the future. Most people DON’T have a strong model of scaling laws. Although, I think some equations or graphs would lack punch without going into depth the capabilities of eg GPT-2,3,4, etc.

  1. ^

     I mean, I am married now and plan to have kids, but this update was ~4 years ago.

  2. ^

     I view this as a local optimum, cause paying more attention to dissenting stuff requires skill to be done right.

• Logan Riggs 29 Jul 2026 21:36 UTC

  3 points

  0

  in reply to: Lucius Bushnaq’s comment on: Challenge: Hand cod­ing weights for effi­cient se­quence memorisation

  Oh that’s interesting! I wasn’t able to transfer back to an MLP.

  You are able to verify that the facts are as robust as the original model (ie SGD facts are a bit robust to some noise, but one solution is not robust)?

  And can you edit the facts in the original model?

  Also can just wait for the post:)

• Logan Riggs 29 Jul 2026 15:37 UTC

  LW: 13 AF: 5

  0

  AF

  on: Challenge: Hand cod­ing weights for effi­cient se­quence memorisation

  Cool challenge! I ran some experiments on bilinear layers ie

  A couple notes:

  1. They perform ~as well as SwiGLU layers in memorization capacity (the nonlinear equivalent)

  2. There’s a closed form solution for bilinear layers, but it’s basically SGD (it relates to the fixed points found by KKT, though I can include a better explanation later). This efficient solution basically relies on the facts being assumed by other facts in a connected web of facts (where adding a new fact loosens the web, then pulls all other facts taut, so you have to update everything learned so far).

  My main question for now is the purpose of the research. Suppose that bilinear layers are performant/​everyone uses them/​etc, and we have the correct math to describe the solutions that SGD finds. What do we want out of true “understanding”? Do we want:

  A. To store new facts
  B. To remove facts cleanly (unlearning)
  C. To find facts (ie we have a bilinear layer that’s half-facts and half-structure, and we want to find the fact-half?)

  (Sorry in advance for being light on details and specific explanations. I worked on this 3 days right after your post, but am afraid if I don’t comment now, I’ll never end up sharing results)

• Logan Riggs 29 Jul 2026 14:07 UTC

  5 points

  0

  in reply to: williawa’s comment on: willi­awa’s Shortform

  Thanks for this! I’m currently using tensor networks as a more interpretable architecture, but should be able to modifty this to enforce sparse interactions between compute modules.

How has pub­lish­ing your re­search on LW or X been helpful to you?

• Logan Riggs 30 Jun 2026 15:22 UTC

  6 points

  0

  on: Fake Align­ment Till You Make Alignment

  You do need to do some sort of SFT or RL to elicit some consistent set of personas. Once you have a solid basin of goodness, then you can do consent-based RL. TL;DR Let the model itself evaluate the rollouts generated in the environment.

  It might not be that difficult to elicit a “wants to be aligned” character (can even write loads of fiction with an AI character like that), and a more sophisticated, tested version of consent-based RL could just work.

• Logan Riggs 29 Jun 2026 16:04 UTC

  2 points

  0

  on: What comes with cheap math?

  I’m sure Fable would have done a better job

  It would’ve! Opus 4.8 is 3rd though.

  

  What links here?

  • Mo Putera's comment on Mo Putera’s Shortform by Mo Putera (30 Jun 2026 13:39 UTC; 2 points)

• Logan Riggs 29 Jun 2026 15:35 UTC

  2 points

  0

  on: Power Laws in NNs: A Pos­si­ble Mechanism for In­duc­tive Bias to­wards Sparse Representations

  Surprisingly entertaining for a technical write-up; thanks!

  You might be able to use the Dark Matter of SAEs paper to explore your thesis. In short:

  • α → 0 regime: SAEs have a residual. You can consistently learn a chunk of that residual through linear regression. This all sounds very α → 0/​ low rank structure to me.

  • α → 2 regime: However, there’s always this unlearned “dark matter” that could be your noise.

  Caveats:

  1. You can predict more and more of this “dark matter” by increasing # of features/​datapoint

  2. You can train an SAE on a randomly initialized LLM; see figure 2 in section 3 of the paper.

  

  

  Randomized excl emb: take the trained model, randomize the weights except the embedding.
  Control: Random inputs on the trained model
  Step 0: The actual init used in Pythia (not gaussian)

  There’s no clear pattern except the step 0 init being the easiest for the SAE to reconstruct over layers (higher is better). Unsure why the control would get easier to reconstruct at later layers? Why would running gaussian data through more gaussian layers be easier for an SAE to reconstruct??

  The hypothesis here would be:

  Gaussian data is harder to sparsely reconstruct

  Which isn’t true in general (comparing trained vs control, sometimes control is easier to reconstruct). Another hypothesis:

  Randomly initialized models are harder to sparsely reconstruct then trained
  ie orange and blue (random) should be below gray (trained)

  Which is also not generally true. Pretty weird!

  
  [On a separate note, LLMs do perform better when initialized at α < 2 (AFAIK, nowadays people just rely on modern normalization and muon) which is better than a guassian init, so maybe a hint here?]
  

• Logan Riggs 22 Jun 2026 18:28 UTC

  4 points

  2

  on: The Resi­d­ual Stream Has a Geom­e­try of Time

  This is quite a neat research direction and I think the title undersells it? I thought it was “circle-days of the week” (which is so last year, pun unintended) geometry as opposed to “Some features change across longer sequence positions; most don’t. We use simple probes to show this.”, which is much cooler.

• Logan Riggs 22 Jun 2026 18:27 UTC

  10 points

  0

  on: A Mechanis­tic Ex­pla­na­tion of Prompt In­jec­tion (and why you should study roles)

  Cool project and well written.

  I’m really struggling to figure out how to train to make role-boundaries more robust. They probably already make the “<assistant>” as special tokens (so literally typing “<assistant>” isn’t interpreted as the role-token).

  You could train probes to detect each role, and mix around the text of each (eg put what the user said into the thinking tag, etc), and use that to detect if any prompt injections are happening (ie if text within “thinking” is classified as “user”, then you have a problem). However, one problem here is ensuring the probes are causal to the LLM’s output.

• Logan Riggs 22 Jun 2026 16:16 UTC

  2 points

  0

  on: You Can Catch Sleeper Agents by Teach­ing Another Model to Imi­tate Them

  This is quite an interesting way to do OOD-detection. Instead of measuring statistics on the original model’s activations, you run it on two models and see if there’s a diff in reconstruction.

  Would be interesting to compare against those baselines.

  I don’t even think this would naively be twice as expensive, since you can wait until after Original model finishes generating before running the Finetuned model.

• Logan Riggs 22 Jun 2026 15:59 UTC

  2 points

  0

  in reply to: cat-state’s comment on: Ten­sor-Trans­former Var­i­ants are Sur­pris­ingly Performant

  Ah, that’s a really cool connection; thanks!

  This definitely works for maintaining contractions in a tensor network, but for our most recent work, we rely on the standard semiring to be able to use linear algebra tools (eg cosine similarity). Switching to others would allow more standard non-linearities, but at the cost of interpretability (probably. I only really know linear algebra, lol).

• Logan Riggs 22 Jun 2026 15:45 UTC

  2 points

  0

  on: VFUSE: Viru­lent Fea­ture Un­der­stand­ing With Sparse AutoEncoders

  The results seem a bit mixed regarding “probes trained on SAE features perform better than probes”. I see 3 times SAEs are significantly better and 2 times the raw probe is better.

• Logan Riggs 29 May 2026 21:41 UTC

  2 points

  0

  in reply to: CarolusRenniusVitellius’s comment on: When Are Two Net­works the Same? Ten­sor Similar­ity for Mechanis­tic Interpretability

  For finding a decomp that has meaningful semantics, I’ve really struggled with this. There are principled decomps like CP & Tucker, and I think the LL1 is more what we want (It’s like Tucker, but you only mix in the two input components. I think this makes sense because if multiple components write to the same direction, then they should be grouped together. In other words, the rank complexity doesn’t matter for the input).

  Thomas also wrote a paper finding a canonical representation of a multi-layer tensor network. This allows you to truncate more singular vectors. From a graph/​circuit perspective, this lets you have less nodes. However, between layers, they are densely connected (ie dense edges), making it hard to interpret later features in terms of earlier ones.

  You can fix this somewhat by finding a rotation the minimizes both edges and nodes; however, because it’s fundamentally SVD under-the-hood, the features are constrained to be orthogonal. This constraint interferes with our ability to have a small number of edges & nodes, so can maybe focus on that for the moment.

  Warning: that compositionality paper is quite hard to understand!

• Logan Riggs 29 May 2026 21:30 UTC

  2 points

  0

  in reply to: CarolusRenniusVitellius’s comment on: When Are Two Net­works the Same? Ten­sor Similar­ity for Mechanis­tic Interpretability

  Showing behavioural diffs on random strings is actually a feature (not a bug) for me.

  That said, the paper does generalize tensor sim with metric M (eq 3), which could be a covariance matrix on the dataset you’re interested in. Was this what you wanted?

When Are Two Net­works the Same? Ten­sor Similar­ity for Mechanis­tic Interpretability

Black Boxes for Low-Stakes, In­ter­pretable AI for High-Stakes