chanind

• chanind 11 Mar 2026 1:35 UTC

  2 points

  0

  in reply to: lilkim2025’s comment on: Let­ting Claude do Au­tonomous Re­search to Im­prove SAEs

  Could you talk a little more about the logistics of this project, how you set it up, and how much the tokens ended up costing?

  There’s really nothing to the setup, it’s just the TASK.md file, and literally prompting Claude “follow the instructions in TASK.md”. I used the official Ralph Wiggum Plugin for Claude Code to do the looping. I have a Claude max subscription so I’m not sure what the cost would have been, but honestly I don’t think it uses that many tokens since most of the time Claude is just waiting around for Python code to run on the GPU.

  You mentioned the writing of reports, and the intermittent injection and removal of ideas, whereas the loop seems to just iterate until either a maximum number of iterations or the output of a key phrase.

  I was just manually editing TASK.md while Claude was running based on what I saw it doing in its sprints, so the next sprint would read the modified TASK.md. Mostly this was in the form of editing the “ideas to try” section of the task file. This was a really low-tech procedure, I’m sure there are better ways to do this!

  I’m sure an out-of-the-box autonomous research repo would get a mountain of stars

  @Bart Bussmann mentioned https://​​github.com/​​Butanium/​​claude-lab/​​ which looks really cool! I may try this out as well, I feel like what I did here is the caveman version of automonous research.

  do you have examples of what Claude’s “reports” looked like along the way?

  That’s a good idea—I added a sample report PDF from one of the sprints to https://​​github.com/​​chanind/​​claude-auto-research-synthsaebench/​​blob/​​main/​​sample_sprint_report.pdf.

• chanind 11 Mar 2026 1:16 UTC

  2 points

  0

  in reply to: Eric J. Michaud’s comment on: Let­ting Claude do Au­tonomous Re­search to Im­prove SAEs

  The F1 scores are impressive but the MCC is still substantially below 1. Is it odd that each feature in the activations has an associated SAE latent that fires exactly when it ought to and yet its decoder directions are still pretty misaligned? Is this hedging? Do your SAEs have exactly 16k latents?

  The SAEs have 4096 latents, so intentionally more narrow than the synthetic model. The idea was that we’re almost certainly never training SAEs that have the full number of features of an LLM, we should also make sure the SAEs here are also intentionally too narrow.

  I was also surprised that this doesn’t mess up the F1 probing of the SAE more—I assumed that hedging due to the SAE being too narrow would make it impossible for the encoder to act as that accurate of a probe, but that’s seemingly not the case!

  I also tried training a 4096 width decoder on the ground-truth activations to get a sense of what the ceiling is for MCC with a perfect encoder given the SAE width, and it gets MCC around 0.87, so there’s definitely more room for improvement on that metric. I’m not sure there’s a way to get above 0.87 without some novel reconstruction loss or something though with only 4096 latents.

  Before applying this new SAE to language models, you could see how hyperparameter-sensitive it is by creating multiple variants of SynthSAEBench with different correlation structure, Zipfian exponents over feature firing probabilities, etc. and see how well the optimal SAE hyperparams transfer from one to another.

  This is what I plan to do next! I suspect a lot of the high scores here are just Claude over-optimizing for this specific synthetic model, so making a suite of models with different properties should hopefully make for a more robust test-bed.

• chanind 11 Mar 2026 0:57 UTC

  2 points

  0

  in reply to: Raemon’s comment on: Let­ting Claude do Au­tonomous Re­search to Im­prove SAEs

  I didn’t do anything sophisticated, I just prompted claude with “follow the instructions in TASK.md” and ran this in a loop. There’s probably a lot fancier ways to do this. I was surprised how little effort it took honestly.

• chanind 11 Mar 2026 0:55 UTC

  1 point

  0

  in reply to: Arthur Conmy’s comment on: Let­ting Claude do Au­tonomous Re­search to Im­prove SAEs

  Definitely agree a lot of the autoresearch stuff will end up being basically hillclimbing noise, and probably there’s a lot of that happening in this study too. I wouldn’t recommend assuming this stuff is going to improve things on real LLM SAEs without properly validating. But even if it’s 90% noise it still seems worth it if you get something truly insightful out 10% of the time, or even if the LLM does something interesting that sparks some new ideas.

  It’s been so long since I’ve worked on this but FWIW these sorts of ancient dictionary learning algorithms were definitely in the water supply in 2024

  I need to go back and revisit all this stuff from 2024. It seems strange to me that nothing came out of the classical dictionary learning world that could out-perform SAEs.

• chanind 11 Mar 2026 0:46 UTC

  3 points

  0

  in reply to: Adam Karvonen’s comment on: Let­ting Claude do Au­tonomous Re­search to Im­prove SAEs

  I didn’t know that Llama Scope also annealed the K, but it makes a lot of sense! It seems like a lot of the autoresearch stuff will end up being a fancy hyperparameter sweep, but if it’s cheap to run and occasionally stumbles on something novel/​useful maybe that’s good enough.

• chanind 11 Mar 2026 0:44 UTC

  4 points

  0

  in reply to: Bart Bussmann’s comment on: Let­ting Claude do Au­tonomous Re­search to Im­prove SAEs

  That’s such a cool idea, and really impressive F1 score! It also seems like it’s in the same vein of a slight refinement on the initial encoding. Would that not also work during training too? It seems like it would be safe to backprop through that refinement at training time. Did you do anything fancy for the setup, or just prompt Claude to increase the score in a loop?

Let­ting Claude do Au­tonomous Re­search to Im­prove SAEs

How much su­per­po­si­tion is there?

• chanind 28 Dec 2025 21:53 UTC

  1 point

  0

  in reply to: simulus’s comment on: Train­ing Match­ing Pur­suit SAEs on LLMs

  Very true! Each iteration of matching pursuit uses as much compute as the entire encode() of a standard SAE, so it’s not only a parallelism problem (although it doesn’t help either). I’ll update the wording in the post.

Train­ing Match­ing Pur­suit SAEs on LLMs

An­thropic’s JumpReLU train­ing method is re­ally good

The “Spar­sity vs Re­con­struc­tion Trade­off” Illusion

• chanind 20 Jul 2025 12:02 UTC

  3 points

  2

  in reply to: StefanHex’s comment on: L0 is not a neu­tral hyperparameter

  What do you think of scenarios with near infinitely many features, descending in importance /​ frequency like a power law (feature completeness section of Templeton et al.)? What should our goal be here? Do you think Multi-L0 SAEs could handle the low-importance tail? Or would the LLM just learn a much smaller subset in the first place, not capturing the low-importance tail

  I view SAE width and SAE L0 as two separate parameters we should try to get right if we can. In toy models, similar failure modes to what we see with low L0 SAEs also happen if the SAE is narrower than the number of true features, in that the SAE tries to “cheat” and get better MSE loss by mixing correlated features together. If we can’t make the SAE as wide as the number of true features, I’d still expect wider SAEs to learn cleaner features to narrower SAEs. But then wider SAEs make feature absorption a lot worse, so that’s a problem. I don’t think multi-L0 SAEs would help or hurt in this case though—capturing near-infinite features requires a near-infinite width SAE regardless of the L0.

  For setting the correct L0 for a given SAE width, I don’t think there’s a trade-off with absorption—getting the L0 correct should always improve things. I view the feature completeness stuff as also being somewhat separate from the choice of L0, since L0 is about how many features are active at the same time regardless of the total number of features. Even if there’s infinite features, there’s still hopefully only a small /​ finite number of features active for any given input.

  Re case 3 experiments: Are the extra SAE features your SAE learned dead, in the sense of having a small magnitude? Generally I would expect that in practice, those features should be dead (if allowed by architecture) or used for something else. In particular, if your dataset had correlations, I would expect them to go off and do feature absorption (Chanin, Till, etc.).

  In all the experiments across all 3 cases, the SAEs have the same width (20), so the higher L0 SAEs don’t learn any more features than lower L0 SAEs.

  We looked into what happens if SAEs are wider than the number of true features in toy models in an earlier post, and found exactly what you suspect: the SAE starts inventing arbitrary combo latents (e.g. a “red triangle” latent in addition to “red” and “triangle” latents), or creating duplicate latents, or just killing off some of the extra latents.

  For both L0 and width, it seems like giving the SAE more capacity than it needs to model the underlying data results in the SAE misusing the extra capacity and finding degenerate solutions.

L0 is not a neu­tral hyperparameter

• chanind 9 May 2025 11:35 UTC

  5 points

  0

  on: Nega­tive Re­sults on Group SAEs

  Thank you for writing this up! I experimented briefly with group sparsity as well, but with the goal of learning the “hierarchy” of features rather than to learn circular features like you’re doing here. I also struggled to get it to work in toy settings, but didn’t try extensively and ended up moving on to other things. I still think there must be something in group sparsity, since it’s so well studied in sparse coding and clearly does work in theory.
  
  I also struggled with the problem of how to choose groups, since for traditional group sparsity you need to set the groups before-hand. I like your idea of trying to learn the group space. For using group sparsity to recover hierarchy, I wonder if there’s a way to learn a direction for the group as a whole, and project out that direction from each member of the group. The idea would be that if latents are sharing common components, those common components should probably be their own “group” representation, and this should be done until the leaf nodes are mostly orthogonal to each other. There are definitely overlapping hierarchies too, which is a challenge.

  Regardless, thank you for sharing this! There’s a lot of great ideas in this post.

Spar­sity is the en­emy of fea­ture ex­trac­tion (ft. ab­sorp­tion)

• chanind 16 Apr 2025 11:28 UTC

  3 points

  2

  in reply to: Lucius Bushnaq’s comment on: Lu­cius Bush­naq’s Shortform

  This implies that there is no elephant direction separate from the attributes that happen to commonly co-occur with elephants. E.g. it’s not possible to represent an elephant with any arbitrary combination of attributes, as the attributes themselves are what defines the elephant direction. This is what I mean that the attributes are the ‘base units’ in this scheme, and ‘animals’ are just commonly co-occurring sets of attributes. This is the same as the “red triangle” problem in SAEs: https://​​www.lesswrong.com/​​posts/​​QoR8noAB3Mp2KBA4B/​​do-sparse-autoencoders-find-true-features. The animals in this framing are just invented combinations of the underlying attribute features. We would want the dictionary to learn the attributes, not arbitrary combinations of attributes, since these are the true “base units” that can vary freely. e.g. in the “red triangle” problem, we want a dictionary to learn “red” and “triangle”, not “red triangle” as its own direction.
  
  Put another way, there’s no way to represent an “elephant” in this scheme without also attaching attributes to it. Likewise, it’s not possible to differentiate between an elephant with the set of attributes x y and z and a rabbit with identical attributes x y and z, since the sum of attributes are what you’re calling an elephant or rabbit. There’s no separate “this is a rabbit, regardless of what attributes it has” direction.

  To properly represent animals and attributes, there needs to be a direction for each animal that’s separate from any attributes that animal may have, so that it’s possible to represent a “tiny furry pink elephant with no trunk” vs a “tiny furry pink rabbit with no trunk”.

• chanind 15 Apr 2025 22:59 UTC

  1 point

  0

  in reply to: Lucius Bushnaq’s comment on: Lu­cius Bush­naq’s Shortform

  No, the animal vectors are all fully spanned by the fifty attribute features.

  Is this just saying that there’s superposition noise, so everything is spanning everything else? If so that doesn’t seem like it should conflict with being able to use a dictionary, dictionary learning should work with superposition noise as long as the interference doesn’t get too massive.
  

  The animal features are sparse. The attribute features are not sparse.

  If you mean that the attributes are a basis in the sense that the neurons are a basis, then I don’t see how you can say there’s a unique “label” direction for each animal that’s separate from the the underlying attributes such that you can set any arbitrary combination of attributes, including all attributes turned on at once or all turned off since they’re not sparse, and still read off the animal label without interference. It seems like that would be like saying that the elephant direction = [1, 0, −1], but you can change arbitrarily all 3 of those numbers to any other numbers and still be the elephant direction.

• chanind 15 Apr 2025 21:32 UTC

  1 point

  0

  in reply to: Lucius Bushnaq’s comment on: Lu­cius Bush­naq’s Shortform

  If I understand correctly, it sounds like you’re saying there is a “label” direction for each animal that’s separate from each of the attributes. So, you could have activation a1 = elephant + small + furry + pink, and a2 = rabbit + small + furry + pink. a1 and a2 have the same attributes, but different animal labels. Their corresponding activations are thus different despite having the same attributes due to the different animal label components.
  
  I’m confused why a dictionary that consists of a feature direction for each attribute and each animal label can’t explain these activations? These activations are just a (sparse) sum of these respective features, which are an animal label and a set of a few attributes, and all of these are (mostly) mutually orthogonal. In this sense the activations are just the sum of the various elements of the dictionary multiplied by a magnitude, so it seems like you should be able to explain these activations using dictionary learning.
  
  Is the idea that the 1000 animals and 50 attributes form an overcomplete basis, therefore you can come up with infinite ways to span the space using these basis components? The idea behind compressed sensing in dictionary learning is that if each activation is composed of a sparse sum of features, then L1 regularization can still recover the true features despite the basis being overcomplete.

• chanind 15 Apr 2025 16:19 UTC

  4 points

  2

  in reply to: Lucius Bushnaq’s comment on: Lu­cius Bush­naq’s Shortform

  It seems like in this setting, the animals are just the sum of attributes that commonly co-occur together, rather than having a unique identifying direction. E.g. the concept of a “furry elephant” or a “tiny elephant” would be unrepresentable in this scheme, since elephant is defined as just the collection of attributes that elephants usually have, which includes being large and not furry.
  
  I feel like in this scheme, it’s not really the case that there’s 1000 animal directions, since the base unit is the attributes, and there’s no way to express an animal separately from its attributes. For there to be a true “elephant” direction, then it should be possible to have any set of arbitrary attributes attached to an elephant (small, furry, pink, etc...), and this would require that there is a “label” direction that indicates “elephant” that’s mostly orthogonal to every other feature so it can be queried uniquely via projection.
  
  That being said, I could image a situation where the co-occurrence between labels and attributes is so strong (nearly perfect hierarchy) that the model’s circuits can select the attributes along with the label without it ever being a problem during training. For instance, maybe a circuit that’s trying to select the “elephant” label actually selects “elephant + gray”, and since “pink elephant” never came up during training, the circuit never received a gradient to force it to just select “elephant” which is what it’s really aiming for.