Congrats @wesg on the huge win for interpretability! In the paper, you asked:
“The J-lens… accounts for less than a tenth of the overall activity in Claude’s internal processing. So what is all the rest of the neural network doing?”
You indirectly answer this by ablating the J-space to measure the effect of the rest—I read the non-J space directly by using Anthropic’s NLAs on Llama-70B, and found that it’s causal even when the model flatly denies it.
I split concept vectors into their J-space/non-J parts, and ran Lindsey’s “Introspection Awareness” experiment on both halves.
The model denies any non-J injection, but the NLA reads it! Here’s some examples:
And the non-J side isn’t a weak residual—it drives the injected token’s logprob up +21 nats while the model names it 0% of the time and the NLA ~100%. It’s causal, but the model is blind to it.
I even stress-tested this with “Franken-vectors” gluing one concept’s subconscious half to another’s conscious half. The model only ever names the J-side concept; the NLA reads both!
This gets at @Neel Nanda’s skepticism from his review, flagging that he wasn’t “confident enough in NLAs to be confident the ground truth in the case studies is correct.”
The injection setup gives that ground truth by construction: known concept in, and the NLA names it with zero false positives across four controls (no-injection, random-in-J, random-out-of-J, mislabeled). But this is on synthetic injections, so the natural next step is applying on natural activation scenarios like line counting and eval awareness.
I’m also interested in your take on how fixed the J-rank should be—I found conscious reportability scales smoothly as we scale rank (as non-J space goes dark), so the mental workspace doesn’t have a sharp cutoff. Curious whether the boundary you drew is based on reportability, causality, or something else.
If interested, I go more in depth about this J-space x NLA study in my LW writeup with code on GitHub. Thank you and congrats again!
Thanks for the kind words Joseph, appreciate the read!