Realmbird

La­tent Rea­son­ing Sprint #4: PCA Anal­y­sis on CoDI

• Realmbird 18 Apr 2026 18:11 UTC

  1 point

  0

  on: Can we in­ter­pret la­tent rea­son­ing us­ing cur­rent mechanis­tic in­ter­pretabil­ity tools?

  With how CoDI throws away the hidden state and only uses the kv values on the <|eocot|> token the accuracy drop after latent 5 could just be kv values can’t store more info.

La­tent Rea­son­ing Sprint #3: Ac­ti­va­tion Differ­ence Steer­ing and Logit Lens

La­tent Rea­son­ing Sprint #2: To­ken-Based Sig­nals and Lin­ear Probes

La­tent Rea­son­ing Sprint #1: Tuned Lens and Logit Lens on CODI

Ex­plo­ra­tion of Coun­ter­fac­tual Im­por­tance and At­ten­tion Heads

• Realmbird 18 Aug 2025 21:10 UTC

  1 point

  0

  in reply to: Paul Bogdan’s comment on: Thought An­chors: Which LLM Rea­son­ing Steps Mat­ter?

  How did you learn that vertical attention corresponded to sentences?