StefanHex

Karma: 1,705

Stefan Heimersheim. Research Scientist at Apollo Research, Mechanistic Interpretability. The opinions expressed here are my own and do not necessarily reflect the views of my employer.

Residual stream norms grow exponentially over the forward pass

StefanHex and TurnTrout

May 7, 2023, 12:46 AM

77 points

24 comments11 min readLW link

A circuit for Python docstrings in a 4-layer attention-only transformer

StefanHex and Jett Janiak

Feb 20, 2023, 7:35 PM

96 points

8 comments21 min readLW link

How-to Transformer Mechanistic Interpretability—in 50 lines of code or less!

StefanHexJan 24, 2023, 6:45 PM

47 points

5 comments13 min readLW link

Reinforcement Learning Goal Misgeneralization: Can we guess what kind of goals are selected by default?

StefanHex and Julian_R

Oct 25, 2022, 8:48 PM

15 points

2 comments4 min readLW link

Research Questions from Stained Glass Windows

StefanHexJun 8, 2022, 12:38 PM

4 points

0 comments2 min readLW link

CNN feature visualization in 50 lines of code

StefanHexMay 26, 2022, 11:02 AM

17 points

4 comments5 min readLW link

StefanHex

Resi­d­ual stream norms grow ex­po­nen­tially over the for­ward pass

A cir­cuit for Python doc­strings in a 4-layer at­ten­tion-only transformer

How-to Trans­former Mechanis­tic In­ter­pretabil­ity—in 50 lines of code or less!

Re­in­force­ment Learn­ing Goal Mis­gen­er­al­iza­tion: Can we guess what kind of goals are se­lected by de­fault?

Re­search Ques­tions from Stained Glass Windows