Jett(Jett Mayzner)

AISC pro­ject: TinyEvals

Poly­se­man­tic At­ten­tion Head in a 4-Layer Transformer

• Jett 9 Oct 2023 8:41 UTC

  1 point

  0

  on: A Com­pre­hen­sive Mechanis­tic In­ter­pretabil­ity Ex­plainer & Glossary

  The activation patching, causal tracing and resample ablation terms seem to be out of date, compared to how you define them in your post on attribution patching.

An ad­ver­sar­ial ex­am­ple for Direct Logit At­tri­bu­tion: mem­ory man­age­ment in gelu-4l

A cir­cuit for Python doc­strings in a 4-layer at­ten­tion-only transformer