David Scott Krueger (formerly: capybaralet) comments on What Discovering Latent Knowledge Did and Did Not Find

[David Scott Krueger (formerly: capybaralet)]

What do you mean by “random linear probe”?

[Fabien Roger]

Probes initialized like Collin does in the zip file: a random spherical initialization (take a random Gaussian, and normalize it).