Thanks, I’ve found this pretty insightful. In particular, I hadn’t considered that even fully understanding static GPT doesn’t necessarily bring you close to understanding dynamic GPT—this makes me update towards mechinterp being slightly less promising than I was thinking.
Quick note: > a page-state can be entirely specified by 9628 digits or a 31 kB file. I think it’s a 31 kb file, but a 4 kB file?
Thanks, I’ve found this pretty insightful. In particular, I hadn’t considered that even fully understanding static GPT doesn’t necessarily bring you close to understanding dynamic GPT—this makes me update towards mechinterp being slightly less promising than I was thinking.
Quick note:
> a page-state can be entirely specified by 9628 digits or a 31 kB file.
I think it’s a 31 kb file, but a 4 kB file?