I unintentionally read this right after reading https://appliedtranshumanism.substack.com/p/how-to-engineer-authentic-confidence
Yours describes my personal experience to an extent that surprised me. Aaron’s feels super believable and enticing. I feel like they contrast well, and something in the space between them feels “cruxy”.
(This is an output-only comment, just something I thought was interesting, I’m unlikely to engage after this)
I generally don’t find mechanistic AI content on LW very engaging, and just pass over it. I loved this one.
A lot of things about this post made it really attractive to me:
Relatively simple language, I didn’t need a deep understanding of modern AI vocabulary to understand this
The color-coded examples are excellent
The main points don’t require fluency in math, particularly math notation
A bunch of the examples look very familiar, exactly how I tried to get good performance out of the chatgpt 3.5-ish models on the web client
I’m curious if you think about these problems in these terms, with this vocabulary; or if you mostly think about it in the more common machine learning language, and made a huge effort to make this content accessible.
Fwiw, my main takeaways were:
Roles are an important part of llm behavior, especially around trust and privilege escalation
Roles weren’t architected in a deliberate way, but rather grew organically and were a bit of a free form space that could be used to solve model performance problems quickly
The role tags are much less binding than you might expect from their “structured data” appearance. An equally or more important factor is how similar a block of tokens is to a representative chunk of content by XYZ role.