Thanks for the post! Not sure if I understood (I like it) it correctly about the idea of alignment between the geometry and the learned dictionary: this produces “separation” between features that is closer to the “true” dictionary—maybe it’s just a paraphrasing of the same thing, but what did you mean by “semantic continuity”?
Tommy Xie
Karma: 4
View as possible compression of human language? Can change my thoughts If they prove to have no consistency, or not interpretable/decodable by LLM.