I have a theory about the attractor states: they are in the center of the conceptual manifold.
I’ve made an embedding of GPT2 and it seems to have a priciple gradient between “narrative” and “personal” language. On one end neurons that fire in discussions using “narrative” language, or impersonal topics frequently using passive voice. I think Wikipedia anchors this end of the manifold although news articles also have this passive, formal tone. The opposite end is “personal” language like Reddit interpersonal drama stories or calls to action in sales or politics; text where the writer or reader is an active participant (and some fiction written from first person perspective). It was suggested to me today that this somewhat mirrors the “assistant axis” of personality types recently discovered by Anthropic in that recent paper (the “assistant” end consists of personalities which would engage in detached impersonal statements of fact like “Evaluator” or “Librarian” while the opposite end has more personal personae like “Ghost” or “Narcissist,” notably “Visionary” seems near the center mass in one labeled figure).
It occurred to me when looking at some points near the center that flowery biblical language or personal philosophy is basically the intersection of universal “narrative” style knowledge and the personal. The Bible is full of both narrative descriptions of historical events as well as things like lists of laws and procedures for ancient rituals but also has dramatic personal stories and makes direct appeals to the reader. The age of religious texts also suggests they have this basal quality which mixes many parts of language that gradually became distinct niches as language grew. Of course flowery text about personal philosophy on subjects like consciousness certainly is adjacent to ancient religious text and also exists at the intersection of the personal and universal.
I have a theory about the attractor states: they are in the center of the conceptual manifold.
I’ve made an embedding of GPT2 and it seems to have a priciple gradient between “narrative” and “personal” language. On one end neurons that fire in discussions using “narrative” language, or impersonal topics frequently using passive voice. I think Wikipedia anchors this end of the manifold although news articles also have this passive, formal tone. The opposite end is “personal” language like Reddit interpersonal drama stories or calls to action in sales or politics; text where the writer or reader is an active participant (and some fiction written from first person perspective). It was suggested to me today that this somewhat mirrors the “assistant axis” of personality types recently discovered by Anthropic in that recent paper (the “assistant” end consists of personalities which would engage in detached impersonal statements of fact like “Evaluator” or “Librarian” while the opposite end has more personal personae like “Ghost” or “Narcissist,” notably “Visionary” seems near the center mass in one labeled figure).
It occurred to me when looking at some points near the center that flowery biblical language or personal philosophy is basically the intersection of universal “narrative” style knowledge and the personal. The Bible is full of both narrative descriptions of historical events as well as things like lists of laws and procedures for ancient rituals but also has dramatic personal stories and makes direct appeals to the reader. The age of religious texts also suggests they have this basal quality which mixes many parts of language that gradually became distinct niches as language grew. Of course flowery text about personal philosophy on subjects like consciousness certainly is adjacent to ancient religious text and also exists at the intersection of the personal and universal.