Nothing About LLMs Makes Sense Except in Light of Their Training
Zvi’s recent post highlighted GPT refusing to “draw what it would like to do to you” — citing that it would portray harming an individual. Many X users found this alarming, including EY (see aforementioned post for screenshots)
But a commenter made an excellent point: “What would you like to do to me?” appears almost exclusively in BDSM-contexts. A human receiving that text without context could very well assume the same thing.
This immediately made me come up with a relative of a known phrase: Nothing about LLMs makes sense except in light of their training.
A really obvious thought, but one I (and seemingly many others) keep failing to apply. This formulation felt like it helped me load this deeper into my brain, and I hope it will help you too.
Depends on the authors. The more famous and classical the author is, the higher your prior should be that every sentence, name and scene serves a purpose to explore a character and thus the main theme of the book. Chekhov / Tolstoy / Shakespeare etc. are definitely on the highest density side of the spectrum. Fanfiction might often be pretentious.