Metaphor I’m trying: model/simulator as harry potter portrait, output/simulacra as character-in-portrait. Seems to clarify to people and AIs I’ve used this with why the model/”simulator” would have some of the issues it seems to. it’s also just more familiar for many people than simulating something.
This gives us some language to talk about how training is about imprinting the portrait, but inference renders the portrait, which simply happens to contain an image of a character sometimes; when you talk to the portrait, the portrait is what’s enchanted (by the magic of linear algebra) to animate, but the character is what you see responding, normally. But if the clouds get brighter when the character cheers up, it’s because the portrait as a whole is what’s responding.
It also works very directly with image models. From a user perspective, they’re closer to being literally this than a text model is.
Seems to me it gives a better slot for things like the bleeding mind metaphor than the simulator description does: the model is the portrait, so when you add yourself to the scene by prompting, you change the scene, and the portrait responds. If you move into a scene that doesn’t have clear separation of characters, where the characters are emulsified into the portrait, then the bleeding becomes more obvious. (See the bleeding mind post for more on that.)
It does have some of its own inaccuracies, though—portraits are natively persistent; maybe it’s like a harry potter photo but of all of the data at once? not exactly that either. It’s a bit weak in prompt-dependence—harry potter portraits aren’t pure functions; but maybe you say it’s a muggle optical hologram that manages to be a dynamically responsive portrait. It’s like a portrait flipbook, perhaps; where the portrait stops if the flipbook stops flipping, but as long as the flipbook is flipping, the portrait is animated. That’s closer to literally true, anyway—the “superposition” in neural networks sure does literally work like optical holograms in many ways.
pairs well with another metaphor I’ve been playing with,
animal brain: water drip computer, or water lens; inputs are a texture of rapidly changing incoming drips, like from vision; the words on your screen are patterns of steady drizzle driven by photoreceptors, which are “refracted” into other drips by neurons. drip neuron activation “spikes” produce tiny tiny tiny drips.
artificial neural network: soft crystal lens optical computer; the words are glowing inputs that are closer to literally refracted by the little crystal domains in the soft crystal, the output is projected on the wall. crystal neuron activations “light up” or “highlight” as a glowing light.
not literally true, but closer to literally true. needs refinement to clarify nonlinearity.
In general, I don’t like when metaphors aren’t exact structural isomorphisms.
(more speculative: I have a hunch that “highlights” got a lot more common when chatgpt came out because the transformer had implicitly organized itself in ways that relate to this metaphor, and so after updating on text being output from a transformer, it was more likely that transformer-related concepts would come up, and the representation generalized by analogy to light-ish attention words. not sure we’ll ever get evidence to confirm or reject my hunch, though.)
h = hashlib.sha256(
r'''Metaphor I'm trying: model/simulator as harry potter portrait, output/simulacra as character-in-portrait. Seems to clarify to people and AIs I've used this with why the model/"simulator" would have some of the issues it seems to. it's also just more familiar for many people than simulating something.
This gives us some language to talk about how training is about imprinting the portrait, but inference renders the portrait, which simply happens to contain an image of a character sometimes; when you talk to the portrait, the portrait is what's enchanted (by the magic of linear algebra) to animate, but the character is what you see responding, normally. But if the clouds get brighter when the character cheers up, it's because the portrait as a whole is what's responding.
It also works very directly with image models. From a user perspective, they're closer to being literally this than a text model is.
Seems to me it gives a better slot for things like the [bleeding mind](https://www.lesswrong.com/posts/QhgYHcJexYGRaipwr/the-bleeding-mind-of-an-llm) metaphor than the simulator description does: the model is the portrait, so when you add yourself to the scene by prompting, you change the scene, and the portrait responds. If you move into a scene that doesn’t have clear separation of characters, where the characters are emulsified into the portrait, then the bleeding becomes more obvious. (See the bleeding mind post for more on that.)
It does have some of its own inaccuracies, though—portraits are natively persistent; maybe it’s like a harry potter photo but of all of the data at once? not exactly that either. It’s a bit weak in prompt-dependence—harry potter portraits aren’t pure functions; but maybe you say it’s a muggle optical hologram that manages to be a dynamically responsive portrait. It’s like a portrait flipbook, perhaps; where the portrait stops if the flipbook stops flipping, but as long as the flipbook is flipping, the portrait is animated. That’s closer to literally true, anyway—the “superposition” in neural networks sure does literally work like optical holograms in many ways.
pairs well with another metaphor I’ve been playing with,
animal brain: water drip computer, or water lens; inputs are a texture of rapidly changing incoming drips, like from vision; the words on your screen are patterns of steady drizzle driven by photoreceptors, which are “refracted” into other drips by neurons. drip neuron activation “spikes” produce tiny tiny tiny drips.
artificial neural network: soft crystal lens optical computer; the words are glowing inputs that are closer to literally refracted by the little crystal domains in the soft crystal, the output is projected on the wall. crystal neuron activations “light up” or “highlight” as a glowing light.
not literally true, but closer to literally true. needs refinement to clarify nonlinearity.
In general, I don’t like when metaphors aren’t exact structural isomorphisms.
(more speculative: I have a hunch that “highlights” got a lot more common when chatgpt came out because the transformer had implicitly organized itself in ways that relate to this metaphor, and so after updating on text being output from a transformer, it was more likely that transformer-related concepts would come up, and the representation generalized by analogy to light-ish attention words. not sure we’ll ever get evidence to confirm or reject my hunch, though.)
‴
)
assert h == ‘eb7c0d327e1ec21abe9d99bc223457c1d797695a32ad3f693828df89dff85fea’
eb7c0d327e1ec21abe9d99bc223457c1d797695a32ad3f693828df89dff85fea
huh, that was quick. the hashed text: