Words as Mental Paintbrush Handles

(We should be done with the mathy posts, I think, at least for now. But for­give me if, iron­i­cally, I end up re­sort­ing to Ra­tion­al­ity Quotes for a day or two. I’m cur­rently at the AGI-08 con­fer­ence, which, as of the first ses­sion, is not nearly so bad as I feared.)

Sup­pose I tell you: “It’s the strangest thing: The lamps in this ho­tel have tri­an­gu­lar light­bulbs.”

You may or may not have vi­su­al­ized it—if you haven’t done it yet, do so now—what, in your mind’s eye, does a “tri­an­gu­lar light­bulb” look like?

In your mind’s eye, did the glass have sharp edges, or smooth?

When the phrase “tri­an­gu­lar light­bulb” first crossed my mind—no, the ho­tel doesn’t have them—then as best as my in­tro­spec­tion could de­ter­mine, I first saw a pyra­mi­dal light­bulb with sharp edges, then (al­most im­me­di­ately) the edges were smoothed, and then my mind gen­er­ated a loop of floures­cent bulb in the shape of a smooth tri­an­gle as an al­ter­na­tive.

As far as I can tell, no de­liber­a­tive/​ver­bal thoughts were in­volved—just word­less re­flex flinch away from the imag­i­nary men­tal vi­sion of sharp glass, which de­sign prob­lem was solved be­fore I could even think in words.

Believe it or not, for some decades, there was a se­ri­ous de­bate about whether peo­ple re­ally had men­tal images in their mind—an ac­tual pic­ture of a chair some­where—or if peo­ple just naively thought they had men­tal images (hav­ing been mis­led by “in­tro­spec­tion”, a very bad for­bid­den ac­tivity), while ac­tu­ally just hav­ing a lit­tle “chair” la­bel, like a LISP to­ken, ac­tive in their brain.

I am try­ing hard not to say any­thing like “How spec­tac­u­larly silly,” be­cause there is always the hind­sight effect to con­sider, but: how spec­tac­u­larly silly.

This aca­demic paradigm, I think, was mostly a de­ranged legacy of be­hav­iorism, which de­nied the ex­is­tence of thoughts in hu­mans, and sought to ex­plain all hu­man phe­nom­ena as “re­flex”, in­clud­ing speech. Be­hav­iorism prob­a­bly de­serves its own post at some point, as it was a per­ver­sion of ra­tio­nal­ism; but this is not that post.

“You call it ‘silly’,” you in­quire, “but how do you know that your brain rep­re­sents vi­sual images? Is it merely that you can close your eyes and see them?”

This ques­tion used to be harder to an­swer, back in the day of the con­tro­versy. If you wanted to prove the ex­is­tence of men­tal imagery “sci­en­tifi­cally”, rather than just by in­tro­spec­tion, you had to in­fer the ex­is­tence of men­tal imagery from ex­per­i­ments like, e.g.: Show sub­jects two ob­jects and ask them if one can be ro­tated into cor­re­spon­dence with the other. The re­sponse time is lin­early pro­por­tional to the an­gle of ro­ta­tion re­quired. This is easy to ex­plain if you are ac­tu­ally vi­su­al­iz­ing the image and con­tin­u­ously ro­tat­ing it at a con­stant speed, but hard to ex­plain if you are just check­ing propo­si­tional fea­tures of the image.

To­day we can ac­tu­ally neu­roimage the lit­tle pic­tures in the vi­sual cor­tex. So, yes, your brain re­ally does rep­re­sent a de­tailed image of what it sees or imag­ines. See Stephen Koss­lyn’s Image and Brain: The Re­s­olu­tion of the Imagery De­bate.

Part of the rea­son peo­ple get in trou­ble with words, is that they do not re­al­ize how much com­plex­ity lurks be­hind words.

Can you vi­su­al­ize a “green dog”? Can you vi­su­al­ize a “cheese ap­ple”?

“Ap­ple” isn’t just a se­quence of two syl­la­bles or five let­ters. That’s a shadow. That’s the tip of the tiger’s tail.

Words, or rather the con­cepts be­hind them, are paint­brushes—you can use them to draw images in your own mind. Liter­ally draw, if you em­ploy con­cepts to make a pic­ture in your vi­sual cor­tex. And by the use of shared la­bels, you can reach into some­one else’s mind, and grasp their paint­brushes to draw pic­tures in their minds—sketch a lit­tle green dog in their vi­sual cor­tex.

But don’t think that, be­cause you send syl­la­bles through the air, or let­ters through the In­ter­net, it is the syl­la­bles or the let­ters that draw pic­tures in the vi­sual cor­tex. That takes some com­plex in­struc­tions that wouldn’t fit in the se­quence of let­ters. “Ap­ple” is 5 bytes, and draw­ing a pic­ture of an ap­ple from scratch would take more data than that.

“Ap­ple” is merely the tag at­tached to the true and word­less ap­ple con­cept, which can paint a pic­ture in your vi­sual cor­tex, or col­lide with “cheese”, or rec­og­nize an ap­ple when you see one, or taste its archetype in ap­ple pie, maybe even send out the mo­tor be­hav­ior for eat­ing an ap­ple...

And it’s not as sim­ple as just call­ing up a pic­ture from mem­ory. Or how would you be able to vi­su­al­ize com­bi­na­tions like a “tri­an­gu­lar light­bulb”—im­pos­ing tri­an­gle­ness on light­bulbs, keep­ing the essence of both, even if you’ve never seen such a thing in your life?

Don’t make the mis­take the be­hav­iorists made. There’s far more to speech than sound in air. The la­bels are just poin­t­ers—”look in mem­ory area 1387540″. Sooner or later, when you’re handed a poin­ter, it comes time to derefer­ence it, and ac­tu­ally look in mem­ory area 1387540.

What does a word point to?