Maybe Lying Can’t Exist?!

How is it pos­si­ble to tell the truth?

I mean, sure, you can use your lar­ynx to make sound waves in the air, or you can draw a se­quence of sym­bols on pa­per, but sound waves and pa­per-mark­ings can’t be true, any more than a leaf or a rock can be “true”. Why do you think you can tell the truth?

This is a pretty easy ques­tion. Words don’t have in­trin­sic on­tolog­i­cally-ba­sic mean­ings, but in­tel­li­gent sys­tems can learn as­so­ci­a­tions be­tween a sym­bol and things in the world. If I say “dog” and point to a dog a bunch of times, a child who didn’t already know what the word “dog” meant, would soon get the idea and learn that the sound “dog” meant this-and-such kind of furry four-legged an­i­mal.

As a for­mal model of how this AI trick works, we can study sender–re­ceiver games. Two agents, a “sender” and a “re­ceiver”, play a sim­ple game: the sender ob­serves one of sev­eral pos­si­ble states of the world, and sends one of sev­eral pos­si­ble sig­nals—some­thing that the sender can vary (like sound waves or pa­per-mark­ings) in a way that the re­ceiver can de­tect. The re­ceiver ob­serves the sig­nal, and makes a pre­dic­tion about the state of the world. If the agents both get re­warded when the re­ceiver’s pre­dic­tion matches the sender’s ob­ser­va­tion, a con­ven­tion evolves that as­signs com­mon-us­age mean­ings to the pre­vi­ously and oth­er­wise ar­bi­trary sig­nals. True in­for­ma­tion is com­mu­ni­cated; the sig­nals be­come a shared map that re­flects the ter­ri­tory.

This works be­cause the sender and re­ceiver have a com­mon in­ter­est in get­ting the same, cor­rect an­swer—in co­or­di­nat­ing for the sig­nals to mean some­thing. If in­stead the sender got re­warded when the re­ceiver made bad pre­dic­tions, then if the re­ceiver could use some cor­re­la­tion be­tween the state of the world and the sender’s sig­nals in or­der to make bet­ter pre­dic­tions, then the sender would have an in­cen­tive to change its sig­nal­ing choices to de­stroy that cor­re­la­tion. No con­ven­tion evolves, no in­for­ma­tion gets trans­ferred. This case is not a mat­ter of a map failing to re­flect the ter­ri­tory. Rather, there just is no map.


How is it pos­si­ble to lie?

This is … a sur­pris­ingly less-easy ques­tion. The prob­lem is that, in the for­mal frame­work of the sender–re­ceiver game, the mean­ing of a sig­nal is sim­ply how it makes a re­ceiver up­date its prob­a­bil­ities, which is de­ter­mined by the con­di­tions un­der which the sig­nal is sent. If I say “dog” and four-fifths of the time I point to a dog, but one-fifth of the time I point to a tree, what should a child con­clude? Does “dog” mean dog-with-prob­a­bil­ity-0.8-and-tree-with-prob­a­bil­ity-0.2, or does “dog” mean dog, and I’m just ly­ing one time out of five? (Or does “dog” mean tree, and I’m ly­ing four times out of five?!) Our sender–re­ceiver game model would seem to fa­vor the first in­ter­pre­ta­tion.

Sig­nals con­vey in­for­ma­tion. What could make a sig­nal, in­for­ma­tion, de­cep­tive?

Tra­di­tion­ally, de­cep­tion has been re­garded as in­ten­tion­ally caus­ing some­one to have a false be­lief. As Bayesi­ans and re­duc­tion­ists, how­ever, we en­deavor to pry open an­thro­po­mor­phic black boxes like “in­tent” and “be­lief.” As a first at­tempt at mak­ing sense of de­cep­tive sig­nal­ing, let’s gen­er­al­ize “caus­ing some­one to have a false be­lief” to “caus­ing the re­ceiver to up­date its prob­a­bil­ity dis­tri­bu­tion to be less ac­cu­rate (op­er­a­tional­ized as the log­a­r­ithm of the prob­a­bil­ity it as­signs to the true state)”, and gen­er­al­ize “in­ten­tion­ally” to “benefit­ing the sender (op­er­a­tional­ized by the re­wards in the sender–re­ceiver game)”.

One might ask: why re­quire the sender to benefit in or­der for a sig­nal to count as de­cep­tive? Why isn’t “made the re­ceiver up­date in the wrong di­rec­tion” enough?

The an­swer is that we’re seek­ing an ac­count of com­mu­ni­ca­tion that sys­tem­at­i­cally makes re­ceivers up­date in the wrong di­rec­tion—sig­nals that we can think of as hav­ing been op­ti­mized for mak­ing the re­ceiver make wrong pre­dic­tions, rather than ac­ci­den­tally hap­pen­ing to mis­lead on this par­tic­u­lar oc­ca­sion. The “re­wards” in this model should be in­ter­preted mechanis­ti­cally, not nec­es­sar­ily men­tal­is­ti­cally: it’s just that things that get “re­warded” more, hap­pen more of­ten. That’s all—and that’s enough to shape the evolu­tion of how the sys­tem pro­cesses in­for­ma­tion. There need not be any con­scious mind that “feels happy” about get­ting re­warded (al­though that would do the trick).

Let’s test out our pro­posed defi­ni­tion of de­cep­tion on a con­crete ex­am­ple. Con­sider a fire­fly of the fic­tional species P. rey ex­plor­ing a new area in the for­est. Sup­pose there are three pos­si­bil­ities for what this area could con­tain. With prob­a­bil­ity 13, the area con­tains an­other P. rey fire­fly of the op­po­site sex, available for mat­ing. With prob­a­bil­ity 16, the area con­tains a fire­fly of a differ­ent species, P. reda­tor, which eats P. rey fire­flies. With prob­a­bil­ity 12, the area con­tains noth­ing of in­ter­est.

A po­ten­tial mate in the area can flash the P. rey mat­ing sig­nal to let the ap­proach­ing P. rey know it’s there. Fire­flies evolved their epony­mous abil­ity to emit light speci­fi­cally for this kind of sex­ual com­mu­ni­ca­tion—po­ten­tial mates have a com­mon in­ter­est in mak­ing their pres­ence known to each other. Upon re­ceiv­ing the mat­ing sig­nal, the ap­proach­ing P. rey can elimi­nate the preda­tor-here and noth­ing-here states, and up­date its what’s-in-this-area prob­a­bil­ity dis­tri­bu­tion from { mate, preda­tor, noth­ing} to { mate}. True in­for­ma­tion is com­mu­ni­cated.

Un­til “one day” (in evolu­tion­ary time), a mu­tant P. reda­tor emits flashes that imi­tate the P. rey mat­ing sig­nal, thereby lur­ing an ap­proach­ing P. rey, who be­comes an easy meal for the P. reda­tor. This meets our crite­ria for de­cep­tive sig­nal­ing: the P. rey re­ceiver up­dates in the wrong di­rec­tion (re­vis­ing its prob­a­bil­ity of a P. reda­tor be­ing pre­sent down­wards from to 0, even though a P. reda­tor is in fact pre­sent), and the P. reda­tor sender benefits (be­com­ing more likely to sur­vive and re­pro­duce, thereby spread­ing the mu­tant alle­les that pre­dis­posed it to emit P. rey-mat­ing-sig­nal-like flashes, thereby en­sur­ing that this sce­nario will sys­tem­at­i­cally re­cur in fu­ture gen­er­a­tions, even if the first time was an ac­ci­dent be­cause fire­flies aren’t that smart).

Or rather, this meets our crite­ria for de­cep­tive sig­nal­ing at first. If the P. rey pop­u­la­tion coun­ter­adapts to make cor­rect Bayesian up­dates in the new world con­tain­ing de­cep­tive P. reda­tors, then in the new equil­ibrium, see­ing the mat­ing sig­nal causes a P. rey to up­date its what’s-in-this-area prob­a­bil­ity dis­tri­bu­tion from { mate, preda­tor, noth­ing} to { mate, preda­tor}. But now the coun­ter­adapted P. rey is not up­dat­ing in the wrong di­rec­tion. If both mates and preda­tors send the same sig­nal, than the like­li­hood ra­tio be­tween them is one; the ob­ser­va­tion doesn’t fa­vor one hy­poth­e­sis more than the other.

So … is the P. reda­tor’s use of the mat­ing sig­nal no longer de­cep­tive af­ter it’s been “priced in” to the new equil­ibrium? Should we stop call­ing the flashes the “P. rey mat­ing sig­nal” and start call­ing it the “P. rey mat­ing and/​or P. reda­tor prey-lur­ing sig­nal”? Do we agree with the ex­ec­u­tive in Mo­ral Mazes who said, “We lie all the time, but if ev­ery­one knows that we’re ly­ing, is a lie re­ally a lie?”

Some au­thors are will­ing to bite this bul­let in or­der to pre­serve our tidy for­mal defi­ni­tion of de­cep­tion. (Don Fal­lis and Peter J. Lewis write: “Although we agree [...] that it seems de­cep­tive, we con­tend that the mat­ing sig­nal sent by a [preda­tor] is not ac­tu­ally mis­lead­ing or de­cep­tive [...] not all sneaky be­hav­ior (such as failing to re­veal the whole truth) counts as de­cep­tion”.)

Per­son­ally, I don’t care much about hav­ing tidy for­mal defi­ni­tions of English words; I want to un­der­stand the gen­eral laws gov­ern­ing the con­struc­tion and per­ver­sion of shared maps, even if a de­tailed un­der­stand­ing re­quires re­vis­ing or split­ting some of our in­tu­itive con­cepts. (Cailin O’Con­nor writes: “In the case of de­cep­tion, though, part of the is­sue seems to be that we gen­er­ally ground judg­ments of what is de­cep­tive in terms of hu­man be­hav­ior. It may be that there is no neat, uni­tary con­cept un­der­ly­ing these judg­ments.”)

Whether you choose to de­scribe it with the sig­nal/​word “de­cep­tive”, “sneaky”, Täuschung, הונאה, 欺瞞, or some­thing else, some­thing about P. reda­tor’s sig­nal us­age has the op­ti­miz­ing-for-the-in­ac­cu­racy-of-shared-maps prop­erty. There is a fun­da­men­tal asym­me­try un­der­ly­ing why we want to talk about a mat­ing sig­nal rather than a 2/​3-mat­ing-1/​3-prey-lur­ing sig­nal, even if the lat­ter is a bet­ter de­scrip­tion of the in­for­ma­tion it con­veys.

Brian Skyrms and Jeffrey A. Bar­rett have an ex­pla­na­tion in light of the ob­ser­va­tion that our sender–re­ceiver frame­work is a se­quen­tial game: first, the sender makes an ob­ser­va­tion (or equiv­a­lently, Na­ture chooses the type of sender—mate, preda­tor, or null in the story about fire­flies), then the sender chooses a sig­nal, then the re­ceiver chooses an ac­tion. We can sep­a­rate out the propo­si­tional con­tent of sig­nals from their in­for­ma­tional con­tent by tak­ing the propo­si­tional mean­ing to be defined in the sub­game where the sender and re­ceiver have a com­mon in­ter­est—the branches of the game tree where the play­ers are try­ing to com­mu­ni­cate.

Thus, we see that de­cep­tion is “on­tolog­i­cally par­a­sitic” in the sense that holes are. You can’t have a hole with­out some ma­te­rial for it to be a hole in; you can’t have a lie with­out some shared map for it to be a lie in. And a suffi­ciently de­cep­tive map, like a suffi­ciently holey ma­te­rial, col­lapses into noise and dust.


Bibliography

I changed the species names in the stan­dard story about fire­flies be­cause I can never re­mem­ber which of Pho­turis and Phot­inus is which.

Fal­lis, Don and Lewis, Peter J., “Toward a For­mal Anal­y­sis of De­cep­tive Sig­nal­ing”

O’Con­nor, Cailin, Games in the Philos­o­phy of Biol­ogy, §5.5, “De­cep­tion”

Skyrms, Brian, Sig­nals: Evolu­tion, Learn­ing, and In­for­ma­tion, Ch. 6, “De­cep­tion”

Skyrms, Brian and Bar­rett, Jeffrey A., “Propo­si­tional Con­tent in Sig­nals”