Wait, this doesn’t seem right. Say 49% of people are good and truthful, 49% are evil and truthful, and 2% are evil liars. You meet a random person and are deciding whether to be friends with them. Apriori they’re about equally likely to be good or evil. You ask “are you good?” They say “yeah”. Now they are much more likely to be good than evil. So if the person is in fact an evil liar, their lie had the intended effect on you. It wasn’t “priced into the equilibrium” or anything.
The technical explanation is still correct in the narrow sense—the message can be interpreted as “I’m either good or an evil liar”, and it does increase the probability of “evil liar”. But at the same time it increases the probability of “good” relative to “evil” overall, and often that’s what matters.
Agreed. In this post, lying is reinterpreted as an honest signal that there are liars in our midst. The response to lying happens through a passive process of evolution.
In the human world, to accuse somebody of lying means not only that we’ve updated our probabilities on whether they‘re a liar, but is a signal to others that we should fight back, to the liar that they’ve been discovered, and to ourselves that we should protect ourselves against the threat. You could say we have an active cultural immune system against lying. “Lying” is a reference to deceptive human behavior that takes place within this context.
What would a collective of AI agents do? Hard to say. Maybe something akin to what we do, or maybe something entirely different due to its greater speed, intelligence, and different construction.
Under typical game-theoretic assumptions, we would assume all players to be strategic. In that context, it seems much more natural to suppose that all evil people would also be liars.
For me, the main point of the post is that you can’t simultaneously (1) buy the picture in which the meaning of a signal is what it probabilistically implies, and (2) have a sensible definition of “lying”. So when you say “2% are evil liars”, Zack’s response could be something like, “what do you mean by that?”—you’re already assuming that words mean things independent of what they signal, which is the assumption Zack is calling out here (on my reading).
Under typical game-theoretic assumptions, we would assume all players to be strategic. In that context, it seems much more natural to suppose that all evil people would also be liars.
Why? Maybe some evil people are ok with kicking puppies but not with lying—that’s part of their utility function. (If such differences in utility functions can’t exist, then there’s no such thing as “good” or “evil” anyway.)
Wait, this doesn’t seem right. Say 49% of people are good and truthful, 49% are evil and truthful, and 2% are evil liars. You meet a random person and are deciding whether to be friends with them. Apriori they’re about equally likely to be good or evil. You ask “are you good?” They say “yeah”. Now they are much more likely to be good than evil. So if the person is in fact an evil liar, their lie had the intended effect on you. It wasn’t “priced into the equilibrium” or anything.
The technical explanation is still correct in the narrow sense—the message can be interpreted as “I’m either good or an evil liar”, and it does increase the probability of “evil liar”. But at the same time it increases the probability of “good” relative to “evil” overall, and often that’s what matters.
Agreed. In this post, lying is reinterpreted as an honest signal that there are liars in our midst. The response to lying happens through a passive process of evolution.
In the human world, to accuse somebody of lying means not only that we’ve updated our probabilities on whether they‘re a liar, but is a signal to others that we should fight back, to the liar that they’ve been discovered, and to ourselves that we should protect ourselves against the threat. You could say we have an active cultural immune system against lying. “Lying” is a reference to deceptive human behavior that takes place within this context.
What would a collective of AI agents do? Hard to say. Maybe something akin to what we do, or maybe something entirely different due to its greater speed, intelligence, and different construction.
Under typical game-theoretic assumptions, we would assume all players to be strategic. In that context, it seems much more natural to suppose that all evil people would also be liars.
For me, the main point of the post is that you can’t simultaneously (1) buy the picture in which the meaning of a signal is what it probabilistically implies, and (2) have a sensible definition of “lying”. So when you say “2% are evil liars”, Zack’s response could be something like, “what do you mean by that?”—you’re already assuming that words mean things independent of what they signal, which is the assumption Zack is calling out here (on my reading).
Why? Maybe some evil people are ok with kicking puppies but not with lying—that’s part of their utility function. (If such differences in utility functions can’t exist, then there’s no such thing as “good” or “evil” anyway.)