I think this was excellent work that no one (rounding down) appreciated at the time because I sacrificed readability by optimizing for comprehensiveness. If it helps, I have now composed a Twitter-optimized summary:
Time for a Twitter-optimized capsule 𧔠of my 2021 philosophy of language thesis about why choosing bad definitions is relevantly similar to lying! If you wouldnât lie, you also shouldnât say, âitâs not lying; Iâm just defining words in a way that I prefer.â 1â24
Some people say: the borders of a category are like the borders of a countryâthey have consequences, but thereâs no sense in which some possible borders can be objectively worse than others. 2â24
But category âbordersâ or âboundariesâ are just a visual metaphor corresponding to a kind of probabilistic model. Editing the âboundaryâ means editing the modelâs predictions. There is a sense in which some models can be objectively worse than others! 3â24
Imagine having to sort a bunch of blue egg-shaped things (which contain vanadium) & red cubes (that donât). Technically, you donât actually need separate âblue eggâ & âred cubeâ categories. You could just build up a joint probability table over all objects and query that. 4â24
But thatâs unwieldy. Thinking about âblue eggâ & âred cubeâ as separate categories and computing the properties of an object conditional on its category is much more efficient. 5â24
âComputing the properties of an object conditional on its categoryâ can be visualized as category âboundariesâ in a picture. 6â24
But the picture is an illustration of the math; you canât change the picture without changing the math. Itâs not like national borders at all. The U.S. purchasing Alaska (non-contiguous with the 48 states) wasnât about editing a probabilistic model. 7â24
In itself, this doesnât yet explain whatâs wrong with âsquigglyâ, âgerrymanderedâ categories. You can still make predictions with squiggly categories. 8â24
But if approximately-correct answers are at all more useful than totally-wrong answers, squiggly categories are mathematically just worse (going by the mean squared error). If your âblue eggsâ category contains some red cubes, youâll drill for vanadium where there isnât any. 9â24
The best categories are subjective in the sense that they depend on what youâre trying to predict, but thatâs not the same thing as the category boundary itself being subjective. Given what you want to predict, the model (and thus the âboundaryâ) is determined by the data. 10â24
Some people say: okay, but what if I really do have a preference for using a particular squiggly boundary, intrinsically, not in a way that arises from desired predictions and the data distribution? Thatâs just how my utility function is! Whatâs irrational about that? 11â24
Letâs interrogate this. What would it mean, to have such an exotic utility function? There is a trivial sense in which any pattern of behavior, however bizarre, could be rationalized in terms of preferring to take the actions that I do in the situation that I face. 12â24
But a theory that explains everything explains nothing. The explanatory value of the âutility functionâ formalism isnât that it can justify anything given a choice of âutilityâ, but in the constraints it articulates on coherent behaviors (given, yes, a choice of âutilityâ). 13â24
If your gambling behavior violates the independence axiom with respect to money, that doesnât automatically make you irrational, but it does mean that youâre acting as if you care about something else besides moneyâthat youâll sacrifice some money for that something else. 14â24
Similarly, if your communication signals arenât explainable in terms of conveying probabilistic predictions, that does imply that you care about something else than conveying probabilistic predictionsâthat youâll sacrifice clarity (of predictions) for that something else. 15â24
But what might that something else be, concretely? Itâs hard to see where a completely arbitrary, hardwired, âjust becauseâ preference for using a particular category boundary would come from! Why would that be a thing? Why?? 16â24
A much more plausible reason to sacrifice clarity of predictions is because you donât want other agents to make accurate predictions. (Because if those others had better models, theyâd make decisions that harm your interests.) Thatâs deception. 17â24
Thereâs no functional difference between saying âI reserve the right to lie p% of the time about whether something belongs to a categoryâ and adopting a new category system that misclassifies p% of things. The inputâoutput relations are the same. 18â24
A related reason for unnatural categories: itâs tempting to âwireheadâ by choosing a map that looks good, instead of the map that reflects the territory (which might be unpleasant to look at). Thatâs self-deception. 19â24
If I want to believe Iâm pretty & funny, it might be tempting to redefine âprettyâ & âfunnyâ such that they include me. But that would just be fooling myself; it doesnât actually work for making me pretty & funny (with respect to the usual meanings). 20â24
Sometimes things resemble another in some but not all aspects. This is mimickry. Itâs deceptive if the point is for another agent to treat the mimic as the original against the agentâs interestsâbut itâs not deceptive if the agent really doesnât care about the difference. 21â24
If agents sharing a language disagree about which aspects âcountâ, theyâll fight over the definitions of words: animal advocates would prefer if plant-based meat substitutes counted as ârealâ meat, to make it hard for carnivores to insist on the dead-animal kind. 22â24
Philosophy itself canât determine which definition is right (which depends on the empirical merits), but philosophy does clarify whatâs happening in this kind of conflictâthat departing from the empirical merits extracts a cost in the form of worse predictions. 23â24
Thereâs no functional difference between saying âI reserve the right to lie p% of the time about whether something belongs to a categoryâ and adopting a new category system that misclassifies p% of things. The inputâoutput relations are the same.
If Iâm honest about the boundaries of my new category system, how is this deceptive? You know that my âbleggâ category includes a small number of things that you would prefer to define as rubes, so when I tell you something is a blegg, you know that means it has an X% chance of being a mutually-agreed blegg and a 100-X% chance of being (in your eyes) a rube with properties that I consider definitive of a blegg. From your perspective, I may be concealing some relevant information, but Iâm doing so openly and allowing you to draw correct probabilistic inferences.
Thatâs not the same as âI reserve the right to lie p% of the time about whether something belongs to a categoryâ; itâs the same as âI will consistently âlieâ about which of these categories some things belong to, because those things have properties that are not part of the usual definitions of the categories but which I consider important, namely <x> or <y>, and I will consistently say that things with <x> belong in category A and things with <y> belong in category Bâ. Which would be a weird way to put it, because Iâm not actually lying if the meaning of my words is clear (albeit not informative in exactly the way you would prefer) and I am neither deceiving you nor intending to deceive you.
Excellent question, thanks for commenting! (And for your patience.) The part of the original post that that Tweet is summarizing are the paragraphs after âSuppose Iâm selling you some number of gold and silver bars [...]â. As you observe, whether itâs âlyingâ to use a category label depends on what the label is construed to âcanonicallyâ mean. The idea here is that, as far as signal processing goes, thereâs an isomorphism between âlying p% of the timeâ with respect to the maximally-informative categories, and choosing different categories that conceal information. So if the new categories arenât deceptive because the receiver knows about them, is lying therefore not deceptive if the speaker already has it âpriced inâ that the sender lies this-and-such proportion of the time? I discuss this problem further in âMaybe Lying Canât Exist?!â and âComment on âDeception is Cooperationââ.
I think this was excellent work that no one (rounding down) appreciated at the time because I sacrificed readability by optimizing for comprehensiveness. If it helps, I have now composed a Twitter-optimized summary:
If Iâm honest about the boundaries of my new category system, how is this deceptive? You know that my âbleggâ category includes a small number of things that you would prefer to define as rubes, so when I tell you something is a blegg, you know that means it has an X% chance of being a mutually-agreed blegg and a 100-X% chance of being (in your eyes) a rube with properties that I consider definitive of a blegg. From your perspective, I may be concealing some relevant information, but Iâm doing so openly and allowing you to draw correct probabilistic inferences.
Thatâs not the same as âI reserve the right to lie p% of the time about whether something belongs to a categoryâ; itâs the same as âI will consistently âlieâ about which of these categories some things belong to, because those things have properties that are not part of the usual definitions of the categories but which I consider important, namely <x> or <y>, and I will consistently say that things with <x> belong in category A and things with <y> belong in category Bâ. Which would be a weird way to put it, because Iâm not actually lying if the meaning of my words is clear (albeit not informative in exactly the way you would prefer) and I am neither deceiving you nor intending to deceive you.
Excellent question, thanks for commenting! (And for your patience.) The part of the original post that that Tweet is summarizing are the paragraphs after âSuppose Iâm selling you some number of gold and silver bars [...]â. As you observe, whether itâs âlyingâ to use a category label depends on what the label is construed to âcanonicallyâ mean. The idea here is that, as far as signal processing goes, thereâs an isomorphism between âlying p% of the timeâ with respect to the maximally-informative categories, and choosing different categories that conceal information. So if the new categories arenât deceptive because the receiver knows about them, is lying therefore not deceptive if the speaker already has it âpriced inâ that the sender lies this-and-such proportion of the time? I discuss this problem further in âMaybe Lying Canât Exist?!â and âComment on âDeception is Cooperationââ.