Thanks, you are right and the thing I originally typed is wrong. I edited the comment.
Zack_M_Davis(Zack M. Davis)
Karma: 8,815
Thanks for the reply! (Strong-upvoted.) I’ve been emotionally trashed today and didn’t get anything done at my dayjob, which arguably means I shouldn’t be paying attention to Less Wrong, but I feel the need to type this now in the hopes of getting it off my mind so that I can do my dayjob tomorrow.
In your epistemic-status thread, you express sadness at “the fact that nobody’s read A Human’s Guide to Words or w/e”. But, with respect, you … don’t seem to be behaving as if you’ve read it? Specifically, entry #30 on the list of “37 Ways Words Can Be Wrong” is—I’ll quote it in full—
Your definition draws a boundary around things that don’t really belong together. You can claim, if you like, that you are defining the word “fish” to refer to salmon, guppies, sharks, dolphins, and trout, but not jellyfish or algae. You can claim, if you like, that this is merely a list, and there is no way a list can be “wrong”. Or you can stop playing nitwit games and admit that you made a mistake and that dolphins don’t belong on the fish list. (Where to Draw the Boundary?)
That is, in 2008, as part of “A Human’s Guide to Words”, Eliezer Yudkowsky explicitly uses this specific example of whether dolphins are fish, and characterizes the position that dolphins are fish as “playing nitwit games” (!). This didn’t seem particularly controversial at the time?
Then, thirteen years later, in the current year, you declare that “The definitional gynmastics required to believe that dolphins aren’t fish are staggering” (staggering!), and Yudkowsky retweets you. (In general, retweets are not necessarily endorsements—sometimes people just want to draw attention to some content without further comment or implied approval—but I’m inclined to read this instance as implying approval, partially because this doesn’t seem like the kind of thing someone would retweet for attention-without-approval, and partially because of the working relationship between you and Yudkowsky.)
But this is pretty strange, right? It would seem that sometime between 2008 and the current year, the rationalist “party line” (as observed in the public statements of SingInst/MIRI leadership) on whether dolphins are fish shifted from (my paraphrases) “No; despite the surface similarities, that categorization doesn’t carve reality at the joints; stop playing nitwit games” to “Yes, because of the surface similarities; those who contend otherwise are the ones playing nitwit games.” A complete 180° reversal, on this specific example! Why? What changed? Surely if “cognitively useful categories should carve reality at the joints, and dolphins being fish doesn’t do that” was good philosophy in 2008, it should still be good philosophy in 2021?
It would make sense if people’s opinions changed due to new arguments—if people’s opinions changed because of reasons. Indeed, Yudkowsky’s original “stop playing nitwit games” dismissal was sloppy and flawed, and I ended up having the occasion to elaborate on the specific senses in which dolphins both do, and do not, cluster with fish in my 2019 “Where to Draw the Boundaries?”
(Get it? ”… Boundaries?”, plural, in contrast to ”… Boundary?”, singular, because I’m talking about how you can legitimately have multiple different category systems depending on which subspace of configuration space is decision-relevant in a particular context.)
But when I look at the thing you posted and Yudkowsky retweeted (even if it was a shitpost, your epistemic-status followup thread still contends “but also y’all know i’m right”), it doesn’t look like the party line about dolphins changed because of reasons. You didn’t even acknowledge the reversal, despite explicitly lamenting (in the followup thread) that people haven’t read “A Human’s Guide to Words”.
Am I the only one creeped out by this? To illustrate why I’m freaked out—why I’ve been freaked out to a greater or lesser degree almost constantly for the past five years—imagine that in a fictional 2008, the Singularity Institute for Artificial Intelligence were at war with Eastasia for harboring the terrorist unFriendly AI Emmanuel GoldstAIn. It would make sense if, on 21 November 2014, Luke Muehlhauser were to announce:
We’re making some changes! First, we’re now going to be the Machine Intelligence Research Institute, or MIRI for short, instead of SingInst. And the reason for that is, the old name is no longer appropriate because we’re no longer unambiguously “for Artificial Intelligence” after we figured out that it’s probably going to destroy all value in our future lightcone. Second, a leaked pastebin revealed that Emmanuel GoldstAIn is actually being harbored by Eurasia, not Eastasia. Whoops! We’ll be winding down our war with Eastasia with the hope to be ready to declare war on Eurasia in time for our winter fundraiser. Third, we’re calling it “aligned” instead of “Friendly” AI now, and the reason for that is because Stuart Russell convinced us it’s a less goofy name.
That would make sense, because in this story, Luke is acknowledging the changes, and giving reasons for why it’s correct for the things to change. If Luke were to just say out of the blue on 21 November 2014 that the war with Eurasia is going well, without any indication that anything had changed for any reason, you would expect someone to notice.
Or, imagine if in 2014, Yudkowsky suddenly started saying the Copenhagen interpretation of quantum mechanics is correct, without acknowledging that anything had changed. That’s how weird this is.(Revised: Adele Lopez points out that this is wrong.)And on this classification-of-dolphins issue (specifically, literally, dolphins in particular), it seems like something has changed, and everyone is pretending not to have noticed. Why? What changed? I have my theory, but I could be biased—I want to hear yours! I want to hear yours in public. Do you have a cheerful price for this? I could go up to $2000 for a public reply.
Reply to Nate Soares on Dolphins
Did … did you save this table a long time ago?? Weak 3-votes have been gone since February 2020 for privacy reasons.
The karma-to-strong-vote-power-mapping can be found in the site’s open-sourced codebase, and Issa Rice’s alternative viewer has the list of actual user vote-powers.
Okay. I give up. I really liked your 11 May comment, and it made me optimistic that this conversation would lead somewhere new and interesting, but I’m not feeling optimistic about that anymore. (You probably aren’t, either.) This was fun, though: thanks! You’re very good at what you do!
I’m not sure exactly what distinction you’re appealing to
Thanks for asking! More detail: if you’re building a communication system to transmit information from one place to another, the signals/codewords you use are arbitrary in the sense that it doesn’t matter which you use as long as the reciever of the signals knows what they mean (the conditions under which they are sent).
(Well, the codeword lengths turn out to matter, but not the codewords themselves.)
If I’m publishing a weather report on my website about whether it’s “sunny” or “cloudy” today, it doesn’t matter whether I give it to you in JSON and English (
{"weather": "sunny"}/{"weather": "cloudy"}), or HTML and Spanish<h1>soleado</h1>/<h1>nublado</h1>: whichever one I choose, you can use it to make the same predictions about what you’ll experience when you go outside.In contrast, the choice of where I draw the boundary between what constitutes a “sunny” vs. a “cloudy” day does make a difference to your predictions. What a signal like
{"weather": "sunny"}means is different if I only send it when there’s not a single cloud in the sky, or if I only don’t send it when it’s completely overcast.The choice between Fred and George, or between Uluru and Ayers Rock, is analogous to the difference between
{"weather": "sunny"}and<h1>soleado</h1>; I consider the psychology of why a human might prefer the sound of one name over another to be out of scope of the hidden-Bayesian-structure-of-cognition thing I’ve been trying to talk about for the last thirty-eight months.being asked to use pronouns that you find inappropriate for the people they refer to, which you say amounts to asking you to lie
Not “lying” exactly, but rather that actually-existing-English-speakers naturally interpret she and he as conveying sex-category information, even if this seems like a weird or bad design, if we were somehow in the position of designing a natural language from scratch. (You could propose that it shouldn’t work that way, but the language is already “widely deployed”; the act of proposing a change doesn’t automatically update how 370,000,000 people interpret their native language.)
That’s why trans people care about being referred to with the correct pronoun in the first place! If the pronouns didn’t convey sex-category information and you could just choose one or the other arbitrarily with no difference in meaning, then there would be no reason to care, unless you had an æsthetic preference for the voiceless palato-alveolar fricative, or for words with two letters rather than three.
because in the particular case that sparked this particular discussion that didn’t happen
Yes, it did: did you see the “Related: Timeless Slate Star Codex / Astral Codex Ten piece” link at the bottom of the post? That’s why I commented. Do I feel a little guilty now that the OP author expressed dissatisfaction with the semi-derailed thread? A little! But, ultimately, I think it’s morally right to pay the cost of being a litle annoying from time to time to try to halt the spread of this bonkers meme that continues to exert influence to this day! (I’m guessing that in the counterfactual where ”… Not Man for the Categories” was never published, but Eukaryote still wrote the (excellent) post about trees, it wouldn’t have contained the advice to “Acknowledge that all of our categories are weird and a little arbitrary”, which is a very different and (I claim) much worse pedagogical emphasis from what was laid out in the Sequences.)
Eliezer Yudkowsky once wrote about dark side epistemology: wrong lessons about how to think that people only have an incentive to invent in order to force a conclusion that they can’t get on the merits. That’s what I think is happening here: if we actually had magical sex-change technology such that people who wanted to change sex could do so, then everyone else would use the corresponding language because it was straightforwardly true, and no one would have invented this deranged “gerrymandered categories are Actually Fine” argument in the first place!
neither rationality nor rationalism includes any obligation to optimize your words for the exact same thing as your thoughts.
But—don’t you want the language you speak to your friends to be the same as the language you use to organize your own thoughts? How can you accept a wall between the world you see, and the world you’re allowed to talk about? Doesn’t your soul die a little bit?
“person whose gender-related characteristics are, collectively, more like those of the average woman than like those of the average man”. (That definition is kinda-circular but in a way that does no actual harm
Tangential, but—what’s even the motivation for the circularity here? What’s wrong with “adult human female”?
Okay, I get it: we want to be trans-inclusive. But the clean way to do that is, “Adult human females, plus sufficiently successful male mimics thereof in the context that I’m using the word.” (See the explanation of mimickry and story about robot ducks in ”… Optimized for Deception.”) We can accomodate mimics in the domain that their mimickry is successful without trashing our ability to acknowledge the existence of the original thing!
ambiguities around the edges unless the person using that word tells you explicitly and in detail where they draw the boundaries. [...] exactly how “towns” shade into “villages” and “cities” for me [...] how “beautiful” shades into weaker terms like “pretty” or “sexy” or “elegant”, or how I categorize turquoise-ish colours
Sex actually seems significantly disanalogous to all of these examples, because municipality size, beauty, and color are all continuous (you know, like, for all ε, there exists a δ, such that if |color1 − color2| < δ, then |greenness(color1) − greenness(color2)| < ε), whereas sex is functionally binary: there’s a morph that procduces eggs, and a morph that produces sperm, but no continuum between them that produces a continuum of intermediate-sized gametes. The existence of various intersex conditions (which some authors call “disorders of sex development”), and cross-sex hormone replacement therapy, don’t substantially change this picture, because they’re the result of some specific thing “going wrong” with one of the two-and-only-two evolved developmental processes: you end up with various DSDs and females-on-masculinizing-HRT and males-on-feminizing-HRT each being their own tiny clusters in configuration space that you can draw a category boundary around. True, there is going to be a continuum of HRT dosages (or, say, the degree of how severe a polycystic ovary syndrome case is, if you want to count that), but my point is that the taxonicity means that we don’t need to specify edge cases in detail in order to stick a category boundary between the taxons: males-on-feminizing-HRT aren’t part of the female taxon! They just aren’t!
whether that person would prefer to be treated as male or female.
What does that even mean? (Also, isn’t that kind of sexist?) If Official Gender doesn’t matter once you get to know someone, why would people have this mysterious deep-seated preference to be “treated as male or female”?
Yeah, not-loving the way this thread turned out makes sense. Sorry. Please make sure to downvote any comments that you think are bad.
(A reply to gjm, split off from the comments on “There’s No Such Thing as a Tree”)
would you care either to argue for that principle or explain what weaker principle you are implicitly appealing to here?
No, not really. What actually happened here was, I was annoyed at being accused of not understanding something I’ve been obsessively explaining and re-explaining for multiple years—notice the uncanny resemblance between your comment (“If I and the people I need to talk to about pumpkins spend our days [...]”) and one of my replies from March 2019 (!) to you (!!) (“If I want to use sortable objects [...] If I’m running a factory [...]”)—so I fired off a snippy reply daring you to engage with my latest work. It wasn’t very principled. (It worked, though!)
Affirm your summary points 1–6.
Suppose someone’s legal name, given by their parents, is George, but they hate the way that sounds [...] Suppose our interlocutor actually thinks his name is Fred [...] the big rock in Australia named Uluru, but your interlocutor is an Englishman stuck in the past who insists that it is, and must always be, called Ayers Rock
But these examples are all about which proper name to use, which is not the philosophy-of-language subtopic I’ve been writing about at all! The communicative function of proper names (an arbitrary symbol/”pointer” to refer to some entity) is different from the cognitive function of categories (whereby different entities are considered instances of the “same kind” of thing)! Why did any of these examples seem relevant to you?
where everyone else seems to be drawing them
Yes, I wrote about this in “Schelling Categories, and Simple Membership Tests”. (Pet-topic object-level application: “Self-Identity Is a Schelling Point”.)
I no longer think that’s quite what’s going on, but I do think you’re objecting to more than your more nuanced analyses of category boundaries (e.g., in UCAOFD) justify
I think the actual pouncing algorithm is, “If someone favorably cites Scott Alexander’s ‘The Categories Were Made for Man, Not Man for the Categories’, then pounce.”
I don’t feel guilty about this because that post is utterly pants-on-fire mendacious. To Scott’s credit, the disinformation situation there is at least slightly less bad after he added the edit-note at the bottom after I spent Christmas Day 2019 yelling at him, but I think most readers will fail to notice how much the edit-note undermines the grand moral of the post: if “[i]n most cases plausible definitions will be limited to a few possibilities suggested by the territory” (as the edit-note finally admits), then it’s not true that one “ought to accept an unexpected man or two deep inside the conceptual boundaries of what would normally be considered female [...] There’s no rule of rationality saying that I shouldn’t” (as the main text claims). There are rules!
the point is that all these things require some sort of notion of distance, size, etc., in concept-space.
I agree. Did … did you read “Unnatural Categories Are Optimized for Deception”? The post says this very explicitly in quite a lot of detail with specific numerical examples! (Ctrl-F for “metric”.)
If you’re going to condescend to me like this, I think I deserve an answer: did you read the post, yes or no? I know, it’s kind of long (just under 10,000 words). But … if you’re going to put in the effort to write 500 words allegedly disproving what I “keep saying”, isn’t it worth … actually reading what I say?
Or I’m speaking a slightly different dialect of English from you?? As a point of terminology, I think “fuzzy” is a better word than “arbitrary” for this kind of situation, where I agree that, as a human having a casual conversation, my response to “Is a pumpkin a fruit?” is usually going to be something like “Whatever; if it matters in context, I’ll ask for more specifics”, but as a philosopher of science, I claim that there definite mathematical laws governing the relationship between what communication signals are sent, and what probabilistic inferences a receiver can infer, and the laws permit things like soft k-means clustering, where given some set of data points representing data about plants, the algorithm could say that this-and-such plant has a membership coefficient of 0.34 in the “shrub” cluster and 0.66 in the “tree” cluster, and there would be nothing arbitrary about those numbers as the definite, precise result of what happens when you run this particular clustering algorithm against that particular data. (But the number 0.34 in this blog comment is arbitrary, because I made it up for concreteness while trying to explain what fuzzy clustering is; there’s no reason I couldn’t have chosen a different coefficient.)
But then when I actually look up “arbitrary” and “fuzzy” on Wiktionary, it seems common usage is not unequivocally on my side: your usage of arbitrary fits with the first part of definition 1 (“Based on individual discretion or judgment”), whereas my usage is centered on the second part of definition 1 (“not based on any objective distinction, perhaps even made at random”), with influence from the mathematician’s usage, definition 3 (“Any, out of all that are possible”). And the meaning of fuzzy I want barely even makes the list as a technical reference (“Employing or relating to fuzzy logic”) …
Agreed on weirdness.
On the specific example of trees, John Wentworth recently pointed out that neural networks tend to learn a “tree” concept: a small, local change to the network can add or remove trees from generated images. That kind of correspondence between human and unsupervised (!) machine-learning model concepts is the kind of thing I’d expect to happen if trees “actually exist”, rather than trees being weird and a little arbitrary. (Where things are closer to “actually existing” rather than being arbitrary when different humans and other AI architectures end up converging on the same concept in order to compress their predictions.)
(Now I’m wondering if there’s some sort of fruitful analogy to be made between convergence of tree concepts in different maps, and convergent evolution in the territory; in some sense, the fact that evolution keeps rediscovering the tree strategy makes them less “arbitrary” than if trees had only been “invented once” and all descended from the same ur-tree …)
I expected you to realize how wrong everything you said was
What parts, specifically, are wrong? What is the evidence that shows that those parts are wrong? Please tell me! If I’m wrong about everything, I want to know!
Acknowledge that all of our categories are weird and a little arbitrary
That is not the moral! The moral is that the cluster-structure of similarities induced by phylogenetic relatedness exists in a different subspace from the cluster-structure of similarities induced by convergent evolution! (Where the math jargon “subspace” serves as a precise formalization of the idea that things can be similar in some aspects (“dimensions”) while simultaneously being different in other aspects.) This shouldn’t actually be surprising if you think about what the phrase “convergent evolution” means!
For more on the relevant AI/philosophy-of-language issues, see “Where to Draw the Boundaries?” and “Unnatural Categories Are Optimized for Deception”.
Sexual Dimorphism in Yudkowsky’s Sequences, in Relation to My Gender Problems
actual trans people, or perverts willing to pretend to be trans if it allows them to sneak into female toilets
It gets worse: if the dominant root cause of late-onset gender dysphoria in males is actually a paraphilic sexual orientation, this is a false dichotomy! (It’s not “pretending” if you sincerely believe it.)
So, I started writing an impassioned reply to this (draft got to 850 words), but I’ve been trying to keep my culture war efforts off this website (except for the Bayesian philosophy-of-language sub-campaign that’s genuinely on-topic), so I probably shouldn’t take the bait. (If nothing else, it’s not a good use of my time when I have lots of other things to write for my topic-specific blog.)
If I can briefly say one thing without getting dragged into a larger fight, I would like to note that aggressively encouraging people to consider whether they might be trans is potentially harmful if the popular theory of what “trans” is, is actually false; even if you’re a liberal who wants people to have the freedom to decide how to live their lives unencumbered by oppressive traditions, people might make worse decisions in an environment full of ideologically-fueled misinformation. (I consider trans activism to have been extremely harmful to me and people like me on this account.)
The effective altruist case for regime change??
Well, yes. I’ve got Something to Protect.