Right, I agree that it doesn’t sound difficult from a web-development perspective, but I also think that only praising difficult-to-implement features would create the wrong incentives.
Zack_M_Davis
Karma: 2,694 (LW), 0 (AF)
Minimax Search and the Structure of Cognition!
or establish the case that agreement on those abstractions doesn’t force agreement on the object-level issues
Why would anyone think that agreement on meta-level abstractions (like the thing I was trying to say in “Where to Draw the Boundaries?”) would force agreement on object-level issues? That would be crazy! Object-level issues in the real world are really complicated: you’d need to spend a lot more wordcount just to make a case for any particular object-level claim—let alone reach agreement!
Wow, it’s neat that the LW 2 codebase gives you tools to move a derailed thread to its own post! Good job, whoever wrote that feature!
I’ve found /r/TheMotte (recently forked from /r/slatestarcodex) to be a good place to discuss politically-charged topics? (Again, also happy to talk privately sometime.)
Thanks, these are some great points on some of the costs of decoupling norms! (As you’ve observed, I’m generally pretty strongly in favor of decoupling norms, but policy debates should not appear one-sided.)
someone brings it up all the time
I would want to distinguish “brings it up all the time” in the sense of “this user posts about this topic when it’s not relevant” (which I agree is bad and warrants moderator action) versus the sense of “this user posts about this topic a lot, and not on other topics” (which I think is generally OK).
If someone is obsessively focused on their narrow special interest—let’s say, algebraic topology—and occasionally comments specifically when they happen to think of an application of algebraic topology to the forum topic, I think that’s fine, because people reading that particular thread get the benefit of a relevant algebraic topology application—even if looking at that user’s posting history leaves one with an unsettling sense of, “Wow, this person is creepily obsessed with their hobbyhorse.”
tries to twist other people’s posts towards a discussion of their thing
I agree that this would be bad, but I think it’s usually possible to distinguish “twist[ing] other people’s posts towards a discussion of their thing” from a genuinely relevant mention of the thing that couldn’t (or shouldn’t) be reasonably expected to derail the discussion?
In the present case, my great-great-grandparent comment notes that the list-of-koans format lends itself to readers contributing their own examples in the comments, and I tried to give two such examples (trying to mimic the æsthetic of the OP by continuing the numbered list and Alice/Bob/Charlie/&c. character name sequence), one of which related the theme of the OP to the main point of one of my recent posts.
In retrospect, maybe I should’ve thought more carefully about how to phrase the proposed example in a way that makes the connection to the OP more explicit/obvious? (Probably-better version: “A meaningful ‘Yes’ answer to the question ‘Is G an H?’ requires a definition of H such that the answer could be ‘No’.”)
It’s true that, while composing the great-great-grandparent, I was kind of hoping that some readers would click through the link and read my earlier post, which I worked really hard on and which I think is filling in a gap in “A Human’s Guide to Words” that I’ve seen people be confused about. But I don’t see how this can reasonably be construed as an attempt to derail the discussion? Like, I ordinarily wouldn’t expect a brief comment of the form “Great post! Here’s a couple more examples that occurred to me, personally” to receive any replies in the median case.
(Although unfortunately, it empirically looks like the discussion did, in fact, get derailed. I feel bad for Scott G. that we’re cluttering up his comment section like this, but I can’t think of anything I wish I had done differently other than wording the great-great grandparent more clearly, as mentioned in the paragraph-before-last. Given Vanessa’s reply, I felt justified in writing my counterreply … and here we are.)
It would be perfectly alright for moderators who didn’t want to drive away their visitors to ask this person to stop.
Agreed, the moderators are God and their will must be obeyed.
kick out someone who has a bad reputation that makes important posters unable to post on your website because they don’t want to associate with that person, even IF that person has good behavior
So, the dynamic you describe here definitely exists, but I actually think it’s a pretty serious problem for our collective sanity: if some truths happen to lie outside of Society’s Overton window, then systematic truthseekers (who want to collect all the truths, not just the majority of them that are safely within the Overton window) will find themselves on the wrong side of Respectability, and if people who care about being Respectable (and thereby having power in Society) can’t even talk to people outside the Overton window (not even agree with—just talk to, using, for example, a website), then that could have negative instrumental consequences in the form of people with power in Society making bad policy decisions on account of having inaccurate beliefs.
I want to write more about this in the future (albeit not on Less Wrong), but in the meantime, maybe see the immortal Scott Alexander’s “Kolmogorov Complicity And The Parable Of Lightning” for an expression of similar concerns:
Some other beliefs will be found to correlate heavily with lightning-heresy. Maybe atheists are more often lightning-heretics; maybe believers in global warming are too. The enemies of these groups will have a new cudgel to beat them with, “If you believers in global warming are so smart and scientific, how come so many of you believe in lightning, huh?” Even the savvy Kolmogorovs within the global warming community will be forced to admit that their theory just seems to attract uniquely crappy people. It won’t be very convincing. Any position correlated with being truth-seeking and intelligent will be always on the retreat, having to forever apologize that so many members of their movement screw up the lightning question so badly.
That’s understandable, but I hope it’s also understandable that I find it unpleasant that our standard Bayesian philosophy-of-language somehow got politicized (!?), such that my attempts to do correct epistemology are perceived as attacking people?!
Like, imagine an alternate universe where posts about the minimum description length principle were perceived as an attack on Christians (because atheists often argue that Occam’s razor implies that theories about God are unnecessarily complex), and therefore somewhat unseemly (because politics is the mind-killer, and criticizing a popular religion has inextricable political consequences).
I can see how it would be really annoying if someone on your favorite rationality forum wrote a post about minimum description length, if you knew that their work about MDL was partially derived from other work (on a separate website, under a pseudonym) about atheism, and you happened to think that Occam’s razor actually doesn’t favor atheism.
Or maybe that analogy is going to be perceived as unfair because we live in a subculture that pattern-matches religion as “the bad guys” and atheism as the “good guys”? (I could try to protest, “But, but, you could imagine as part of the thought experiment that maybe Occam’s razor really doesn’t favor atheism”, but maybe that wouldn’t be perceived as credible.)
Fine. We can do better. Imagine instead some crank racist psuedoscientist who, in the process of pursuing their blatantly ideologically-motiviated fake “science”, happens to get really interested in the statistics of the normal distribution, and writes a post on your favorite rationality forum about the ratio of areas in the right tails of normal distributions with different means.
I can see how that would be really annoying—maybe even threatening! Which might make it all the more gratifying if you can find a mistake in the racist bastard’s math: then you could call out the mistake in the comments and bask in moral victory as the OP gets downvoted to oblivion for the sin of bad math.
But if you can’t find a mistake—if, in fact, the post is on-topic for the forum and correct in the literal things that it literally says, then complaining about the author’s motive for being interested in the normal distribution doesn’t seem like an obviously positive contribution to the discourse?—even if you’re correct about the author’s motive. (Although, you might not be correct.)
Like, maybe statistics is part of the common interest of many causes, such that, as a matter of local validity, you should assess arguments about statistics on their own merits in the context that those arguments are presented, without worrying about how those arguments might or might not be applied in other contexts?
What, realistically, do you expect the atheist—or the racist, or me—to do? Am I supposed to just passively accept that all of my thoughts about epistemology are tainted and unfit for this forum, because I happen to be interested in applying epistemology to other topics (on a separate website, under a pseudonym)?
I think the grandparent is an on-topic response to the OP, relating the theme of the OP (about how if you don’t have negative feedback or “No”s, then that makes positive feedback or “Yes”es less significant) to both a hypothetical example about social network voting mechanisms, and, separately, to another philosophy topic (about the cognitive function of categories) that I’ve been thinking a lot about lately! That’s generally what happens when people comment on posts: they think about the post in the context of their own knowledge and their own priorities, and then write a comment explaining their actual thoughts!
Like, if you think the actual text of anything I write on this website is off-topic, or poorly-reasoned, or misleading on account of omitting relevant considerations, then please:
Downvote it, and
Leave a critical comment explaining what I got wrong (if you have time).
Those actions are unambiguously prosocial, because downvotes help other users decide what’s worth their time to read, and criticism of bad reasoning helps everyone reading get better at reasoning! But criticizing me because of what you know about my personal psychological motives for making otherwise-not-known-to-be-negative contributions seems … maybe less obviously prosocial?
Like, what happens if you apply this standard consistently? Did you know that Eliezer Yudkowsky’s writings that are ostensibly about human rationality, were actually mostly conceived in the context of his plans to build a superintelligence to literally take over the world?! (Although he denies it, of course.) That’s politics! Should we find it unpleasant that Yudkowsky always brings his hobbyhorse in, but in an “abstract” way that doesn’t allow discussing the actual object-level political question about whether he should rule the world?
Am I wrong here? Like, I see your concern! I really do! I’m sorry if we happen to be trapped in a zero-sum game whereby my attempts to think seriously in public about things I’m interested in ends up imposing negative externalities on you! But what, realistically, do you expect me to do? Happy to talk privately sometime if you’d like. (In a few weeks; I mostly want to focus on group theory and my dayjob for the rest of May.)
This is great! One subtle advantage of the list-of-koans format is that it provides a natural prompt for the reader to think up their own as an exercise.
-
Irene wants to believe the claim “G is an H.” H is a fuzzy category and the word “H” can be used in many ways depending on context. Irene finds she can make the sentence “G is an H” more probable by deliberately choosing a sufficiently broad definition of “H”, but only at the cost of making the word less useful.
-
Jessica posts something to a social-media platform that has “Like”s, but not downvotes. She doesn’t know whether no one saw her post, or if lots of people saw it but they all hated it.
-
(I continue to regret my slow reply turnaround time.)
But there are times when it’s not a dishonest rhetorical move to do this, right?
Right. In Scott’s example, the problem was using the “eargrayish” concept to imply (bad) inferences about size, but your example isn’t guilty of this.
However, it’s also worth emphasizing that the inferential work done by words and categories is often spread across many variables, including things that aren’t as easy to observe as the features that were used to perform the categorization. You can infer that “mice” have very similar genomes, even if you never actually sequence their DNA. Or if you lived before DNA had been discovered, you might guess that there exists some sort of molecular mechanism of heredity determining the similarities between members of a “species”, and you’d be right (whereas similar such guesses based on concepts like “eargrayishness” would probably be wrong).
Since it doesn’t seem to make sense to never use a word to point to a cluster in a “thin” subspace, what is your advice for when it’s ok to do this or accept others doing this?
Um, watch out for cases where the data clusters in the “thin” subspace, but doesn’t cluster in other dimensions that are actually relevant in the context that you’re using the word? (I wish I had a rigorous reduction of what “relevant in the context” means, but I don’t.)
As long as we’re talking about animal taxonomy (dolphins, mice, elephants, &c.), a concrete example of a mechanism that systematically produces this kind of distribution might be Batesian or Müllerian mimicry (or convergent evolution more generally, as with dolphins’ likeness to fish). If you’re working as a wildlife photographer and just want some cool snake photos, then a concept of “red-‘n’-yellow stripey snake” that you formed from observation (abstractly: you noticed a cluster in the subspace spanned by “snake colors” and “snake stripedness”) might be completely adequate for your purposes: as a photographer, you just don’t care whether or not there’s more structure to the distribution of snakes than what looks good in your pictures. On the other hand, if you actually have to handle the snakes, suddenly the difference between the harmless scarlet kingsnake and the poisonous coral snake (“red on yellow, kill a fellow; red on black, venom lack”) is very relevant and you want to be modeling them as separate species!
(continued from sister comment)
My reply to Wei is now up. (I finally looked at his four links and didn’t end up engaging with them, but I endorse Benquo’s comment on #4.)
I also left a brief reply to your comment about chronic fatigue syndrome, and a reply to your comment critiquing the paragraph about “poison.” I hope this helps clarify what I’m trying to communicate.
Unfortunately, I don’t think your participation here has been a net-positive for the value of the comments section, and (with some sadness) I have decided to add you to the “Banned Users” list in the moderation section of my account settings.
It’s not clear whether or not that important difference is supposed to imply to the reader that one is better then the other. Given that there seems to be a clear value judgement in the others, maybe it does here?
All three paragraphs starting with “There’s an important difference [...]” are trying to illustrate the distinction between choosing a model because it reflects value-relevant parts of reality (which I think is good), and choosing a model because of some non-reality-mapping consequences of the choice of model (which I think is generally bad).
words that are unnecessarily obscure (most people in society won’t understand what wasting cycles is about)
The primary audience of this post is longtime Less Wrong readers; as an author, I’m not concerned with trying to reach “most people in society” with this post. I expect Less Wrong readers to have trained up generalization instincts motivating the leap to thinking about AIs or minds-in-general even though this would seem weird or incomprehensible to the general public.
To those people who proofread and appeartly didn’t find an issue in that sentence, is it really necessary to mix all those different issues into a 6-line sentence?
It’s true that I tend to have a “dense” writing style (with lots of nested parentheticals and subordinate clauses), and that I should probably work on writing more simply in order to be easier to read. Sorry.
CFS and SEID are both cases where certain states correlate with each other Zacks post doesn’t help us at all to reason about whether we should prefer CFS or SEID as a term.
I’m definitely not claiming to have the “correct” answer to all terminological disputes. (As the post says, “Of course, there isn’t going to be a unique way to encode the knowledge into natural language.”)
Suppose, hypothetically, that it were discovered that there are actually two or more distinct etiologies causing cases that had historically been classified as “chronic fatigue syndrome”, and cases with different etiologies responded better to different treatments. In this hypothetical scenario, medical professionals would want to split what they had previously called “chronic fatigue syndrome” into two or more categories to reflect their new knowledge. I think someone who insisted that “chronic fatigue syndrome” was still a good category given the new discovery of separate etiologies would be making a mistake (with respect to the goals doctors have when they talk about diseases), even if the separate etiologies had similar symptoms (which is what motivated the CFS label in the first place).
In terms of the configuration space visual metaphor, we would say that while “chronic fatigue syndrome” is a single cluster in the “symptoms” subspace of Diseasespace, more variables than just symptoms are decision-relevant to doctors, and the CFS cluster doesn’t help them reason about those other variables.
Hi, Wei—thanks for commenting! (And sorry for the arguably somewhat delayed reply; it’s been a really tough week for me.)
can you give a concrete example that might come up on LW or otherwise have some relevance to us?
Is Slate Star Codex close enough? In his “Anti-Reactionary FAQ”, Scott Alexander writes—
Why use this made-up word [“demotism”] so often?
Suppose I wanted to argue that mice were larger than grizzly bears. I note that both mice and elephants are “eargreyish”, meaning grey animals with large ears. We note that eargreyish animals such as elephants are known to be extremely large. Therefore, eargreyish animals are larger than noneargreyish animals and mice are larger than grizzly bears.
As long as we can group two unlike things together using a made-up word that traps non-essential characteristics of each, we can prove any old thing.
This post is mostly just a longer, more detailed version (with some trivial math) of the point Scott is making in these three paragraphs: mice and elephants form a cluster if you project into the subspace spanned by “color” and “relative ear size”, but using a word to point to a cluster in such a “thin”, impoverished subspace is a dishonest rhetorical move when your interlocutors are trying to use language to mostly talk about the many other features of animals which don’t covary much with color and relative-ear-size. This is obvious in the case of mice and elephants, but Scott is arguing that a similar mistake is being made by reactionaries who classify Nazi Germany and the Soviet Union as “demotist”, and then argue that liberal democracies suffer from the same flaws on account of being “demotist.” Scott had previously dubbed this kind of argument the “noncentral fallacy” and analyzed how it motivates people to argue over category boundaries like “murder” or “theft.”
Downthread, you wrote—
My interest in terminological debates is usually not to discover new ideas but to try to prevent confusion (when readers are likely to infer something wrong from a name, e.g., because of different previous usage or because a compound term is defined to mean something that’s different from what one would reasonably infer from the combination of individual terms).
I agree that preventing confusion is the main reason to care about terminology; it only takes a moderate amount of good faith and philosophical sophistication for interlocutors to negotiate their way past terminology clashes (“I wouldn’t use that word because I think it conflates these-and-such things, but for the purposes of this conversation …” &c.) and make progress discussing actual ideas. But I wanted to have this post explaining in detail a particular thing that can go wrong when philosophical sophistication is lacking or applied selectively, which was mostly covered by Eliezer’s “A Human’s Guide to Words”, but of which I hadn’t seen the “which subspace to pay attention to / do clustering on” problem treated anywhere in such terms.
Sure, casual use of categories is convenient and pretty good for a lot of purposes. [...] Where precision matters, though, you’re better off using more words. Don’t try to cram so much inferential power into a categorization that’s not a good fit for the domain of predictions you’re making.
So, I actually don’t think “casual” vs. “precise” is a good characterization of the distinction I was trying to make in the grandparent! I’m saying that for “sparse”, tightly-clustered distributions in high-dimensional spaces, something like “essentialism” is actually doing really useful cognitive work, and using more words to describe more basic, lower-level (“precise”?) features doesn’t actually get you better performance—it’s not just about minimizing cognitive load.
A good example might be the recognition of accents. Which description is more useful, both for your own thinking, and for communicating your observations to others—
“She has a British accent”; or
At the level of consciousness, it’s much easier to correctly recognize accents than to characterize and articulate all the individual phoneme-level features that your brain is picking up on to make the categorization. Categories let you make inferences about hidden variables that you haven’t yet observed in a particular case, but which are known to correlate with features that you have observed. Once you hear the non-rhoticity in someone’s speech, your brain also knows how to anticipate how they’ll pronounce vowels that they haven’t yet said—and where the person grew up! I think this is a pretty impressive AI capability that shouldn’t be dismissed as “casual”!
You’re being bizarrely demanding, and I don’t understand why. Have I done something to offend you somehow? (If I have accidentally offended and there’s some way I could make amends, feel free to PM me.)
I agree that authors advocating an idea should provide examples. That’s why the OP does, in fact, provide some examples (about dolphins, abstract points in ℝ³, and job titles). I also have a couple other cached “in the wild” examples in mind that I intend to include in my reply to Wei (e.g., search for the word eargreyish in Scott Alexander’s “Anti-Reactionary FAQ”). But, as the grandparent mentions, Wei specifically asked if I had any thoughts on four of his comments (which I still haven’t read, incidentally). I can’t possibly have cached such thoughts in advance!
Writing good comments takes nontrivial time and mental energy and given that at least some Less Wrong readers probably have things like jobs (!) or possibly even families (?!), I really don’t think it’s reasonable to infer that someone is incapable of offering a satisfactory reply just because they haven’t replied within a couple days.
I had a really stressful day yesterday. I just got home today. After posting this comment, I want to make dinner and relax and read the new Greg Egan novel for a while. After that, I intend to spend some time writing blog comment replies—to Wei, to Dagon again, to someone on Reddit—and then maybe to some of your comments, if I still have time. (I also need to look up what I need to bring to my DMV appointment tomorrow.) Please be patient with me—although if you’re so dissatisfied by both the post, and my comments so far, then I fear my future comments are unlikely to be that much more to your liking, so it’s not clear why you should be so eager to see them be posted faster.
In conclusion, I’m sorry you didn’t like my blog post about the information theory of dolphins. Please feel free to downvote it if you haven’t already.
That’s also likely why WeiDei’s request to get practical examples went unanswered.
Alternative explanation: that comment was made on a Sunday afternoon in my timezone, I have a Monday-through-Friday dayjob that occupies a lot of my attention, and I wanted to set aside a larger block of time to read through the four comments (and surrounding context) Wei linked (1 2 3 4) and think carefully about them before composing a careful reply. (I spent my Sunday afternoon writing budget on my reply to dadadarren, which took a while because I had to study the “Ugly duckling theorem” Wikipedia page he linked.) In contrast, a reply like this one, or my reply to Dagon don’t require additional studying time to compose, which is why I can manage to type something like this now without being too late to my dayjob.
your post is detached from any empiricsm but about the search of essenses of words.
I don’t think this is a fair characterization of the post.
I need to go get dressed and catch a train now. I’ll ping you when my reply to Wei is up.
We agree that models are only better or worse for a purpose, but …
Ask whether this creature needs air. Ask how fast it swims. etc.
If there are systematic correlations between many particular creature-features like whether it needs air, how fast it swims, what it’s shaped like, what its genome is, &c., then it’s adaptive to have a short code for the conjunction of those many features that such creatures have in common.
Category isn’t identity, but the cognitive algorithm that makes people think category is identity actually performs pretty well when things are tightly-clustered in configuration space rather than evenly distributed, which actually seems to be the case for a lot of things! (E.g., while there are (or were) transitional forms between species related by evolutionary descent, it makes sense that we have separate words for cats and dogs rather than talking about individual creature properties of ear-shape, &c., because there aren’t any half-cats in our real world.)
Saying that the primary intention which which language is used isn’t to create some effect in the recipient of the language act
It’s notable to me that both of the passages from this post that you quoted in the great-grandparent comment were from the final section. Would your assessment of the post change if you pretend it had ended just before the Musashi quote, with the words “resulting in fallacies of compression”?
I was trying to create an effect in the recipients of the language act by riffing off Yudkowsky’s riff off Musashi in “Twelve Virtues of Rationality”, which I expected many readers to be familiar with (and which is the target of the hyperlink with the text “is quoted”). My prereaders seemed to get it, but it might have been the wrong choice if too many readers’ reactions were like yours.
Thanks for the pointer! I’ve played with word2vec and similar packages before, but had never thought to explore how those algorithms connect with the content of “A Human’s Guide to Words”.
Um, in context, this sounds to me like you’re arguing that by writing “Where to Draw the Boundaries?” and my secret (“secret”) blog, I’m trying to get people to accept harmful situations? Am I interpreting you correctly? If so, can you explain in detail what specific harm you think is being done?