I’d put you in a cluster with Lahwran and Said Achmiz on this, if that helps clarify the gestalt I’m trying to identify. By contrast, I’d say that the cluster Benito’s pointing to—which I’d say mainly publicly includes me, Jessicata, Zvi Mowshowitz, and Romeo, though obviously also Michael Vassar—is organized around the idea that if you honestly apply loose gestalt thinking, it can very efficiently and accurately identify structures in the world—all of which can ultimately be reconciled—but that this kind of honesty necessarily involves reprogramming ourselves to not be such huge tools, and most people, well, haven’t done that, so they end up having to pick sides between perception and logic.
Benquo
Karma: 10,415
My impression is that for the majority of my audience, my efforts to show how everything adds up to normality are redundant, and mostly they’re going by a vague feeling. Overall, it seems to me that there are people trying to do the kind of translational work Davis is asking for, but the community is not, as a whole, applying the sort of discernment that would demand such work. Whether or not this is the problem Davis is trying to identify, I’m worried enough about it that LessWrong has been getting less and less interesting to me as a community to engage with. You’re probably by far the person most worth reaching who isn’t already in my “faction,” such as it is, and Davis is one of the few others who seem to be trying to make sense of things at all.
The rhetorical structure of this post seems to imply a substantially different model of who your audience is, and what sort of appeals will work, than the model explicitly described. Since the question of which arguments will work on your intended audience is actually the whole point of your post, I think you should do an internal double crux on this issue, the results of which I expect will change your entire strategic sense of what’s going on in the Rationalist community the value of schisms, etc. I’m happy to spend plenty of time in live conversation on this if you’re interested and think seeking that sort of mutual understanding might be worth your time.
Explicitly, it seems like you’re saying that the way in which woo ideas are being brought into the community discourse is objectionable, because people are simply adopting new fragmentary beliefs that on the surface blatantly contradict much of the other things they believe, without doing the careful work of translation, synthesis, and rigorous grounding of concepts. You’re arguing that we should be alarmed by this trend, and engage in substantial retrenching. This is fundamentally an appeal to rational monism.
But rhetorically, you begin by offering a simple list of the names of things admitted into the conversation, and implicitly ask the reader to agree that this is objectionable before talking about method at all (and you don’t go into enough detail on the type of skepticism and rigor you’re suggesting for me to have a sense of whether I even agree.) The implied model here is that for most of your readers appeals to reason are futile, and you can at best get them to reject some ideas as foreign.
I think that the second model—the one you used to decide how to write the post—is a better representation of the current state of the Rationalist community than the first one. I don’t see much value in preserving or restoring the “integrity” of such a community (constituting in practice the cluster of people vaguely attracted to the Sequences, HPMoR, EA, the related in-person Bay Area community, CFAR, and the MIRI narrative). I see a lot of value in a version of this post clearly targeted to the remnant better-described by the first model. It would be nice if we could communicate about this in a way that didn’t hurt the feelings of the MOPs too badly, since they never wanted to hurt anyone.
it’s not obvious to me that humans are friendly if you scale them up
It seems like any AI built by multiple humans coordinating is going to reflect the optimization target of the coordination process building it, so we had better figure out how to make this so.
How is that relevant to the OP?
I think my basic worry is that if there’s not an active culture-setting drive against concern-trolling, then participating on this site will mean constant social pressure against this sort of thing. That means that if I try to do things like empathize with likely readers, take into account feedback, etc., I’ll either gradually become less clear in the direction this kind of concern trolling wants, or oppositionally pick fights to counteract that, or stop paying attention to LessWrong, or put up crude mental defenses that make me a little more obtuse in the direction of Said. Or some combination of those. I don’t think any of those are great options.
No one here since Eliezer seems to have had both the social power and the willingness to impose new—not quite standards, but social incentive gradients. The mod team has the power, I think.
Thanks for clarifying that you’re firmly in favor of at least tolerating this kind of speech. That is somewhat reassuring. But the culture is also determined by which things the mods are willing to ban for being wrong for the culture, and the implicit, connotative messages in the way you talk about conflicts as they come up. The generator of this kind of behavior is what I’m trying to have an argument with, as it seems to be to be basically embracing the drift towards pressure against the kind of clarity-creation that creates discomfort for people connected to conventional power and money. I recognize that asking for an active push in the other direction is a difficult request, but LessWrong’s mission is a difficult mission!
You got it right, 1 was the suggested change I was most disappointed by, as the weakening of rhetorical force also took away a substantive claim that I actually meant to be making: that GiveWell wasn’t actually doing utilitarian-consequentialist reasoning about opportunity cost, but was instead displaying a sort of stereotyped accumulation behavior. (I began Effective Altruism is Self-Recommending with a cute story about a toddler to try to gesture at a similar “this is stereotyped behavior, not necessarily a clever conscious scheme,” but made other choices that interfered with that.)
4 turned out to be important too, since (as I later added a quote and link referencing) “unfairness” literally was a stated motivation for GiveWell—but Zack didn’t know that at the time, and the draft didn’t make that clear, so it was reasonable to suggest the change.
The other changes basically seemed innocuous.
Who, specifically, is the enemy here, and what, specifically, is the evil thing they want?
It seems to me as though you’re describing motives as evil which I’d consider pretty relatable, so as far as I can tell, you’re calling me an enemy with evil motives. Are people like me (and Elizabeth’s cousin, and Elizabeth herself, both of whom are featured in examples) a special exception whom it’s nonsuspect to call evil, or is there some other reason why this is less suspect than the OP?
If your concern is that this is evidence that the OP is wrong (since it has conflict-theoretic components, which are mindkillers), it seems important to establish that there are important false object-level claims, not just things that make such mistakes likely. If you can’t do that, maybe change your mind about how much conflict theory introduces mistakes?
If you’re just arguing that laying out such models are likely to have bad consequences for readers, this is an important risk to track, but it’s also changing the subject from the question of whether the OP’s models do a good job explaining the data.
I’m much more interested in finding out what your model is after having tried to take those considerations into account, than I am in a point-by-point response.
Talk is cheap, especially when claiming not to hold opinions widely considered blameworthy.
Buchanan’s academic career (and therefore ability to get our attention) can easily depend on racists’ appetite for convenient arguments regardless of his personal preferences.
This is an excellent analytical account of the underlying dynamics. It also VERY strongly resembles the series of blame-deflections described in Part II Chapter VII of Atlas Shrugged (the train-in-the-tunnel part), where this sort of information suppression ultimately backfires on the nominal beneficiary.
I don’t quite draw the line at denotative vs enactive speech—command languages which are not themselves contested would fit into neither “conflict theory” nor “mistake theory.”
“War is the continuation of politics by other means” is a very different statement than its converse, that politics is a kind of war. Clausewitz is talking about states with specific, coherent policy goals, achieving those goals through military force, in a context where there’s comparatively little pretext of a shared discourse. This is very different from the kind of situation described in Rao where a war is being fought in the domain of ostensibly “civilian” signal processing.
Mechanism design is, to a large extent, a conflict theory, because it assumes conflicts of interest between different agents, and is determining what consequences should happen to different agents, e.g. in some cases “who we should be angry at” if that’s the best available implementation.
“Conflict theory” is specifically about the meaning of speech acts. This not the general question of conflicting interests. The question of conflict vs mistake theory is fundamentally, what are we doing when we talk? Are we fighting over the exact location of a contested border, or trying to refine our compression of information to better empower us to reason about things we care about?
For me, a conflict theorist is someone who thinks the main driver of disagreement is self-interest rather than honest mistakes.
I don’t see how to reconcile this with:
Conflict theorists treat politics as war. Different blocs with different interests are forever fighting to determine whether the State exists to enrich the Elites or to help the People.
I think this schema could benefit from a distinction between rules for internal and external consumption. For external consumption there’s some benefit to having (implied) policies for exceptions that are responsive to likely costs, both the internal costs of acting according to the rule in an emergency, and the external cost of having one’s expectations violated.
But for internal consumption, it makes more sense to, as Said Achmiz points out, just change the rule to a better one that gets the right answer in this case (and all the prior ones). I think people are confused by this in large part because they learned, from authoritarian systems, to rule themselves as though they were setting rules for external consumption or accountability, instead of reasoning about and doing what they want.
This leads to a weird tension where the same person is sternly setting rules for themselves (and threatening to shame themselves for rule violations), and trying to wiggle out of those same rules as though they were demands by a malevolent authority.
This is maybe half or more of what Robin Hanson wrote about back when it was still all on overcomingbias.com
Link seems broken
The “different masters” thing is a special case of the problem of accepting feedback (i.e. learning from approval/disapproval or reward/punishment) from approval functions in conflict with each other or your goals. Multiple humans trying to do the same or compatible things with you aren’t “different masters” in this sense, since the same logical-decision-theoretic perspective (with some noise) is instantiated on both.
But also, there’s all sorts of gathering data from others’ judgment that doesn’t fit the accountability/commitment paradigm.
I expect it would be quite useful both here and more generally for you to expand on your model of this.