The impossible problem of due process

[mingyuan]

I wrote this entire post in February of 2023, during the fallout from the TIME article. I didn’t post it at the time for multiple reasons:

• because I had no desire to get involved in all that nonsense

• because I was horribly burned out from my own community conflict investigation and couldn’t stand the thought of engaging with people online

• because I generally think it’s bad to post on the internet out of frustration or outrage

But after sitting on it for a full year, I still think it’s worth posting, so here it is. The only edits I have made since February 16th, 2023, were to add a couple interstitial sentences for clarity, and change ‘recent articles’ to ‘articles from February 2023’. So, it’s not intended to be commenting on anything more recent than that.

I am precommitting to not engaging with any comments, because I am mostly offline and I think that is good. I probably won’t even look at this post again for several weeks. Do what you will. Here is the post:

---

Note: I am erring on the side of not naming any names in this article. There is one exception for the sake of clarity.

In my time overseeing the global rationalist community, living in the Bay community, and just generally being a person, I’ve seen a lot of people face up to complicated conflicts.

People often get really mad at each other for mishandling these cases, and will sometimes publicly point to these failures as reasons to condemn a person or group. However, I challenge you to point to a single entity in the world that has figured out a process for handling non-criminal misconduct that you would be happy with no matter whether you were the aggrieved or the accused party. Maybe such a thing exists, but if so I have not heard of it.

This post is a survey of the different ways that people try to resolve community conflicts, and the ways that each of them fail.

Committees/​panels

In cases of major conflict or disagreement, it often seems like the right thing to do to convene a panel of impartial judges and have them hear all the evidence. I personally know of at least seven specific cases of this happening in the rationalist community. Here are some of the problems with this approach.

Investigations eat up hundreds of person-hours

The case I’m most familiar with has been investigated four different times, by different people and from different angles. Five separate reports have been written. At time of writing the situation has dragged out for three full years, and it’s consumed over 100 hours of my time alone, and who knows how much time for the other like 30 people involved.

You might think “holy shit, at that point who even cares, this is obviously not worth all those precious life hours that those 30 people will never get back, just ban the guy.” I’m inclined to agree, but unfortunately:

Panels generally don’t have much real ability to enforce things

If the members of your community don’t agree with your decision to ban someone, you can’t force them to abide by your decision.

Here are the actions available to you:

• Announce your decision to everyone in the community

• Ban the person from spaces that you personally have control over, which may include your home, events you are organizing, and online spaces like Discord servers, Google groups, etc.

• Make recommendations for the behavior of other people and institutions

• Apply vague social pressure in the hope of making people follow your recommendations

Here are things you cannot do:

• Make people stop being friends with the person

• Make the person stop holding events in their own home or in public

Panels act like they are courts of law

In a court of law, you are presumed innocent unless and until you can definitively be proven guilty of a specific crime. But this is far too strict a standard to use in community dispute — well-kept gardens die by pacifism, and you shouldn’t have to keep someone in your community if they’re causing problems, even if you can’t find definitive proof that they did some particular thing that everyone would agree is really bad.

On the opposite extreme, in #MeToo you are presumed guilty as soon as there’s an allegation against you. The intent is to counteract the perceived phenomenon of assault allegations being dismissed by default, but obviously believing 100% of allegations uncritically would go too far in the other direction. In 2014, Scott estimated that “greater than 0.3% of men get falsely accused of rape sometime in their lives, and the most likely number is probably around 3%.”

I think the best you can do is to make a holistic judgment, not relying on any one piece of evidence. You should be able to pass judgment on someone if they’re just generally agreed to be terrible by many people, even if they haven’t done any one provable thing that is really egregious.

But panelists often feel like they have to adhere to a really high standard of evidence, because:

Panels often lack a secure sense of legitimacy

If the community disagrees with your ruling, they may not only refuse to adhere to it, but also decide that they no longer trust you. In the worst case, they may decide you need to be removed from power or even socially destroyed. I’ve seen this rip communities apart. Angry constituents threaten panelists’ reputations and livelihoods, and the reputations of their friends and allies. People may divide into factions that never forgive each other.

With this hanging over you, it’s hard to not take it into account when making your ruling.

Favorable rulings can lend legitimacy to bad actors

If the panel ends up officially condoning the behavior of a bad actor — something they are likely to do even if they personally think the person is terrible, if they believe they need standards for evidence and conviction comparable to those in an actual court of law — that gives the bad actor legitimacy, and can allow them to do more damage.

Extended investigations are extremely stressful for all parties involved

More than 50% of the reason I stepped down as global organizer was that I became entangled in all these horrible conflicts with very angry people on all sides.

The worst conflict dragged out for about 8 months. I was involved first as an investigator, then, after I passed my judgment and the accused parties took issue with my decision and my process, I became the subject of an investigation where the original accused party accused me of unfair conduct, abuse of power, and unfitness for leadership. They took their accusations both to the person they believed to be my boss, and to the people I oversaw, trying to destroy those people’s trust in me. I received dozens or sometimes hundreds of messages a day from them, and was accused of stonewalling and dishonestly refusing to respond when I had really legitimate excuses like “my sister is in the hospital with internal bleeding”, or “I’m on a transatlantic flight and don’t have internet” or “it is literally the middle of the night where I am” or “this has been going on for four weeks and I want to live my life”.

At one of the four heights of the conflict, all I did was cry, complain to anyone who would listen, drink alcohol, and then cry more. None of this was worth it. It burned me out horribly, burned out at least two other organizers, and made me and dozens of other people miserable.

---

After the mishandling of the Brent Dill case (details below), one of the three panelists essentially went into hiding, avoiding all social events for months.

---

The aggrieved parties are often dredging up trauma and sometimes become too exhausted to continue.

There is often no way to find the objective truth

Much of the time, it just comes down to one person’s word against another, and you often have no way of knowing who to trust.

• You can’t assume that everyone is engaging in the investigation in good faith

  • The accused party has incentive to defend themselves whether or not they actually did the thing.

  • It’s possible that the aggrieved party is lying.

• Remembering specific details of events that you have no record of is really hard

  • Your brain naturally assembles events into a narrative and will interpret or even alter memories to conform to that narrative.

• People can legitimately and in good faith interpret the same events very differently

  • In one case I’m familiar with, there was a dispute as to whether one party had attempted to blackmail the other, in a private verbal conversation where only the two parties to the dispute were present. They actually ended up publicly agreeing on approximately the words that were said, but the complainant felt that the tone implied a threat, while the other party claimed that no threat was intended. These two versions of events do not contradict each other.

---

Other approaches

Throwing out due process is also problematic

I have had two friends thrown out of the community without due process, and it was absolutely awful for them. For one of them, I was never told enough about the situation to make my own judgment, but the other one was thrown out basically just because a couple powerful people didn’t like his vibes.

I’ve also learned: It is easier to remove good people from your community than to remove people who are aggressive, manipulative, narcissistic, or psychopathic, because the former will remain reasonable throughout the process, while the latter will fight dirty.

Codes of conduct can’t save you

Sometimes people suggest that maybe you could avoid all this litigation if you just make a really good and clear code of conduct and then throw people out if they violate it. That would be nice, but it doesn’t work in practice for rationalist communities, mostly because:

You can’t impose strictures unilaterally /​ without the consent of the governed

When joining a workplace, a university, or a club sport, you are opting into a code of conduct, including the part where there’s an entity that enforces that code of conduct on you. It may not be explicitly on your mind, but at some level you’re aware that that’s part of the package you’re choosing to sign up for.

Workplaces can have codes of conduct, because they are (often) neatly circumscribed social environments, and have a clear purpose. The same is often true of recreational sports. It is not true of the rationality community.

At a local meetup group, where the organizer is widely liked and recognized as the group leader, the leader may have the power to exclude people from their events – especially if the the meetings are held at a private location like the organizer’s house. But the rationalist community also encompasses various workplaces and group houses, which have various amounts of power to exclude people.

The lack of a single institution makes the problem more difficult, but it’s also not something that can be easily changed (nor do I think it should be changed).

Introducing codes of conduct creates weird dynamics

The most common response I get from organizers when I bring up codes of conduct is: “It’s true that we don’t want women to be driven off by a bunch of awkward men asking them out, but if we make everyone read a document that says ‘Don’t ask a woman out the first time you meet her’, then we’ll immediately give the impression that we have a problem with men awkwardly asking women out too much — which will put women off anyway.”

This is a good point and I don’t know how to get around it, so I generally just don’t recommend formal codes of conduct.

It’s difficult to write rules that will actually lead to the thing you want

As a (maybe weak) example, most people are assuming some implicit code of conduct in everyday interactions, like “Don’t be a sexual predator”, “Don’t be a dickhead”, and “Don’t ruin other people’s lives.” But when you try to litigate these things, you run into problems like:

Don’t be a sexual predator → What constitutes predation? What if I thought it was consensual and they didn’t? What if it was technically statutory rape but the younger party was 17 and enthusiastically consented and has never regretted it? What if I didn’t do anything that looks wrong if you go strictly by the book, but I nonetheless traumatized a bunch of people and feel no remorse about it?

Don’t be a dickhead → What if 90% of people think I’m dickhead but I can find four witnesses who really like me? What if I provide a lot of value to your community; at what point am I so valuable that I can get away with being a dickhead? What if nonstandard social interactions are part of the way I provide value?

Don’t ruin other people’s lives → I didn’t ruin their lives, they’re just oversensitive. Also, throwing me out of the community would ruin my life, therefore, you can’t do it.

Going to the authorities

In multiple articles from February 2023, it’s suggested that a responsible course of action in sexual misconduct cases would involve encouraging women to report their experiences to the police. They also mention Title IX as an example of what we should be aspiring to. I think both of these are things that just sound nice as soundbites, but are actually not good suggestions.

I’m including this section mostly in response to those articles, but it will probably be obvious that I don’t actually take these suggestions seriously.

Title IX

As a student at a US university who has experienced sexual harassment or assault, you have recourse to the university’s Title IX Coordinator. If you choose to report, the university’s Title IX Hearing Panel will investigate the case by talking to you, the accused, and any other relevant parties such as witnesses.

If this sounds like my description of panels above, that’s because it is pretty much the same thing. It has all the pitfalls of panels, with the additional downsides that the people are more likely to be strangers to you and are less likely to be aligned with your interests. (For one thing, the college has a vested interest in making sure you do not report sexual assault to the police.)

I had several friends in college go to the school authorities after being sexually assaulted. I do not think any of them were happy with either the experience or the outcomes.

I remember reading an article (though I can’t find it; it was nine years ago) about one of my friend’s cases, where the assailant’s father came in and said that the whole thing was preposterous and my friend was just a jilted lover who wanted attention. I don’t think an ideal process would open complainants up to public defamation by the fathers of their assailants.

Other prominent incidents that happened at my school:

• A student was expelled after being accused of sexual assault, and then sued the school for it

• Someone anonymously posted a list on Tumblr of campus rapists, and everyone was very upset

The police

It’s confusing to me that this is even a suggestion, because I frankly don’t see what the police are expected to do in most cases. If a person has been sexually assaulted in a way that left them with both injuries and the perpetrator’s DNA on them, and they want to press charges that might result in a criminal sentencing, then that seems like the kind of thing the police are equipped to handle. Otherwise, I don’t see how going to the police does much good. I couldn’t find reliable statistics (I found many but they were highly varied), but the general vibe seems to be that sexual assault cases are rarely prosecuted.

Maybe the idea is that if the police receive multiple accusations against a person, they will eventually take them seriously, and maybe that person will become a registered sex offender or something? I don’t really know how this works but in any case it doesn’t seem like an obviously good way to resolve disputes within a community.

This suggestion is also confusing because there’s a lot of other messaging that the police are not to be trusted, so like, why should we go to them in this specific scenario?

HR departments

HR departments exist to serve the company, not the employees, and so they are, in general, morally corrupt.

---

Case study

Let’s take a look at the case of Brent Dill. This is the only name I will be naming in this post, because I thought that obfuscating it would do little good and just be unnecessarily confusing.

I’m including this case study because I think a concrete example is a lot more helpful than just saying “Look, we’ve tried justice panels and they didn’t work as intended, take my word for it.” It’s also a case where most of the important details are already public, and where the case is widely considered to be closed.

Relevant reading:

• Mittens Cautious: Warnings

• CFAR: ACDC Update and Apology

• CFAR: CFAR​​’s Mistakes Regarding Brent

• Thing Of Things: Whisper Networks, Callout Posts, and Expulsion: Three Imperfect Ways of Dealing With Abuse

Whisper network

When I arrived in the Bay in June of 2017, fresh out of college, Brent was still in good standing in the community. I knew many people who were friends with him or considered themselves part of his circle.

However, even then, there were plenty of people who found Brent to be toxic and dangerous — it was not a secret or even a controversial statement that Brent had a lot of darkness in him; I’d say that pro-Brent and anti-Brent people mostly just differed in how they assessed the expected value of interactions with him.

As I was a young woman and a new member of the community, multiple people were quick to tell me to stay away from Brent; after meeting him I immediately agreed with that assessment and went on to join in warning other people about him.

CFAR panel

In 2017, CFAR convened a panel of three people (only one of whom was a CFAR employee) to hear community disputes. This panel was called the Alumni Community Dispute Council, or ACDC for short.

In January of 2018, one of Brent’s ex partners came to ACDC with allegations of a pattern of abuse by Brent. ACDC investigated this case for many hours, and in April, they privately recommended to CFAR that no action be taken and that Brent remain a community member in good standing.

I knew one of the three ACDC panelists well at the time, and that person did not agree with the conclusion that Brent should remain in the community. They were the first to warn me and many other young women to stay away from Brent, and I had previously seen them try pretty hard to block Brent from attending a private CFAR event. The fact that this person ended up party to a pro-Brent ruling indicates that something in the process went very wrong.

Since it is now widely agreed that kicking Brent out of the community was the correct course of action, I think it’s reasonable to ask how this ruling could have happened.

I think the ACDC panelists were well-intentioned, but as CFAR’s Executive Director put it in CFAR​​’s Mistakes Regarding Brent, they were “in over their heads.” Panels may be equipped to hear cases of clear-cut assault, but it’s much more difficult to make a ruling on a potentially abusive but nominally consensual relationship between two adults.

In addition, this case had some factors that made it difficult beyond just the inherent complexity of figuring out interpersonal conflicts between adults. Specifically:

• Brent’s activities were enabled by several powerful friends, one of whom was a senior staff member at CFAR at the time. It’s likely that this created pressure on ACDC to rule in favor of Brent for fear of upsetting that person, even if this pressure was only experienced at a subconscious level.

• Brent was a skilled manipulator, and seems to have successfully controlled the frame of the ACDC investigation. Quoting from CFAR’s apology: “The [ACDC] report restated Brent’s oft-repeated narratives about himself almost verbatim, and largely read like a pro-Brent press release.”

These factors are not unique to Brent. In most of the truly complex and protracted cases that I’m familiar with, at least one party has been significantly manipulative. Many well-intentioned people, even those who are good at dealing with conflict or have other qualities that might make them a good fit for a conflict resolution panel, are still susceptible to this kind of manipulation. It’s also just very difficult to come to a conclusion when not all parties are engaging in good faith.

Medium posts

In September of 2018, two of Brent’s former partners and one of his former friends posted publicly about their experiences with him on Medium, under the name Mittens Cautious. The case was thereby brought before the court of public opinion, and public opinion came down firmly on the side of Mittens Cautious.

When people found out about ACDC’s previous ruling on Brent, many were appalled that ACDC had seen the evidence laid out in the Medium posts and ruled that it was okay for Brent to continue on like that. ACDC was disbanded within a week of the first Medium post, and people publicly questioned whether they could trust CFAR or the members of ACDC going forward.

REACH panel

Immediately after the Medium posts went up, the Berkeley REACH (a rationalist & EA community center that existed at the time) announced the formation of a panel intended to apply actual due process to the Brent case. This panel was elected by a vote of REACH stakeholders, and carried out an investigation over the next three months.

In December, the panel released a statement saying that they recommended that Brent be banned from REACH, but that, following legal advice they’d received, they would not be publicly sharing their reasoning. It’s not mentioned in the linked statement, but I believe the concrete worry was that publishing a statement that casts someone in a negative light opens you up to a libel lawsuit. So, the community as a whole gained very little from this repeated investigation.

---

When does community resolution work?

I have seen some local rationalist communities handle conflicts and bans relatively gracefully. These communities:

• are relatively small — about 150 people or fewer, which probably-not-coincidentally is Dunbar’s number,

• have explicit structure — most are centered around a weekly or monthly event and have one to four people in leadership positions, and

• are fairly bounded — generally strictly social, lacking the entanglement of personal and professional life found in the Bay and sometimes in EA.

In those communities, processes for hearing interpersonal complaints can sometimes actually work. Some groups (generally smaller ones) elect a benevolent dictator; others let all their members vote to elect a panel.

In a sub-Dunbar community, everyone (approximately) knows each other, which makes it easier to trust the decisionmakers and the process that chose them.

Also, in a meetup-based community, the decisionmakers have some real enforcement authority, if they find that sanctions are necessary. They can’t stop anyone from being friends with the accused, but they definitely can exclude them from meetups and online spaces — and if the meetups (and online spaces) are the heart of the community, this is a pretty effective way of excluding the person from the community as a whole.

However, having a small, structured community is not sufficient to ensure that this will work.

What would be the elements of a good system?

Now that we’ve seen all the ways that things can go wrong, it should be easy to reverse those and see how things could go right! If we can’t look at all the factors and then figure out the optimal course of action from first principles, how can we call ourselves rationalists?!

That’s a joke; there’s a reason this post is called “The impossible problem of due process”. That said, I think that a good system for community resolution of conflicts would probably have the following elements:

• Trust: The decisionmaking body needs to have legitimacy in the eyes of the community, and the process needs to have sufficient transparency to allow people to judge whether they approve of it.

• Enforcement ability: If you decide a person should face consequences for their actions, you need to be able to credibly and legitimately enforce those consequences.

• Holistic judgment: A person should be kicked out if they seem, on the whole, to be bad for the community. They don’t need to be found guilty beyond reasonable doubt of a specific egregious crime.

[AnnaSalamon]

I appreciate this post for spelling out an unsolved problem that IMO is a major reason it’s hard to build good community gatherings among large groups of people, and for including enough detail/​evidence that I expect many, after reading it, can see how the trouble works in their own inside views. I slightly wish the author had omitted the final section (“What would be the elements of a good system?”), as it seems less evidence-backed than the rest (and I personally agree with its claims less), and its inclusion makes it a bit harder for me to recommend the article to those needing a problem-description.

[Raemon]

Huh, I just re-looked over the final section after reading your comment here. The final section seems to have a fairly reasonable epistemic status, and the 3 bullets seem pretty reasonable to me. Curious to hear what you disagree about them?

[AnnaSalamon]

I mean, I might be being dumb on all these points. But I personally disagree about:

• There being a viable “good system for community resolution of conflicts” in larger-than-Dunbar groups (to be fair, the post author does too… except then not at the end?)

• Phrasing the cause-of-enforcement as “you decide a person should ‘face consequences for their actions’” (IMO, kicking people out of a community should usually be more about “they impose risks/​costs we can’t live with” and less about “making them face consequences”)

• A sort of missing mood in the third bullet point (“Holistic judgment: a person should be kicked out if they seem, on the whole, to be bad for the community. They don’t need to be found guilty beyond reasonable doubt of a specific egregious crime.”) I agree with the denotation of what’s written. But there are two memorable-to-me cases where I was part of kicking someone out of the bay area rationalist community (not Brent, nothing most overseas readers would’ve heard about; quieter affairs); and where their lives and sanity rapidly got a lot worse, to the point where I’d put like 30% that the decision to exile them “ruined their lives”. I wouldn’t advise past-me against either decision, because they were people we really didn’t know how to live with, with major repeated situations. But … if someone seemed, on the whole, to be a mild force for boredom and awkwardness in the community, say, I certainly woudln’t kick them out? (And again, I assume neither would the post’s author, mingyuan; but I wish the bullet point e.g. said “if they seem, on the whole, to be someone we can’t live with in a healthy fashion”, or else differentiated between kicking someone out of a random meetup, vs doing things that’ll trigger exile from a place that includes almost all a person’s social ties, built up over years).

I think my problem with the last section is only that it is not up to the very high standard that the rest of the post seems to me to hit, in which things are made unusually clear to even a young/​inexperienced reader who is happy to believe relayed events but who wants to see the why of things for themself. (And I’m not providing these ‘disagreements’ because I think the article would be better with my opinions instead of the authors; I don’t think I”m especially correct about these matters; I’m providing them as evidence that this part of the article is less visibly-true-to-all-readers, e.g. to me)

[mingyuan]

(I’m rarely online but for the record) I agree that the last section is weak and specifically agree with all of the points you made in this comment, including agreeing with the observation that exile from the community can ruin people’s lives.

[Screwtape]

I’ve had this on my to-review log all review season, and I guess I’m getting this in mere hours before it closes.

What does this post add to the conversation?

The most important piece I think this adds is that the problem is not simple.

I keep seeing people run into one of these community conflict issues, and propose simple and reasonable sounding ways to solve it. I do not currently believe the issue is impossible, but it’s not as simple as some folks think when they stare at it the first time.

How did this post affect you, your thinking, and your actions?

Context: For a time, Mingyuan’s role involved overseeing the global Astral Codex Ten community, particularly the in-person parts. When she resigned from that position, I’m the one who took it up. I got to see a version of this post before it went live, shortly after I took up the role.

According to her, more than 50% of the reason she stepped down was she became entangled in community conflicts. When you are something like a month into a new role and your predecessor hands you a document saying a problem is impossible and it also caused her to resign, and you already have a fresh instance of the problem in your inbox? That tends to affect you, your thinking, and your actions.

Does it make accurate claims? Does it carve reality at the joints? How do you know? /​ Is there a subclaim of this post that you can test?

Well, I haven’t stepped down because of becoming entangled in community conflict yet. This kind of thing is in the top three most likely reasons I step down if I do someday resign though. (Citation, my my own self evaluation.)

Do investigations eat up hundreds of person-hours? Yep, they totally can if you let them. Skill and practice and a willingness to triage can cut that down a lot.

Do panels generally have much real ability to enforce things? Here I half disagree. The rationality community in particular is fuzzy and amorphous, without a clear single roster or doorway. In that situation, the panel can’t enforce things. More structured communities- including sub-communities within the rationalist community- can potentially give panels enforcement ability. The LessWrong site itself, the r/​rational discord, or an organization and venue space like Mox can delegate a ban decision to a panel and enforce that. That said, this requires the panel being granted this capability, and for the enforcing entity to carry it out.

If you just try and convene a panel mid-conflict, and it doesn’t have a scope or actual authority to do stuff (“actual authority” being, they’ve got the ban command on the forum, or keys to the dormitory, or the people who do will likely follow the panel’s advisement, or something like that) then yeah they’re not going to have much ability to enforce anything.

Do panels act like they are courts of law? Unclear. The ones I’m most familiar with had literal lawyers on them, though not in a professional capacity. I think a little of this is that the panel is trying to have some higher standard of evidence, but also that trust isn’t transitive; it’s just much easier for me to feel confident in what I experienced than it is for a panel to feel confident that I’m relaying my experience accurately.

Do panels often lack a secure sense of legitimacy? Unclear, I’m not in their heads. I do think that giving a panel a clear scope is very useful.

Do favourable rulings lend legitimacy to bad actors? Yep, and more than you’d think. It’s just a very straightforward move if you’re a bad actor who a panel has investigated and deemed fine to bring up that fact as often as it seems helpful. It may be a useful intuition pump to picture some social deception game where an ability can detect what team someone’s on, which has somehow been spoofed or misread. One thing that caught me by surprise is that the rulings don’t even need to be straightforwardly favourable: I’ve seen mixed rulings be quoted in ways I consider pretty out of context.

Are extended investigations stressful for all parties involved? Unclear, I’m not in their heads, but people do keep saying this is true.

Is there often no way to find the objective truth? Yes, at least no reasonable way. One relevant skill is sifting through various claims and noticing which ones would be potentially provable. And unmentioned here is how often no one particular objective truth is significant.

What followup work would you like to see building on this post?

I’ve been trying!

Interest In Conflict Is Instrumentally Convergent is my single best followup, if I had to pick one. I really wish I’d written up more of what I learned from other communities. The rationality community is odd in many ways, but there are things we can learn. Sci-fi conventions, service non-profits, sports leagues, community colleges, martial arts dojos, small churches, all of these groups and more have lessons for us in how to manage and respond to incidents.

If you’re part of a community with a process for incident response, I’d be grateful if you’d write up how that works. You don’t need to give details on any specific incidents; just walking through the steps that would get taken for a couple hypotheticals, and who would handle each part, is useful for comparison.

So, what to vote?

It’s a good essay. It’s going in my collection of things people should read if they’re interested in the topic. It’s relevant to LessWrongers, and it’s relevant to most communities at least as a theory. It’s spot on relevant for my sub-special interest of the last year.

For all that, I personally gave it a +1. I can see a pretty clear argument for +4 and might change my mind. It’s a bit more niche than I’d want for +4, I think it’s coming from a place of (empathized with) frustration and I wish it had some more actionable moves or improvements to offer.

[Ruby]

I think this post does a good job of conveying the challenges here, grounded in actual cases. (It’s hard for me to evaluate whether it does a great job because my pre-existing knowledge on the topic.) I think this stuff is hard and I have so much sympathy for anyone who’s been caught up in it, if they’re weren’t the instigator.

I don’t feel convinced it’s impossible to do this much better. My own median world isn’t very fleshed out, but my gut tells me that dath ilan has figured out some good wisdom and process here, and I trust it. I’d also guess that if Lightcone did more in this realm^[1], we’d eventually figure out some better processes here that make things better for all involved, and possibly not just within our own domain, but guidelines for other groups to follow too.

Given that, I think it’s kinda bad to call the problem impossible. Even if something is hard and we’re unlikely to make progress on it, don’t cause people not try through excessive pessimism.

I’ll probably give this post a 1. I’d be excited if anyone wrote a sequel that was guesses at how to design a better system (inspired by the actual challenges encouraged) and perhaps experimental guide for others to try out.

1. ^

   We were doing much more of these while running the Lightcone Offices. Lighthaven is typically rented out to other groups, and not maintaining enough of it’s own persistent in-person community for us to have reason to do a lot of this kind of adjudication. (Ben’s big investigation started in the Offices day, and is a case from which there is a lot to be learned, but I do think if we unfortunately did more such cases, we would learn and get better).

[Viliam]

A big problem I see is that if the panel explains their reasoning, it exposes them to a libel lawsuit, but if the panel does not expose their reasoning, it allows the bad actor to publish their own bullshit version unopposed.

(Specifically, I have seen “they simply don’t like me because I am politically incorrect” written by people who were actually banned for doing specific bad things to other people in the rationalist community; things that were completely unrelated to politics or anything in that style.)

[M. Y. Zuo]

Bingo, the root problem is pretending to have any quasi-judicial structure/​authority at all.

People of roughly equal status issuing ‘judgements’ or ‘decisions’ on each other really doesn’t make sense for that reason, at best you can do so within a private club and its property lines.

A federation of private clubs may decide to do so, very rarely, only for the most serious cases, because as mentioned in the OP there’s always the risk of some clubs siding with the accused and then deciding to leave, splitting the federation.

[Shankar Sivarajan]

I have seen “they simply don’t like me because I am politically incorrect”

If that’s plausible, I think them lying about it being the case in their specific instances is the lesser problem.

[Viliam]

I guess plausibility is in the eye of the beholder.

To me it seems very unlikely, because I have said a ton of politically incorrect things on LW and I do not remember any negative consequence. Quite the opposite, for me LW and ACX are the places where I can speak most freely. (I have been banned on Reddit, for saying things that seemed very mild to me.)

I do not even remember anyone being banned from LW for expressing an opinion. I admit I am no longer paying attention to bans, but when I did, the typical reason was “an obvious spam account”, and occasionally “creates dozens of sockpuppet accounts and uses them for vote manipulation”. Other things (such as crackpottery or religious zealotry) are solved by the community downvoting the offender. Which is exactly how the karma system is supposed to work.

Now consider how little effort it takes to ban someone from a website (just clicking a button), compared to banning someone from the community (choosing a panel, collecting evidence, dozen people spending 100 hours debating the issue, writing a public announcement). The idea that someone does all this effort for something they wouldn’t care to push a button for is… just absurd.

The number of people banned in real life is very small (I do not know the exact number), and if you ask in private, the typical reason is something like “raped someone” or “stole money from someone”. But you don’t put that in writing, unless you also want a lawsuit on top of those 100s of hours spent on the issue.

But ultimately this is a matter of trust. Like, maybe I am lying to you right now...

Another piece of evidence is that the debate on LW is mostly uncensored. Even if your opinion gets downvoted, it still stays there for everyone to read. So if things like “someone gets banned from rationalist meetups merely for saying [insert heresy]” actually happened, someone would complain about that on LW. If that comment got downvoted, other people would ask why it got downvoted and whether it is true; etc. If the entire comment section got nuked, people would debate that on ACX or Facebook, etc. The rationalist community is connected by various tools (LW, ACX, Reddit, Discord, Facebook, meetups); censorship in one part would be a hot topic in the other parts.

So the fact that you have never seen such controversy debated on LW is also evidence that such things simply do not happen.

[Linch]

https://​​www.smbc-comics.com/​​comic/​​aaaah

[Shankar Sivarajan]

For a community that makes a big deal about “Bayesian reasoning,” it’s amusing and utterly unsurprising that so many here fail utterly to recognize applications of it in practice.

[Elizabeth]

I’ve also learned: It is easier to remove good people from your community than to remove people who are aggressive, manipulative, narcissistic, or psychopathic, because the former will remain reasonable throughout the process, while the latter will fight dirty.

To give a concrete example: the original ACDC memo says Brent cooperated with the panel while the complainant did not, and that’s one reason they trust him (recalled from memory, didn’t double check my wording). Which is insane; an abuser who is good at manipulating people and not traumatized finds it easier to engage in apparent good faith with an investigating org than a traumatized victim. You can try and fix this by being overtly sympathetic towards the victim, but that’s either biasing the outcome or a lie.

[kave]

I feel like this is the opposite of the quoted text? Or your example is of the bad actor both “remaining reasonable” and “fighting dirty”

[Amalthea]

Fighting dirty can involve looking reasonable to the outside, e.g. being willing to lie or bend the truth and distracting from the key issues of the matter—these can all be done civilly.

[kave]

But while I can certainly see failing “to engage in apparent good faith” is like an understandable reaction, it doesn’t really feel like a central example of “remain[ing] reasonable throughout the process”

[Elizabeth]

They reasonably chose to disengage with the process that was miserable for them and had a low expected pay off.

[Vaniver]

When people found out about ACDC’s previous ruling on Brent, many were appalled that ACDC had seen the evidence laid out in the Medium posts and ruled that it was okay for Brent to continue on like that

As I recall, the ACDC had in fact not seen the evidence laid out in the Medium posts. (One of the panelists sent an email saying that they had, but this turned out to be incorrect—there was new information, just not in the section he had read when he sent the email, and prematurely sending that email was viewed as one of the ACDC’s big mistakes, in addition to their earlier ruling.)

[BrassLion]

I’ve seen a bit of this in some organizations I’ve been part of. The most important part I see missing is enforcement powers. If you have a group of excellent and sage judges who can impartially consider the facts but all they can do is issue advisory opinions, all you have is another social bloc taking one side or the other in an interpersonal debate. You have gossip and the whisper network cosplaying a court of law. You have nothing.

I have not the first clue how to handle this outside of a formal organization, but solving this in an organization with a real structure is at least a step forward. Not naming names because I don’t speak for them, but one organization I’ve seen do this well had a small elected committee that handled any complaints about members. They did basic investigation, worked with both accuser and accused, and normally focused on repairing the harm and reintegrating both people into the community (the keyword here is restorative justice, if you want details). Often this caught troublemakers early, and people have been accused of wrongdoing, gone through this process, and remained members of the organization with much drama. This works because the committee also had a hammer: they could ban people from attending organization events temporarily while they put permanently removing the person from the organization to an organization wide vote. While I was part of the organization, no one was removed by a vote, but people have been removed by vote before, and people have quit the org when it became clear that there would be consequences for the bad actions due to this process.

Importantly, the committee was explicitly held to far lower standards than a court of law. They were expected to be impartial (and not handle a case if they couldn’t be), but no one was spending hundreds of hours on a single case. There are no lawyers, no rules of evidence. The trade-off for the whole process being far less formal and less work—not just for the committee members but also for the accuser and accused—is the relatively lightness of the penalties. Defendants in a criminal court need the full protection of formal systems of law because the court can take away your money, your freedom, or even your life. The committee I’m talking about can at worst take away your membership in one organization, and even that has a safeguard. The people who wrote the rules for the committee understood that not being a massive burden on everyone involved was worth the trade-off of less protections. The focus on repairing and preventing harm over punishment also helped—the consequences of ruling the wrong way on something are minimized here, which helps smooth over any wrong decisions.

It’s also worth making it explicit that when this sort of issue comes up, of how to handle accusations of wrongdoing in organizations or groups without involving the courts, it’s almost always about rape. The whisper networks or public accusations that these sorts of more formal structures try to avoid aren’t about someone getting beat up or stolen from, they’re almost always about rape, other sexual assault, or behavior that make people feel like they’re in danger of being raped. If someone beats you up or steals from you, you don’t have to start a whisper campaign, you go to the police (or a lawyer in some cases). Explicitly saying something I don’t have proof of, epistemic status “confident but maybe you shouldn’t be”: this is because the (American) courts are bad at dealing with rape and sexual assault, and in the absence of formal systems of justice you get informal systems of justice, and those tend to suck for everyone, accuser included (public accusations and whisper campaigns aren’t a great way to achieve justice, but if they’re what you have, they’re what you have).

[Viliam]

I agree with most of this, except for the part “if someone steals from you, you can go to police”. Yes, if someone literally grabs your purse and starts running away, call the police.

But there are also situations that are less clear, for example when people didn’t write a contract because they trusted a person from the same community, when they just made a verbal agreement and when the time came for the other person to do what they said, they denied it or said it was all a misunderstanding. Sometimes you have no legal evidence.

Imagine you and me renting a car together for a trip. You rent the car and pay for the gas, and the agreement is that when the trip is over, I will pay you 50% of the expenses. Except that afterwards I don’t give you a cent, I claim that this was a misunderstanding, that I sincerely believed that you were offering me a free ride from the goodness of your heart otherwise I would never have agreed to that, and I am a poor person and don’t have so much money anyway. (For bonus points, I can act offended and accuse you of playing dirty tricks on me.) Now what? It is your name on the car rent, you paid cash for the gas, I deny everything. I don’t think you would have a good case in court, and even if you did, it is probably not worth the effort. You will remember never to trust me again, but next time I will simply misuse someone else.

Situations like this are typically addressed by gossip, but it is your word against my word, and maybe I am more popular than you because I am a charismatic sociopath. Or I may threaten to sue you for slander. Or I may proactively share my story first (how you tried to scam me out of money by offering me a free ride and then trying to make me pay you an absurd amount of money—for better effect I will exaggerate the number) and warn people against trusting you. Or I may start an unrelated conflict with you, for example a political debate on LW or Facebook, and then insist that you just made up the story about the car as a punishment for me expressing insufficiently woke opinions. Etc.

Now what a panel can do in such situation is to listen to both sides, make no conclusion at first, but if I keep having “misunderstandings” with more people who have no past record of similar behavior, after two or three it cumulatively becomes a strong Bayesian evidence that I am actually the bad guy. Even if there is still zero evidence admissible in court.

[Amalthea]

“if I keep having “misunderstandings” with more people who have no past record of similar behavior, after two or three it cumulatively becomes a strong Bayesian evidence that I am actually the bad guy.” It’s not quite that easy. Abuser’s may particularly tend to seek out vulnerable people, and it’s a real effect that when you are already raising a complaint about someone, this may open you up to further abuse by other bad actors, who can now have exploit that you now have spent your social capital. In other words, Bayesian considerations have a place, but you need to be extra careful that you’re not misattributing the correlations.

[Viliam]

Yes, that makes sense. But if we already assume that the truth may be impossible to figure out reliably, no solution is going to be perfect.

(There is also a similar problem, that some people are more likely to be falsely accused than others.)

[BrassLion]

You’re correct. I wish we had any sort of tradition that let people with a minor dispute go before some neutral party without expense or bureaucracy—less in the sense of court of extremely small claims, and more that people should be more willing to say to a trusted friend, “hey, resolve this dispute for us and we’ll buy you dinner.” Then again, this requires you to both trust the same person, and for neither person to be acting in such bad faith that they refuse the process. If there’s a default place people can go, with very low costs in time and no cost in money, refusing to even have an argument heard would be a pretty big strike against you in most situations, and as you said if the panel has multiple cases/​ complaints against a person that’s at least Bayesian evidence that they’re doing something wrong.

[Jiro]

Perfect due process is impossible for the reasons you describe. But there’s a difference between “not perfect” and “egregiously bad”, and if you focus too narrowly on the inability to make the process perfect, people are going to get away with processes that are egregiously bad.

If you wrote this in February, it preceded the Nonlinear accusations. From what I can tell from what I read here, they’re a lot closer to “egregiously bad” than to “not perfect”. Do they change your opinion of due process to any extent?

[Jonathan Claybrough]

“It’s true that we don’t want women to be driven off by a bunch of awkward men asking them out, but if we make everyone read a document that says ‘Don’t ask a woman out the first time you meet her’, then we’ll immediately give the impression that we have a problem with men awkwardly asking women out too much — which will put women off anyway.”

This seems a weak response to me, at best only defensible considering yourself to be on the margin and without thought for longterm growth and your ability to clarify intentions (you have more than 3 words when interacting with people irl).

To be clear explicitly writing “don’t ask women out the first time you meet her” would be terrible writing and if that’s the best writing members of that group can do for guidelines, then maybe nothing is better than that. Still, it reeks of “we’ve tried for 30 seconds and are all out of ideas” energy.

A guidelines document can give high level guidance to the vibe you want (eg. truth seeking, not too much aggressiveness, giving space when people feel uncomfortable, communicating around norms explicitly), all in the positive way (eg. you say what you want, not what you don’t want), and can refer to sub-documents to give examples and be quite concrete if you have socially impaired people around that need to learn this explicitly.

[Nathan Young]

Yeah I agree a bit here. This document itself is a nice case study in how things can go wrong. There could be a similar one for “don’t ask women out”

[Amalthea]

Which document, and in what way?

[Nathan Young]

This post. It gives some useful context around the bad outcomes of panels.

[Benquo]

I think the governance problems you’re describing are hard for two distinct but related reasons worth addressing separately:

1. Disagreement on values

2. Overly broad scope of penalties

Disagreement on Values

Not everyone relates to governance and penalties the same way.

Some people have the naïvely appropriate attitude that governance is to protect public goods; if someone is likely to behave in ways that are dangerous or harmful to others we should identify them, and exclude them when needed to protect others. Punishments are a cost, and the point of graduated punishment and judicial process is to make sure they’re imposed only in cases where the benefit outweighs the cost.

Other people relate to punishments as a scapegoating process where from time to time the vibes demand we other someone, and the narrative about why we’re doing that is just part of the way we negotiate who gets othered. The fact that this is obviously illegitimate and parasitic on the naive attitude doesn’t prevent it from being how a lot of people feel.

Some people instrumentalize the whole thing as a tool to be used against their personal enemies, or a way to demonstrate factional loyalty.

Yet others see some positive opportunity in being naughty and getting away with things, will try to help out others who seem cool, and will try to derail any investigation that threatens this sort of covert mutual protection league.

Obviously we’re not going to get the naïvely good outcomes from a process where a large minority if not an outright majority of people are trying to derail the process one way or another, so there’s little point in trying if we’re not thinking about the adversarial problem explicitly.

Overly broad scope of penalties

One of the major strategic wins scapegoating has scored against pronormativity is that people seem to assume that the natural punishment for sufficiently objectionable behavior is banning and shunning, regardless of the scope of the behavior. In many cases this is a bad penalty to oppose, and opposing excessive penalties will motivate people otherwise sympathetic to the investigative process to opt out, derail, or even exit the community entirely, further worsening the problem of an antinormative majority.

For example, if someone tends to behave objectionably in certain ways to romantic partners, the obvious remedy is to make this information available to people considering dating them, so they can decide whether they find that behavior objectionable enough to avoid; excluding them from common spaces is just ridiculous.

I stopped engaging with or supporting the Bay Area EA /​ Rationalist Community’s attempts to create public spaces, in response to a couple of cases where it seemed like the “investigative” process had been totally compromised by the compulsion to “splitting” behavior where formal community institutions were not trying to figure out and publicize what had happened and why and then come up with rational risk-mitigation measures, but instead trying to figure out who the “bad guy” was and exclude them. For instance, the Brent Dill “investigations” resulted in penalties but no clear findings, and then the only friend who had bothered to help me notice that I’d been participating in a culture of communal dishonesty around EA and AI risk, and figure out how to deconfuse myself about this, was banned for unspecified reasons. Functionally I think I’m justified in construing this as a coverup, not an attempt at enforcing generalizable norms.

Related:

https://​​benjaminrosshoffman.com/​​guilt-shame-and-depravity/​​

https://​​unstableontology.com/​​2021/​​04/​​12/​​on-commitments-to-anti-normativity/​​

[dr_s]

I think another big issue with codes of conduct is that they just shift the burden around. You’re still left with the issue of interpreting the spirit of the norm, deciding if everyone at least made a good faith attempt to stick to it, if good faith is enough, etc. I don’t have much experience with them but I honestly don’t know if they help that much. Seems to me like there are two types of “troublemakers” in communities no matter what:

1. people who are purposefully deceptive and manipulative;

2. people who simply lack the social grace and ability to “read the room” required to meet other’s expectations of social norms adherence rather than just stick to their own interpretation of them.

Type 1 you want to kick out. Type 2 you ideally want to be a lot more graceful and forgiving with, though in some extreme cases you might still need to kick them out if their problems are unfixable and they make no effort whatsoever to at least mitigate the issues. Writing the rules down doesn’t help as long as they’re flexible, because the problem those people have is a lack of the sort of intuition that others possess for grokking flexible rules altogether. And if you make them inflexible you just have a chilling effect on every interaction, and throw away a lot of good with the bad. After all, for example, why shouldn’t someone ask a woman out at their first meeting if they’re both clearly into each other and sparks are flying? These things happen! And people should be able to give it a try, I think it’s important to make it clear that there’s nothing sinful or bad about courtship or flirting per se; too many rigid rules about such personal interactions inevitably carry a sort of puritanical vibe with them, regardless of intention. But as usual, “use your best judgement” has very uneven effects because some people’s best judgement is just not that great to begin with, often through no fault of their own.

[Viliam]

It would be better if the “guardians of proper behavior” had options beyond “permaban or nothing”.

I mean, for serious wrongdoing—permaban; of course. But for mere annoying behavior—verbal admonishment in private, verbal admonishment in public, second admonishment, temporary ban, and only after everything else fails, permaban. This would be a more gentle approach.

Counter-intuitively, the gentler approach would require more authority and trust in the guardians. The verbal admonishment needs to be taken seriously by everyone else, otherwise the admonished person will not take it seriously, and thus it is a waste of time. (Unfortunately, as I know my people, the most likely reaction would be a loud and long debate about what are the proper norms, what is the best algorithm to decide what are the norms, whether the guardians are overstepping the line, and whether telling someone “dude, she said she was not interested, leave her alone” means that the entire community is epistemically rotten, corrupted by wokeness, and tomorrow the wrongthinkers will be probably be taken to death camps.) It is important for the admonishments to be cheap (cognitively and emotionally) for the guardians, otherwise they will hesitate to react, and the entire system will revert back to “we only act when the situation is clearly horrible”, in which case the appropriate outcome is indeed the permaban.

[Elizabeth]

Less Wrong has put a lot of effort into creating other options and automating them. Downvoted allow subtle user feedback. Mass sock puppet voting previously ruined down votes, so they put effort into controlling sock puppets. Weighted votes so trusted people have more impact. Too many downvoted comments will lead to throttling but not bans.

My sense is these have improved things a lot, but we’re also far from my ideal outcome.

[Vanessa Kosoy]

“It’s true that we don’t want women to be driven off by a bunch of awkward men asking them out, but if we make everyone read a document that says ‘Don’t ask a woman out the first time you meet her’, then we’ll immediately give the impression that we have a problem with men awkwardly asking women out too much — which will put women off anyway.”

American social norms around romance continue to be weird to me. For the record, y’all can feel free to ask me out the first time you meet me, even if you do it awkwardly ;)

[gull]

I think this might be typical-minding. The consequences of this dynamic are actually pretty serious at macro-scale e.g. due to reputation of meetups, and evaporative cooling of women and high-status men as they avoid public meetups and stop meeting people new to AI safety.

I’m glad to hear there’s people who don’t let it get to them, because it is frankly pretty stupid that this has the consequences that it does at the macro-scale. But it’s still well-worthy of some kind of solution that benefits everyone.

[Vanessa Kosoy]

I honestly don’t know. The discussions of this problem I encountered are all in the American (or at least Western) context^[1], and I’m not sure whether it’s because Americans are better at noticing this problem and fixing it, or because American men generate more unwanted advances, or because American women are more sensitive to such advances, or because this is an overreaction to a problem that’s much more mild than it’s portrayed.

Also, high-status men, really? Men avoiding meetups because they get too many propositions from women is a thing?

1. ^

   To be clear, we certainly have rules against sexual harassment here in Israel, but that’s very different from “don’t ask a woman out the first time you meet her”.

[Caleb Biddulph]

I think the implication was that “high-status men” wouldn’t want to hang out with “low-status men” who awkwardly ask out women

[gull]

I used the word “high-status men” as a euphemism that I’m not really comfortable talking about in public, did not notice it would be even harder to get for non-americans. My apologies.

I used “high-status men” mainly as the opposite of low-status men, in that they are men who are low status due to being short, ugly, unintelligent, or socially awkward, sufficiently so that they were not able to gain social status. These people are repellent to other men as well as women, sadly. @Roko has been tweeting about fixes to this problem such as reforms in the plastic surgery industry, and EA and rationalists are well above base rate communities (e.g. classical music society) for tolerating/​improving low social skills and male shortness. This is due to primate instincts which usually cannot be overcome, in spite of people feeling optimistic about their ability to overcome them. The degree of social awkwardness is defined/​measured by the harm it does someone; if someone looks “socially awkward” but in a charming or likable way that remains charming or likable, that is not a serious (or even significant) case, as it does not doom someone to low social status.

This is also a reason why so many people have so little tolerance for non-transhumanists as a class of ideologues; non-transhumanists accept the status quo of our current tech level, where human genetic diversity dooms a large portion of people to a pointlessly sad and miserable life without their consent, (on top of dooming everyone to a short life).

[Gunnar_Zarncke]

When joining a workplace, a university, or a club sport, you are opting into a code of conduct, including the part where there’s an entity that enforces that code of conduct on you. It may not be explicitly on your mind, but at some level you’re aware that that’s part of the package you’re choosing to sign up for.

Maybe rationalists should found more regular societies? In Germany, it is said that if three people come together, they found an association (it requires seven). The process is easy and common. It seems the US counterpart is an “501(c)(3) entity” and I have no clue how easy it is, but it shouldn’t be difficult for the average member of this community. That would implicitly provide for some governance structure.

[jefftk]

Creating a 501c3 is not the hard part. The hard part would be convincing a decentralized and informal community that they should accept the proposed central organization, including trusting it when it said who they should associate with.

(Which people are naturally wary of!)

[Gunnar_Zarncke]

Agree. I don’t think it will work well when an informal community already exists. It probably works best when you found the community is small (seven members). After the fact, it might work for sub-communities with specific purposes, such as organizing events.

[Viliam]

If the organization owns the place where people meet, the organization can put someone on a blacklist, and the local community can either meet at that place without the banned person, or coordinate on finding a different place to meet at—which would be difficult if the best local coordinators are members of the organization.

[Gunnar_Zarncke]

After a conversation with ChatGPT and reading up on it, it seems South Korea is furthest in the process of a working due process system for cancelation and defamation dynamics.

https://​​en.wikipedia.org/​​wiki/​​South_Korean_cyber_defamation_law

[AnthonyC]

Good or bad, I doubt it’s a coincidence that this is in a society with more more stringent cultural contexts and assumptions about face and what it is or isn’t ok to say and do in regards to other people’s reputations.

[Nathan Young]

As a counter example I think the EA community health team does better than I would expect. A huge global community and they are relatively trusted and the community takes responsibility for itself. Like I don’t know that I’d say it does well but I think I’d be hard pressed to find an analogous community that has a better such function.

Feels like they have a better batting average than rationalists in dealing with issues like this (I guess at least partly cos events are more central to EA than rationalism). Many other communities seem to have no responsibility at all—many political groups seem rife with issues but noone taking responsibility over them.

I agree overall that this is a really hard problem, though I suppose I am more optimistic than OP.

[tailcalled]

Is it possible to get access to those five reports as a case study?

[hwold]

I am not a community organizer, but if I was, I would just edict the MAD rule : if any dispute can’t be reasonably settled (in a way that would escalate to a panel), both the plaintiff and the defendant are kicked out of the community. The ratio of assholes to victims being low (hopefully), frame it as a noble sacrifice from the victim to keep the rest of the community safe (whoever is the actual victim here). “Some of you may die, but it’s a sacrifice I’m willing to make”.

True, you’re slightly disincentivizing the reporting of true abuses (but not so much, since you’re saving a lot on the trauma of investigation). But you’re also heavily disincentivizing both false accusations and “I’m a good manipulator so I’m confident I will be able to get away with it”. Add to it the man-hours saved, and I’m pretty confident it is a net gain — as long as you can enforce it.

As a bonus, if a dispute end up with community bans, you can be reasonably sure that the plaintiff was the victim. But of course, that only works if you don’t go from that observation to concluding “therefore the victim is un-banned”. A noble sacrifice, really, in service of both community and truth. A sacrifice nonetheless.

[Richard_Kennaway]

This is how adults deal with fights among children. The adults don’t care who started it, who did what to whom, justice, fairness, or anything at all but getting them to stop bothering the adults.

It is not a way to deal with adults.

[jefftk]

There are a bunch of problems with this even if implemented in good faith, but an obvious attack is that you convince a friend external to the community to (meet the minimum qualifications for membership and) accuse your target. This gets your target and your friend banned without investigation, but of course your friend does not care about getting banned.

[Shankar Sivarajan]

both the plaintiff and the defendant are kicked out of the community

This is unironically a great idea. It doesn’t hinge on annoying questions of evidentiary standards, mens rea, seriousness of the charge, conflicts of interest etc. The idea that you can take anyone down, no matter how influential or powerful, if you’re willing to go down with them really appeals to me.

as long as you can enforce it.

Yeah, you probably can’t.

[trevor]

The idea that you can take anyone down, no matter how influential or powerful, if you’re willing to go down with them really appeals to me.

I think this is a mistake; it’s easy to twist people out of shape such that they take themselves down and one of your enemies.

This is prevalent in elite human systems e.g. among corporate executives, and it’s instrumentally convergent so smart psychos will discover opportunities for it here too, and it’s also highly exploitable by tech companies and intelligence agencies if anyone working there ever decides they’d like AI safety out of the way.

[Shankar Sivarajan]

You’re right that this would open up new avenues for inter-elite conflict. That’s the tradeoff for sharply curbing elite abuse of the less powerful.

[trevor]

I think this was good commentary, and I would never have noticed the exploit without it. For the record, I upvoted.

[Shankar Sivarajan]

Disagreement votes don’t lower your karma, do they? I (strongly) disagreed, but I don’t want to even marginally contribute to you getting rate limited. Either way, I have now more than evened it out.

[trevor]

I’ve never been rate-limited, not even on EAforum where I get downvoted a lot, so I worry more about the social status/​brand reputation loss from people seeing my comments with any kind of negative karma attached to them.

[Richard_Kennaway]

The idea that you can take anyone down, no matter how influential or powerful, if you’re willing to go down with them really appeals to me.

Suicide bombing?

[tailcalled]

I can’t help but think a lot of this stuff is caused by an ontology mismatch. People encounter dynamics where they feel stuck or harmed, and they want there to be norms that prevent those dynamics, but norms operate on the level of behavior, which is typically too adaptable and nuanced to really be the source of the problems people encounter. You basically have to directly manage your community on the basis of traits that create patterns of behavior, rather than on the basis of the behavior itself. But this requires a comprehensive ontology of harm-relevant traits to make decisions on the basis of.

(This is related to the fact that Bayesian inference fails if you lack the grain of truth/​realizability assumption. If your prior assumes some people Are Bad but that’s not really how people differ, then your probability estimates for whether someone Is Bad are not gonna converge based on evidence.)

This sort of ontology is usually constructed in a messy way that tends to be pretty controversial among rationalists, and it tends to be communicated very poorly, which tends to make it hard to apply it in practice because every part is going to recursively seem wrong.

I think I know better (or at least, more rationalist-compatible) ways to construct and communicate such ontologies, but I have a hard time finding people who are interested in committing a lot of work to testing it with me, and also I’ve been persuaded that this isn’t the only problem and that there are other problems too, including some of what you stated in the article. In addition to the ontology mismatch issue, two of the problems that I think are most significant are 1) lack of information, 2) corruption of justice processes.

Problems 1 and 2 seem worsened a lot by applying it to conflict rather than to other subjects, so I’m inclined to give up advertising my method for justice processes. Still, if anyone wants to give it a go and is willing to commit a ton of work into it from their community, please reach out to me.

[Amalthea]

I don’t understand what you’re trying to say—would you mind trying to illustrate what you mean?

[tailcalled]

I wouldn’t mind in principle, but it is extremely compressed, and kind of stitched together. As in, while I have examples of individual bits and pieces of my claims, I don’t have any examples that go end-to-end. Instead I derived bits of the theory from the different examples and then stuck those theory-bits together into an overall framework.

So basically I can zoom in on different bits but I don’t have time to zoom in on it all at once. (I am working writing it all up at once, but that’s not ready yet. Partly I’m also delaying because I prefer searching for examples that go end-to-end and because I’m still learning new nuances and techniques.)

[winstonBosan]

TLDR: This is an long metaphor to draw parallels between the Hanseatic League and the broader EA/​LW communities. It is OK to not be a {corporation, societas, collegium, universitus} with common/​top-down/​bottom-up violence-monopolizing system. The price to implement a resolution system people would find satisfactory might be too high.

The fact it is hard to resolve conflict is because it is an integral part of the bargain, not an isolated bug. I personally don’t want us to become a chimera with nine heads—a chimera for the sole purpose so we can utter that “we are a state”.

---

“the [Efficax Altruismus] is not a societas: (a company) for it knows neither a common ownership of goods nor shared ownership of the good, since in the [Efficax Altruismus] there is no joint ownership; nor is it a company formed for certain commercial transactions, since in the [Efficax Altruismus] each individual makes transactions for himself, and the profit and loss falls on each individual…

It is also not a collegium (a college)….since it is formed from separate [communities]. It is also not a universitas (a corporate body), because…for it is required that it has property, a common treasure, a common seal, a common syndicus and a recognised leader.

“the [Efficax Altruismus] is … a firm alliance of many cities, towns and communities for the purpose of ensuring that enterprises on water and on land have the desired and favourable success and that effective protection is provided against pirates and highwaymen, so that the merchants are not deprived of their goods and valuables by their raids.”—The Hanse in Medieval and Early Modern Europe, I think...

[TAG]

Here are the actions available to you:

You didn’t mention community wide votes.

[Review Bot]

The LessWrong Review runs every year to select the posts that have most stood the test of time. This post is not yet eligible for review, but will be at the end of 2025. The top fifty or so posts are featured prominently on the site throughout the year.

Hopefully, the review is better than karma at judging enduring value. If we have accurate prediction markets on the review results, maybe we can have better incentives on LessWrong today. Will this post make the top fifty?

[Chris Lakin]

The non-linear drama might be another good case study

[trevor]

I have had two friends thrown out of the community without due process, and it was absolutely awful for them. For one of them, I was never told enough about the situation to make my own judgment, but the other one was thrown out basically just because a couple powerful people didn’t like his vibes.

I’ve also learned: It is easier to remove good people from your community than to remove people who are aggressive, manipulative, narcissistic, or psychopathic, because the former will remain reasonable throughout the process, while the latter will fight dirty.

This is why it’s so important to never, ever, EVER give people quotas for purging, or rewarding people in any way for racking large numbers of purge cases, no matter how good they make each purge case look.

Goodharting will typically take place if you stack the incentives like that.

[peterbarnett]

The thing you quoted doesn’t imply that there were any quotas or rewards, or metrics being Goodharted. (Definitely agree that quotas or rewards for “purging” is a terrible idea.)

[localdeity]

I have the impression that it is a general problem with public prosecutors (source: cultural osmosis). So if one were to imagine creating someone whose job is analogous to prosecuting criminals, one would have to be very careful about the incentives one sets up for that person, and it’s not obvious that rationalists could walk in and immediately solve it better than society has managed to do with prosecutors.

[df fd]

I assume no one will read this comment

assuming the problem is really intractable and the current panel process is the best available solution, then the standard solution is to put up a [scapegoat]

i.e. civil servant do not want to/​ not able to do something for someone, instead of saying “this is my judgement”, point to an other entity [e.g. code of conduct, boss, etc], and deflect the blame. the point is not to deflect the blame though, but to keep on functioning despite having to make unpopular decisions.

I assume that chatGPT would make an excellent [scapegoat]

feed all the gathered evidences to ChatGPT, ask it for judgement [with the appropriate precondition: ” you are a wise and benevolence judge, etc”], if it agree with the panel decision, then when the inevitable blow back happen you can point to chatGPT and said it agreed with you and it is obviously unbiased

if it disagreed with the panel decision then it would be a sanity check, the panel should find more evidence or double check their reasoning, since ChatGPT can serve as stand in for the average Joe who read all the evidence, if it is not convinced you do not have a convincing case.

[FireStormOOO]

Not a lawyer, but the “can’t explain your reasoning” problem is overblown. Just need to be very diligent in separating facts from the opinions and findings of the panel. There is a reason every report of that sort done professionally sounds the particular flavor of stilted that it does.

“Our panel found that <accused> did <thing>” ← potential lawsuit, hope you can prove that in court. You’re not a fact finder in a court of law, speak as if you are at your own peril.

“Our panel was convened to investigate <accusation> against <accused>. Based on <process>, we believe the accusation credible and recommend the following: …” ← A OK

“Our panel was convened to investigate <accusation> against <accused>. Based on <process>, we were unable to corroborate the accusation and cannot recommend action at this time.” ← A OK

“person X said Y, I did/​didn’t believe them” is basically always fine, provided X actually said Y. Quoting someone else’s statement is well protected, and you’re entitled to your opinions. The trouble happens when your opinions/​findings/​beliefs are stated as facts about what happened instead of facts about what you believe and what evidence you found persuasive.

There’s also nothing stopping the panel from saying “we heard closed door testimony...”, describe the rough topic, speakers relation to the inquiry, and the degree to which it was persuasive.

[FireStormOOO]

There’s a more general concern here for running organizations where anyone can sue anyone at any time for any reason, merit or no. If one allows the barest hint of a lawsuit to dictate their actions, that too becomes another vector through which they can be manipulated. Perhaps a better thing to aim for is “don’t do anything egregious enough a lawyer will take it on contingency”, use additional caution if the potential adversary is much better resourced than you (and can afford sustained frivolous litigation).

[StartAtTheEnd]

I am precommitting to not engaging with any comments, because I am mostly offline and I think that is good

Statements looking like this are the parts I agree with the most! Politics and everything similar kind of suck.
Trying to formalize something subjective, and enforcing a absolute global conclusion to something inherently relative, is not a game worth playing. I’d even argue that not trying in the first place might be for the best. Just go full Taoism here.

handling non-criminal misconduct

Why would you do this? If you’ve been on the internet for over 15 years, you should have noticed that we went from moderating rules and laws, to moderating people based on impressions, feelings and moral values. This is how nautrality died, and how the internet became politicalized. It’s also when the internet started growing stupid (probably because the average intelligence and autistic and nerdy tendencies fell over time)

I don’t think that due process is impossible when it comes to crimes and concrete rules and laws. But when it comes to subjective things like being a “good person” or a “bad person”, it’s very difficult. To be frank, I think it’s bad taste to try and force other people to adopt ones own opinions in the first place. Tribalism and herd dynamics is the very issue, not the solution, as those in the dynamic naively thinks (because they’re converging towards a shared opinion, which they think is equal to solving the problem). The entire problem with echo-chambers are common discrimination is basically that holistic thinking is replaced by thinking in labels.

I think that if you have a community with reasonable people, none of this is a problem. And if you have a community with unreasonable people (which I define as the most common type of person, those prone to conformity and tribalism rather than individualism), then no system can help you. Unreasonable people don’t even want fairness and justice in the first place, but revenge. They’ll call their revenge “justice” of course, but this is only true from their experienced subjective reality /​ storytelling. It’s a sort of delusion, but it’s not inherently bad as those who are too disillusioned end up doing nothing, whereas those who truly care are able to influence the world.

Anyway, trying to solve problematic herd behaviour with a system or process is like trying to solve low intelligence with better education, it won’t work. Social dynamics are self-balancing, if somebody is an unlikable person, they will become disliked over time naturally. Trying to impose artificial systems to overwrite natural systems is where the modern society fails, for you can’t design complex processes/​systems and just have them work. These things work bottom-up, not top-down.

[Jiro]

Social dynamics are self-balancing, if somebody is an unlikable person, they will become disliked over time naturally.

I think that doesn’t count as self-balancing unless that’s the only way to become disliked.

[StartAtTheEnd]

What disturbs this balance is only people trying to control the world rather than letting things follow a natural flow. But (and this might be a lazy tautology), that too is self-balancing in a sense, as a community which cannot hold itself together will split in two or destroy itself.

If something requires constant effort, it’s a sign that you’re trying forcing something into an unnatural state. After reaching this unnatural state, you have a complex problem with many factors, which is why I argue “Avoid getting in the state in the first place, or just let the thing collapse”.

In any case, I think we’re trying to solve the wrong problem here. If somebody hasn’t broken the law, I don’t want to brainstorm how we can punish them and get public opinion on our side in hating them. On the other hand, if somebody is disliked by everyone, then you can argue their innocence all that you want, peoples feelings are unlikely to change by logical argument

[deleted]