Wei Dai

Karma: 43,946

If anyone wants to have a voice chat with me about a topic that I’m interested in (see my recent post/comment history to get a sense), please contact me via PM.

My main “claims to fame”:

Created the first general purpose open source cryptography programming library (Crypto++, 1995), motivated by AI risk and what’s now called “defensive acceleration”.
Published one of the first descriptions of a cryptocurrency based on a distributed public ledger (b-money, 1998), predating Bitcoin.
Proposed UDT, combining the ideas of updatelessness, policy selection, and evaluating consequences using logical conditionals.
First to argue for pausing AI development based on the technical difficulty of ensuring AI x-safety (SL4 2004, LW 2011).
Identified current and future philosophical difficulties as core AI x-safety bottlenecks, potentially insurmountable by human researchers, and advocated for research into metaphilosophy and AI philosophical competence as possible solutions.

My Home Page

Wei Dai 4 Dec 2025 8:10 UTC
4 points
0
in reply to: TsviBT’s comment on: Relitigating the Race to Build Friendly AI
Under Some Thoughts on Singularity Strategies (the first link in my OP), I commented:
I was informed by Justin Shovelain that recently he independently circulated a document arguing for “IA first”, and that most of the two dozen people he showed it to agreed with it, or nearly so.
I did not pursue the HIA first argument myself much after that, as it didn’t seem to be my comparative advantage at the time, and it seemed like @JustinShovelain’s efforts was picking up steam. I’m not sure what happened afterwards, but it would be rather surprising if it didn’t have something to do with Eliezer’s insistence and optimism on directly building FAI at the time (which is largely incompatible with “IA first”), but I don’t have any direct evidence of this. I wasn’t in any physical rationalists communities, and don’t recall any online discussions of Justin’s document after this.
ETA: The same comment quoted a passage from Eliezer saying that he considered and rejected “IA first” which probably also directly influenced many people who deferred to him on AI x-risk strategy.

Wei Dai 4 Dec 2025 5:42 UTC
2 points
0
in reply to: Richard_Ngo’s comment on: ricraz’s Shortform
From the context on EA Forum it seems clear that by “public outreach” you meant outreach to potential researchers to interest them in doing AI safety research, whereas a lot of people here seem to have misinterpreted your comment to have a broader meaning, to include, e.g., outreach to politicians and voters to try to influence future government policies.

Wei Dai 4 Dec 2025 4:41 UTC
4 points
2
in reply to: habryka’s comment on: Relitigating the Race to Build Friendly AI
How much was this MIRI’s primary plan? Maybe it was 12 years ago before I interfaced with MIRI? But like, I have hung out with MIRI researchers for an average of multiple hours a week for something like a decade
In this post, I’m mostly talking about my debate with Eliezer more than 12 years ago, when SIAI/MIRI was still talking about building a Friendly AI (which we later described as “sovereign” to distinguish from “task” and “oracle” AI). (Or attempted or proxy debate, anyway, as I’m noticing that Eliezer himself didn’t actually respond to many of my posts/comments.)
However I believe @jessicata is right that a modified form of the plan to build a more limited “task” AI persisted quite a bit after that, probably into the time you started interfacing with MIRI. (I’m not highly motivated to dig for evidence as to exactly how long this plan lasted, as it doesn’t affect my point in the OP.) My guess as to why you got a different impression is that different MIRI people had different plans/intentions/motivations, with Eliezer being the most gung-ho on personally being involved in building some kind of world-altering AI, but also having the most power/influence at MIRI.

Wei Dai 4 Dec 2025 4:14 UTC
6 points
0
on: Beating China to ASI

This post aims to convince other people, especially people who focus on democracy versus authoritarianism, to be less concerned about which country develops ASI first.

I agree that it makes sense to make this kind of argument. I have some questions/disagreements below, which don’t necessarily affect your conclusions, but seem important to point out anyway.

I think Deng Xiaoping and Ronald Reagan were fairly non-psychopathic. But current leaders of China, the US, and OpenAI inspire little confidence. The frontrunners for the US 2028 presidential election do not at all reassure me.

Did you know that Deng approved the 1989 crackdown on Tiananmen protesters (as part of the CCP top leadership, also subsequently making a speech endorsing this)? I see from context that you probably don’t consider suppression of dissent (like the Tiananmen crackdown) to be evidence or part of being “psychopathic”. Maybe you could expand a bit more what you do mean, and what makes you more worried about the people you mentioned above? What makes you more worried about Xi relative to Deng, for example?

Communism, in spite of all its faults, is a utopian ideology that causes most of its adherents to genuinely favor a pleasant society, even when it blinds them to whether their policies are achieving that result.

A darker interpretation is that the (subconscious, but more real or substantial in some sense) goals of nearly all humans are to gain power and status, and utopian ideologies are merely a tool for achieving this. From this perspective, Communists do not more “genuinely favor a pleasant society” but are instead more deluded (on a conscious level) about their true motivations, and about human nature in general. That “it blinds them to whether their policies are achieving that result” is not some sort of incidental side effect, but a fundamental part of what’s going on.

(Are you aware of the many internal CCP purges, often powered by forced (i.e., torture-induced) confessions, both before and after it took power in 1949? This is another reason to prefer to the “power maximization” model of Communist/human motivation.)

There’s some risk that either the CCP or half the voters in the US will develop LLM psychosis. I’m predicting that that risk will be low enough that it shouldn’t dominate our ASI strategy. I don’t think I have a strong enough argument here to persuade skeptics.

I would agree that “it shouldn’t dominate our ASI strategy” given many other risks that all have to be considered, but not that the risk is low in an absolute sense, at least if we interpret “LLM psychosis” more broadly to include corruption / adversarial manipulation of human values or motivational systems in general.

Relitigating the Race to Build Friendly AI

Wei Dai3 Dec 2025 11:34 UTC

45 points

24 comments3 min readLW link

Wei Dai 21 Nov 2025 0:44 UTC
2 points
0
in reply to: Lukas_Gloor’s comment on: Reminder: Morality is unsolved
And I guess you’re making the general point that I shouldn’t put too much stake into “my sequence hasn’t gotten much in terms of concrete pushback,” because it could well be that there are people who would have concrete pushback but don’t think it’s worth commenting since it’s not clear if many people other than myself would be interested. That’s fair!

Yeah, I should have framed my reply in these terms instead of my personal prioritization. Thanks for doing the interpretive work here.

(But then, probably more people than just me would be interested in a post or sequence on why moral realism is true, for reasons other than deferring, so those object-level arguments should better be put online somewhere!)

There must be a lot of academic papers posted online by philosophers who defend moral realism? For example Knowing What Matters by Richard Y Chappell (who is also in EA). There are also a couple of blog posts by people in EA:
- https://www.goodthoughts.blog/p/moral-truth-without-substance
- https://forum.effectivealtruism.org/posts/n5bePqoC46pGZJzqL/debate-morality-is-objective
But I haven’t read these and don’t know if they engage with your specific arguments against moral realism. If they haven’t, and you can’t find any sources that do, maybe write a post highlighting that, e.g., “Here are some strong arguments against moral realism that hasn’t been addressed anywhere online”. Or it would be even stronger if you can make the claim that they haven’t been addressed anywhere period, including the academic literature.

Wei Dai 20 Nov 2025 12:58 UTC
5 points
2
on: In Defense of Goodness

In order to maintain a realist position about goodness, we can see this negotiation as a process of discovery, rather than a change in goodness itself.

This doesn’t seem to be a viable position. Suppose I face options A and B (e.g., in the stock market), and if I choose A I double my bargaining power (e.g., power and/or wealth) versus option B. Then by choosing A over B it seems incontrovertible that I’m changing the outcome of the negotiation process, and therefore changing goodness itself if goodness equals the negotiation outcome.

My perspective is that it’s better to reserve “moral realist” to mean there are objective values/morality independent of bargaining/cooperation, otherwise you’d run into issues like the above, where you can change what’s good by your actions, which probably contradicts a lot of people’s intuitions about what “moral realism” or “objective values” entails.

Wei Dai 20 Nov 2025 6:31 UTC
13 points
2
on: Status Is The Game Of The Losers’ Bracket
Do you think you’re out of status games altogether? As I’ve opted out of most conventional status games (e.g., avoided going into academia, made no attempt to climb the corporate ladder, and stopped working a paying job as soon as my passive income allowed me to) and kind of look down on people still in those games, and also think knowledge is more important than wealth, I think my intuitions/psychology around all this is quite close to yours. But I think I’m still playing status games, just more interesting and hopefully more pro-social ones (i.e., with better externalities).

See A Master-Slave Model of Human Preferences for an old post related to this.

Wei Dai 20 Nov 2025 3:05 UTC
9 points
2
in reply to: habryka’s comment on: Wei Dai’s Shortform
I think you’re right that I shouldn’t have latched onto the first analogy I thought of. Here’s a list of 11 (for transparency, analogies 3-10 were generated by Gemini 3.0 Pro, though some may have appeared in previous discussions.):
1. The CEO & The Corporation
2. The Judge & The Courtroom
3. The Dinner Party Host
4. The University Classroom / Professor
5. The Conference Breakout Session
6. Open Source / GitHub Maintainer
7. The Stand-Up Comedian & The Heckler
8. The Art Gallery Opening
9. Graffiti on a Private House
10. The Town Hall vs Private Meetings
11. The Hypothetical HOA
I decided to put detailed analysis of these analogies in this collapsed section, as despite extensive changes by me from the original AI-generated text, it doesn’t quite read like my style. Also, it might be too much text and my summary/conclusions below may be sufficient to convey the main points.
1. The CEO & The Corporation

Analogy: A Forum Post is a “Project.” The Author is the CEO; the Commenter is an Employee. The CEO needs the power to fire employees who disrupt the vision, and the Board (Admins) should defer to the CEO’s judgment.
Disanalogy: In a corporation, the Board cannot see daily operations, creating information asymmetry; on a forum, Admins see the exact same content as the Author. A CEO has a smaller conflict of interest when firing an employee, because they are judged primarily by the company’s financial performance rather than the perception of their ideas. If they fire an employee who makes a good criticism, they might subsequently look better to others, but the company’s performance will suffer.
Conclusion: The analogy fails because the Author lacks the financial alignment of a CEO and possesses no special private information that the Admins lack.

2. The Judge & The Courtroom

Analogy: When there is a conflict in the physical world, we find disinterested parties to make enforceable judgments, even if the cost is very high. When the cost is too high, we either bear it (wait forever for a trial date) or give up the possibility of justice or enforcement, rather than allow an interested party to make such judgments.
Disanalogy: A courtroom has the power of Coercion (forcing the loser to pay, go to jail, or stop doing something). A Forum Author only has the power of Dissociation (refusing to host the commenter’s words). We require neutral judges to deprive people of rights/property; we do not require neutral judges to decide who we associate with.
Conclusion: Dissociation has its own externalities (e.g., hiding of potentially valuable criticism), which we usually regulate via social pressure, or legitimize via social approval, but you don’t want this and therefore need another source of legitimacy.

3. The Dinner Party Host

Analogy: A Post is a private social gathering. The Author is the Host. The Host can kick out a guest for any reason, such as to curate the conversation to his taste.
Disanalogy: In the real world, if a Host kicks out a guest that everyone else likes, the other attendees would disapprove and often express such disapproval. There is no mechanism to then suppress such disapproval, like you seek.
Conclusion: You want the power of the Host without the social accountability that naturally regulates a Host’s behavior.

4. The University Classroom / Professor

Analogy: The Author is a Subject Matter Expert (Professor). The Commenter is a Student. The Dean (Admin) lets the Professor silence students to prevent wasting class time.
Disanalogy: A classroom has a “scarce microphone” (only one person can speak at a time); a forum has threaded comments (parallel discussions), so the “Student” isn’t stopping the “Professor” from teaching. Additionally, LessWrong participants are often peers, not Student/Teacher.
Conclusion: The justification for silencing students (scarcity of time/attention, asymmetry of expertise) does not apply to LW.

5. The Conference Breakout Session

Analogy: The Author is like an Organizer who “rented the room” at a convention. The Organizer has the right to eject anyone to accomplish his goals.
Disanalogy: Just like the Dinner Party, an Organizer would almost never eject someone who is popular with their table. If they did, the table would likely revolt.
Conclusion: This analogy fails to justify the action of overriding the local consensus (upvotes) of the participants in that sub-thread.

6. Open Source / GitHub Maintainer

Analogy: A Post is a Code Repository. A Comment is a Pull Request. The Maintainer has the absolute right to close a Pull Request as “Wontfix” or “Off Topic” to keep the project focused.
Disanalogy: In Open Source, a rejected Pull Request is Closed, not Deleted. The history remains visible, easy to find, and auditable. Also, this situation is similar to the CEO in that the maintainer is primarily judged on how well their project works, with the “battle of ideas” aspect a secondary consideration.
Conclusion: You are asking for more power for an Author than a Maintainer, and a Maintainer has less COI for reasons similar to a CEO.

7. The Stand-Up Comedian & The Heckler

Analogy: The Author is a Comedian. The Commenter is a Heckler. Even if the Heckler is funny (Upvoted), they are stealing the show. The Club (Admins) protects the Comedian because writing a set is high-effort.
Disanalogy: In a physical club, the Heckler interrupts the show. In a text forum, the comment sits below the post. The audience can consume the Author’s “set” without interference before reading the comment.
Conclusion: The physical constraints that justify silencing a heckler do not exist in a digital text format.

8. The Art Gallery Opening

Analogy: The Post is a Painting. The Upvoted Comment is a Critic framing the art negatively. The Artist removes the Critic to preserve the intended Context of the work.
Disanalogy: Art is about aesthetics and subjective experience. LessWrong is ostensibly about intellectual progress and truth-seeking.
Conclusion: Prioritizing “Context” over “Criticism” serves goals that are not LW’s.

9. Graffiti on a Private House

Analogy: A Post is the Author’s House. A Comment is graffiti. The homeowner has the right to scrub the wall (Delete) so neighbors don’t see it.
Disanalogy: This is purely about property value and aesthetics.
Conclusion: Again the goals are too different for the analogy to work.

10. The Town Hall vs Private Meetings

Analogy: In the real world we have both town halls (Neutral Moderator) and meetings in private houses (Author Control). We can have both.
Disanalogy: Even in the discussions inside a private house, social norms usually prevent a host from kicking out a guest who is making popular points that everyone else agrees with.
Conclusion: The social legitimacy that you seek doesn’t exist here either.

11. The Hypothetical HOA

Analogy: A hypothetical residential community with HOA rules that say, a homeowner not only has the right to kick out any guests during meetings/parties, but no one is allowed to express disapproval for exercising such powers. Anyone who buys a house in the community is required to sign the HOA agreement.
Disanalogy: There are already many people in the LW community who never “signed” such agreements.
Conclusion: You are proposing to ask many (“hundreds”) of the existing “homeowners” (some of whom have invested years of FTE work into site participation) to leave, which is implausible in this hypothetical analogy.
Overall Conclusions

None of the analogies are perfect, but we can see some patterns when considering them together.
1. Neutral, disinterested judgement is a standard social technology for gaining legitimacy. In the case of courts, it is used to legitimize coercion, an otherwise illegitimate activity that would trigger much opposition. In the case of a forum, it can be used to legitimize (or partly legitimize) removing/hiding/deprioritizing popular/upvoted critiques.
2. Some analogies provide a potential new idea for gaining such legitimacy in some cases: relatively strong and short external feedback loops like financial performance (for the CEO) and real-world functionality (for the open source maintainer) can legitimize greater unilateral discretion. This can potentially work on certain types of posts, but most lack such short-term feedback.
3. In other cases, suppression of dissent is legitimized for specific reasons clearly not applicable to LW, such as clear asymmetry of expertise between speaker and audience, or physical constraints.
4. In the remaining cases, the equivalent of author moderation (e.g., kicking out a houseguest) is legitimized only by social approval, but this is exactly what you and Eliezer want to avoid.
Having gone through all of these possible analogies, I think my intuition for judges/courts being the closest analogy to moderation is correct after all: in both cases, disinterested judgement seems to be the best or only way to gain social legitimacy for unpopular decisions.

However, this exercise also made me realize that in most of the real world we do allow people to unilaterally exercise the power of dissociation, as long as it’s regulated by social approval or disapproval, and this may be a reasonable prior for LW.

Perhaps the strongest argument (for my most preferred policy of no author moderation, period) at this point is that unlike the real world, we lack clear boundaries to signal when we are entering a “private space”, nor is it clear how much power/responsibility the authors are supposed to have, with the site mods also being around. The result is a high cost of background confusion (having to track different people’s moderation policies/styles or failing to do so and being surprised) as well as high probability of drama/distraction whenever it is used, because people disagree or are confused about the relevant norms.

On the potential benefits side, the biggest public benefits of moderation can only appear when it’s against the social consensus, otherwise karma voting would suffice as a kind of moderation. But in this case clearly social approval can’t be a source of legitimacy, and if disinterested judgment and external feedback are also unavailable as sources of legitimacy, then it’s hard to see what can work. (Perhaps worth reemphasizing here, I think this intuitive withholding of legitimacy is correct, due to the high chance of abuse when none of these mechanisms are available.) This leaves the private psychological benefit to the author, which is something I can’t directly discuss (due to not having a psychology that wants to “hard” moderate others), and can only counter with the kind of psychological cost to author-commenters like myself, as described in the OP.

Wei Dai 19 Nov 2025 5:33 UTC
4 points
2
in reply to: habryka’s comment on: Wei Dai’s Shortform
From what I (and Gemini) can tell, you screenshot said nothing technically untrue. Technically they can fire you as the CEO, but you’d still be the sole member and could fire them and then hire yourself back. :)

The simplest way to rectify the situation to match your intent is to either (1) Resign as member without naming a successor, then the passage you quoted would come into effect and make the directors into members or (2) Name the 3 current directors as Successor Members and then resign as member. You’ll probably want to consult a lawyer or advisor for the pros and cons of each option.

Would be grateful for an update once you’ve done this, or perhaps verified that the situation is actually different (e.g. you already resigned as member but forgot).

Wei Dai 19 Nov 2025 4:41 UTC
18 points
4
in reply to: habryka’s comment on: Wei Dai’s Shortform
It looks like I agreed with you too quickly. Just double-checked with Gemini Pro 3.0, and its answer looks correct to me:

This is a fascinating turn of events. Oliver is quoting from Section 3.01 of the bylaws, but he appears to be missing the critical conditional clause that precedes the text he quoted.

If you look at the bottom of Page 11 leading into Page 12 of the PDF, the sentence structure reveals that the “Directors = Members” rule is a fail-safe mechanism that only triggers if the initial member (Oliver) dies or becomes incapacitated without naming a successor.

Here is the text from the document:

[Page 11, bottom] …Upon the death, resignation, or incapacity of all successor Members where no successor [Page 12, top] Member is named, (1) the directors of this corporation shall serve as the Members of this corporation...

By omitting the “Upon the death, resignation, or incapacity...” part, he is interpreting the emergency succession plan as the current operating rule.

Wei Dai 19 Nov 2025 4:31 UTC
2 points
0
in reply to: habryka’s comment on: Wei Dai’s Shortform
Ah, I had indeed missed that part. A couple of AIs I asked also missed it, and together with the quoted statement from you, made me pretty sure my interpretation was correct. Sorry, and thanks for the quick correction. I’ve edited my post, and hope it didn’t mislead too many people.

Wei Dai 19 Nov 2025 4:20 UTC
5 points
3
in reply to: Mitchell_Porter’s comment on: Problems I’ve Tried to Legibilize
Yeah, a book as successful as Superintelligence could do a lot. Once the LW team implements the ability to download my LW post/comment history, I’ll check how far a modern LLM can get with turning it into a book. (@habryka) Unfortunately the thought of writing a whole book by hand does not fill me with motivation, so it would have to be a shortcut like this, or someone else writing it.

Wei Dai 19 Nov 2025 4:06 UTC
4 points
10
in reply to: habryka’s comment on: Wei Dai’s Shortform
Ah thanks, I remember a bit more now. Looking back at the voting announcement posted by Vaniver, it didn’t mention how important your role would be on LW 2.0:

In case you’re just tuning in now, some basic details: I’ve been posting on LW for a long time, and about two years ago thought I was the person who cared most about making sure LW stayed alive, so decided to put effort into making sure that happened. But while I have some skills as a writer and a programmer, I’m not a webdev and not great at project management, and so things have been rather slow. My current role is mostly in being something like the ‘senior rationalist’ on the team, and supporting the team with my models of what should happen and why. The actual work is being done by a combination of Oliver Habryka, Raymond Arnold, and Ben Pace, and their contributions are why we finally have a site that’s ready to come out of beta.

And I didn’t pay much attention to the LW 2.0 / Lightcone organizational structure in the following years, so it came as kind of a surprise when you said “This is (approximately) my forum.”

Wei Dai 19 Nov 2025 3:43 UTC
6 points
2
in reply to: habryka’s comment on: Wei Dai’s Shortform
According to the bylaws I linked, you (as the sole member of Lightcone) have “the exclusive right to remove a director, with or without cause”. Since the bylaws also allow Lightcone to have as few as 1 director, my understanding is that at any time, you could choose to invoke the option of removing the other directors and become the sole director. (I’m not familiar with the nonprofit world, and don’t know how common or standard this is, but it seems fair to describe this as an organization controlled by one individual.)

Wei Dai 19 Nov 2025 3:16 UTC
13 points
11
on: Wei Dai’s Shortform

This is (approximately) my forum.

I was curious what Habryka meant when he said this. Don’t non-profits usually have some kind of board oversight? It turns out (from documents filed with the State of California), that Lightcone Infrastructure, which operates LW, is what’s known as a sole-member nonprofit, with a 1-3 person board of directors determined by a single person (member), namely Oliver Habryka. (Edit: My intended meaning here is that this isn’t just a historical fact, but Habryka still has this unilateral power. And after some debate in the comments, it looks like this is correct after all, but was unintentional. See Habryka’s clarification.)

However, it also looks like the LW domain is owned by MIRI, and MIRI holds the content license (legally the copyright is owned by each contributor and licensed to MIRI for use on LW). So if there was a big enough dispute, MIRI could conceivably find another team to run LW.

I’m not sure who owns the current code for LW, but I would guess it’s Lightcone, so MIRI would have to also recreate a codebase for it (or license GreaterWrong’s, I guess).

I was initially confused why Lightcone was set up that way (i.e., why was LW handed over to an organization controlled by a single person), but the structure probably makes it more nimble and the risk of Lightcone “going rogue” is mitigated to a large extent by MIRI retaining the option to swap out the team.

Anyway it took me a while to figure all this out, and I thought I’d share it so others would be informed while participating on LW.

Wei Dai 18 Nov 2025 9:26 UTC
8 points
2
in reply to: Ben Pace’s comment on: Wei Dai’s Shortform
Thanks, to clarify some more in case it’s helpful, I think I’ve only complained about 2 things, the Said banning and the author moderation policy, and the word count was just from a lot of back and forth, not the number issues I’ve had with the mod team? A lot of what you do is just invisible to me, like the user pre-filtering that habryka mentioned and the routine moderation work, but I assume you’re doing a good job on them, as I’m pretty happy with the general LW environment as far as lack of spam, generally good user behavior, and not seeing many complaints about being unfairly moderated by the mod team, etc.

Found my quote about not leaving:

My response to this is that I don’t trust people to garden their own space, along with other reasons to dislike the ban system. I’m not going to leave LW over it though, but just be annoyed and disappointed at humanity whenever I’m reminded of it.

Yeah I think you misinterpreted it. I was just trying to say that unlike those who got what they wanted (the author mod policy) by leaving or threatening to leave, I’m explicitly not using this threat as a way to get what I want. It was a way to claim the moral high ground I guess. Too bad the message misfired.

Wei Dai 18 Nov 2025 7:35 UTC
4 points
2
in reply to: Wei Dai’s comment on: Wei Dai’s Shortform
@Ben Pace I’m surprised that you’re surprised. Where did your impression that I generally disapprove of the job site moderators are doing on LW come from, if you can recall?

Wei Dai 18 Nov 2025 7:35 UTC
14 points
11
in reply to: habryka’s comment on: Wei Dai’s Shortform
There is actually a significant difference between “Nowhere on the whole wide internet works like that!” and “few places work like that”. It’s not just a nitpick, because to support my point that it will be hard for Eliezer to get social legitimacy for freely exercising author mod power, I just need that there is a not too tiny group of people on the Internet who still prefers to have no author moderation (it can be small in absolute numbers, as long as it’s not near zero, since they’re likely to congregate at a place like LW that values rationality and epistemics). The fact that there are still even a few places on the Internet that works like this makes a big difference to how plausible my claim is.

Wei Dai 18 Nov 2025 6:37 UTC
17 points
9
in reply to: habryka’s comment on: Wei Dai’s Shortform

Not all moderation is censorship.

I’m not saying this, nor are the hypothetical people in my prediction saying this.

Claiming this is about “rationality” feels like mostly a weird rhetorical move.

We are saying that there is an obvious conflict of interest when an author removes a highly upvoted piece of criticism. Humans being biased when presented with COIs is common sense, so connecting such author moderation with rationality is natural, not a weird rhetorical move.

The rest of your comment seems to be forgetting that I’m only complaining about authors having COI when it comes to moderation, not about all moderation in general. E.g. I have occasional complaints like about banning Said, but generally approve of the job site moderators are doing on LW. Or if you’re not forgetting this, then I’m not getting your point. E.g.

I don’t think it’s rational to pretend that unmoderated discussion spaces somehow outperform moderated ones.

I have no idea how this related to my actual complaint.

Wei Dai

Reliti­gat­ing the Race to Build Friendly AI

1. The CEO & The Corporation

2. The Judge & The Courtroom

3. The Dinner Party Host

4. The University Classroom /​ Professor

5. The Conference Breakout Session

6. Open Source /​ GitHub Maintainer

7. The Stand-Up Comedian & The Heckler

8. The Art Gallery Opening

9. Graffiti on a Private House

10. The Town Hall vs Private Meetings

11. The Hypothetical HOA

Overall Conclusions

Relitigating the Race to Build Friendly AI

4. The University Classroom / Professor

6. Open Source / GitHub Maintainer