Viliam

Karma: 28,212

Viliam 6 Aug 2026 8:34 UTC
6 points
0
on: Charles Goodhart Elementary School
I actually taught at a school that did something similar, but most parents were happy about the outcome. I quit after a few months, because I realized I did not want to play this game.
If your situation happened at that school, the parents would probably put their child in a different school, which would solve their problem, but also allow the school to continue existing unchanged.

Viliam 4 Aug 2026 11:33 UTC
4 points
2
in reply to: silentbob’s comment on: Philip Harker’s Shortform
To me, finding undisclosed LLM text just is a very strong indicator the person publishing it wants to get away with sloppy work and doesn’t respect (the time and epistemic environment of) their readers very much.
Yeah, that is exactly how I perceive it.
Even if you let the LLM generate the answer, the least you can do is tell it “be concise” or “give me a summary” or quote the relevant part instead of dumping the entire wall of text. If you don’t bother doing even that, that signals you clearly do not value my time.
Another thing: prompts matter. If you don’t share the prompt, I have no idea whether you asked your AI to “compare X and Y, evaluate pros and cons from various perspectives, and then make a conclusion” or “give me 7 reasons why X is better than Y”. The former is potentially helpful, the lattes is not. And the more AI-psychotic the person seems, the more likely I think it was the latter.

Viliam 4 Aug 2026 11:20 UTC
8 points
2
in reply to: Martin Randall’s comment on: Elizabeth’s Shortform
It is quite possible that dath ilan has deep dysfunctions that Eliezer may be unaware of. For example, what about those near-copies of him that happen to be below average, and fail to channel their energy into becoming a benevolent quasi-cult leader? Will they patiently accept their place at the bottom of the food chain and cooperate, or will they try to overthrow the system? Can the system resist so many smart people trying to overthrow it?
Agree on the likely extinction by smart and enthusiastically pro-tech young Eliezers.
Also, how much does the world actually reflect its median inhabitants? For example, Earth seems both much better (more science) and much worse (more wars) that talking to the median Earth human in isolation would suggest, I think. Some things are median-driven, some things are outlier-driven (science, crime).

Viliam 4 Aug 2026 11:12 UTC
2 points
0
in reply to: lc’s comment on: Elizabeth’s Shortform
I of course don’t want him to pretend otherwise, if that’s the extent of his feelings.
Neither do I, but I’d like to have a more explicit confirmation than a tweet that could be interpreted as a joke.

Viliam 4 Aug 2026 11:10 UTC
2 points
0
in reply to: habryka’s comment on: Elizabeth’s Shortform
We need a video from MAPLE to compare. :D

Viliam 4 Aug 2026 0:15 UTC
6 points
2
in reply to: Taylor G. Lunt’s comment on: LessWrong vs. TikTok: Tips for capturing attention in a non-rational space
Thank you for the information.
I agree that it would be most useful as a propaganda tool for AI Safety, and I wondered whether using it as a propaganda tool (in a wider sense of the word) is actually the typical approach. Seems like yes.
This feels different from YouTube (unless I am wrong about YouTube) where at least naively it seems like the typical approach is to try getting lots of money from ad views. At least it originally was like that.

Viliam 3 Aug 2026 22:13 UTC
4 points
0
on: LessWrong vs. TikTok: Tips for capturing attention in a non-rational space
I have never seen TikTok. Please explain to me: do you somehow directly make money if your videos get sufficiently popular?
Or is the typical motivation on TikTok just general attention seeking, sharing propaganda, and making money indirectly e.g. by promoting your / someone else’s brand in your videos?

Viliam 3 Aug 2026 12:39 UTC
2 points
0
in reply to: Kaj_Sotala’s comment on: The High-Control Dynamics at MAPLE
I’m not sure I understand what connection you’re drawing. Are these people also saying that their experiences vindicate some particular theory about how the mind works?
People who attribute their success to the Law of Attraction believe that it makes the Law of Attraction the correct theory about how the mind works. (The theory does not have many gears, but it is a theory.)
Thank you for the lists!
I notice that for some items in the list, I have also heard opposite evidence… I can’t find a link now, but someone complained that meditation reduced their ability to feel love and joy, so they stopped. I find it plausible that the practice has different people on different people, the question is how many of which. The list could in theory be a starting point to a series of experiments, if we could operationalize the statements.

Viliam 2 Aug 2026 19:13 UTC
14 points
6
in reply to: lc’s comment on: Elizabeth’s Shortform
To take the prompt “what would the world be like if I were the median person” and straightforwardly imagine a world that is the best most productively organized world seems narcissistic on its own.
Doesn’t seem that way to me. The median person in this world has IQ 100. Go 1SD lower, and people are functionally illiterate. Go 2SD higher instead; there you will meet me. Eliezer seems smarter (no, I am not saying the smartest person ever), so maybe add extra 1SD. Now imagine a world where that is the average.
Hard to say what exactly such world would look like, but imagine a world where a typical subreddit is like LessWrong, but specialized on something, and with more users. Imagine such world’s political debates; imagine its institutions. Is it so unlikely that things could be much better? Compared to our world, where if you say “the sky is blue”, many people will immediately object that their holy book says that it is red, or that they have read on social networks that sky is fake; and you can only have policies that such people approve, and you need to staff your institutions with such people.
(But of course, the villains would also be smarter, so there is a chance that would invent smart systems that lock everyone else in their service. The villains would also be better at cooperation.)
Also, it is assumed that the world would not be organized by Eliezer alone, but by a billion copies of him.

Viliam 2 Aug 2026 15:18 UTC
7 points
2
on: LessWrong App
I am not a potential user, but perhaps adding a screenshot (to the github page) would help.

Viliam 2 Aug 2026 15:03 UTC
2 points
2
in reply to: lc’s comment on: Elizabeth’s Shortform
claiming to his followers to be the smartest person in the world
Citation needed.
speaking so literally about a utopian “home planet” that it’s unclear to me whether he thinks of Dath Ilan as a real place
There is a risk that this may be confusing to outsiders, but for insiders clearly a fiction.

Viliam 2 Aug 2026 14:30 UTC
8 points
5
in reply to: programjames’s comment on: Elizabeth’s Shortform
You have a point but this requires more nuance.
Jargon is a tool, sometimes inevitable—for example math is full of jargon.
But it can also be abused strategically. Renaming things needlessly, to make it more difficult to communicate with the outsiders. Renaming things needlessly, to make members believe that the idea originated in the cult (as opposed to being taken from outside and renamed), so they can’t research the idea independently. Renaming things needlessly, to get rid of associations: for example many would agree that hurting people is wrong, but they have no strong opinion on whether frobnicating people is wrong, so you just need to tell them that what seems like hurting people is actually frobnicating, and then they will be more okay with that.

Viliam 2 Aug 2026 13:24 UTC
12 points
7
in reply to: Elizabeth’s comment on: Elizabeth’s Shortform
Agree on 1. I would also add that correctness about one part of the world does not necessarily imply that the entire model is correct. Sometimes the mainstream society has a taboo, and it is easy for bad actors to be (more) true about this specific part, and then leverage the gained trust to promote their own falsehoods.
Agree on 2. Knowing that you are doing the right thing. The initial love bombing; and the feeling that you have a “tribe” you belong to. When the guru tells you to do something, you don’t procrastinate on it… and you may pleasantly surprise yourself about how productive you suddenly become. In case of conflicting motivations (e.g. I want to get rid of addiction, but I also have cool friends who are addicts), the cult may force you to cut the knot. I wish I could get many of these things without the obvious costs that the cult membership brings.
Details are important in 3, but yeah, some things require high investment that is lost if the plan fails. Though it is still possible to increase those costs maliciously.
4 -- I would definitely love to read that.
On 5 I have a post that could have been much better written, but the point is that Lifton’s scale of mind control evaluates eight different criteria, so instead of a “yes or no” answer, you get a value on a scale, and obviously the army would score much higher than everyday institutions. (You can even make an excuse for established institutions without having to pretend that their score is low.)
Another thing I would add to the list are abusive relations, which sometimes could be modeled as a cult with 1 leader and 1 follower.
There should be some qualifier on 6, because I don’t see any group of people actively fighting to make me their cult leader; I guess I am not impressive enough. (If someone desires to kiss my shoes and send me 10% of their salary, please send me a private message, so that we don’t debate such holy things in front of the heathens.)
I wonder what is a more precise model. Maybe most people desire to have a leader, and if they decide that you fit the role, then you have to defend yourself. But if they put someone else in that role, you are safe. The danger mostly applies to celebrities; although you probably could become a local celebrity easier than you realize.
EDIT: After reading this comment I think I understand your point on academia better. Now I wonder how many cult experts were grad students; whether they are familiar with the things that you criticize.
Agree on 7.
Mostly agree on 8, with the caveat that if the information about “good” comes from the leader or his followers, it may be exaggerated. Investigating a list of actual heroes turning villains could be interesting.

Viliam 2 Aug 2026 12:32 UTC
5 points
2
in reply to: herschel’s comment on: The High-Control Dynamics at MAPLE
As the author conducted the research for this document, a small number of prior attempts by MAPLE and OAK trainees to address the attacks in roughly the way that this document does were surfaced. Most of these were published under real names. The author of this document had the opportunity to speak with the authors of some of these prior works. These prior authors were in many cases attacked by the people attacking MAPLE and OAK, and in many cases suffered serious professional and personal repercussions, including being cut off from friendship groups, banned from communities, dis-invited from conferences, and threatened with legal action.
Uhm, name three?
I mean, if there are people who already used their real names in writing, and were already “canceled”, what reason is there to be shy about linking their documents?
.
I find it plausible that there are people who attack MAPLE for the wrong reasons. Any organization of this type that interacts with many people probably has someone like that.
That is irrelevant for the question of whether there are meaningful objections against MAPLE.

Viliam 2 Aug 2026 12:25 UTC
16 points
2
in reply to: Trinley Goldenberg’s comment on: The High-Control Dynamics at MAPLE
framing the word ‘cybregore’ as referring to a mystical being instead of a useful word for something that clearly needs a name is just one example.
It reminds me how Leverage Research first started talking about “demons” as a metaphor for something, but people gradually took it more and more literally, to the point of trying to exorcise them.

Viliam 2 Aug 2026 12:09 UTC
25 points
8
in reply to: Kaj_Sotala’s comment on: The High-Control Dynamics at MAPLE
You seem to reduce Christianity to praying, and Buddhism to exploring mental states, which is basically comparing a layman’s version of Christianity to a nerdy version of Buddhism… and of course, in such comparison, Buddhism comes out as the intellectual one.
But there is also a layman’s version of Buddhism, which includes praying “Namo Amida Butsu” a lot, and giving money to monks in order to gain merit. And there is also a nerdy version of Christianity, containing self-exploration and self-improvement, such as the writings of the Desert Fathers, or the spiritual exercises of St. Ignatius of Loyola; the seven capital (mental) sins, and how to overcome them.
(Perhaps some rationalists should participate in a Jesuit retreat, to learn how to overcome procrastination, so that they can become more productive at saving the world from the AI.)
.
In the things that you describe, it seems to me like you are mixing together two different kinds of statements… not entirely unrelated, but also not one straightforwardly implying the other.
One claim is that the Buddhist tradition has accumulated a lot of expertize at exploring mental states. People who can sit motionless for hours without getting bored, and who can summon feelings of great bliss on command just by using the right kind of mental muscles. I do not doubt that. And it’s cool.
My skepticism is about the implied claim that this is a good use of one’s time. For the religious kind of Buddhist, the math would be obvious—you leave your previous life as a peasant, which probably sucked; you spend your remaining days begging and meditating; and if you do it right, you can save yourself countless reincarnations, many of them extremely unpleasant—yep, totally worth it.
For an atheist from a first-world country, the math is different. Your life is probably kinda okay, you do not believe in afterlife or magic superpowers, so the benefits include cool insights, a form of therapy, maybe a productivity boost; and the cost is the amount of time you have to spend meditating. For some people it is still worth it, but I expect that for some it is not, even if they did everything exactly the right way—the therapy they could receive elsewhere (if we assume a highly competent meditation teacher, we might as well assume a highly competent therapist), the productivity boost is balanced by the time spent meditating and at retreats, and the cool insights...
Well, this is the other group of claims, that the accumulated expertize of Buddhism makes its psychological claims trustworthy (unlike their claims about reincarnation, sex advice, etc). That if you achieve experimental outcomes, then the underlying theory must be correct. Empiricism, isn’t it?
My problem with this is that I also know people who claim getting experimental results from praying to God, or doing magic. Not in the sense of “a miracle happened, and a pile of money appeared on my table”, but in the sense of “God gave me the ability to persevere where otherwise I would have given up” or “God gave me the strength to overcome my bad habits” or “I kept visualizing the success and doing the rituals, and I also worked hard on my project, and after a few years I have succeeded”.
.
One more thing that rubs me the wrong way is that I do not remember seeing anyone put an upper bound to the benefits of meditation (and Buddhism in general). I would like to see someone in the rationalist community say “meditation can help you do X, Y, Z, and that is all it is good for”, and other rationalists agree with it. Make the list of the claims as long as you want, but make it finite. A finite list can be explored, one item at a time. I am not comfortable with keeping the list unbounded.
Why is this so important for me? First, if you make it a ratchet, if it is socially okay to make claims of “meditation helps at X” but not claims of “meditation does not help at Y”, then the list of claims will only grow after time, as the people get used to the old ones, and won’t object too hard against the new ones that are only one step further away. Specifically, I would be happy to hear at least one statement of the kind “Buddhists used to believe, a decade or two ago, that meditation helps at Z, but science shows that it is not true”. Because I find it unlikely that they never make a mistake, which makes it suspicious that I never hear about one when Buddhism/meditation is discussed among the rationalists.
Second, you can’t do the math for “is spending 1000 or 10000 hours meditating worth the results?” properly, if the cost is finite, but the benefits are potentially unlimited.
(And you can notice how for people who still on some level believe in magic, the math comes out systematically better than for people who don’t. So you should expect them to be overrepresented.)
EDIT:
Doing the math properly becomes more important when you include the probability of ending up in an abusive group. Even if most Buddhist groups are harmless, their intensity probably correlates with the harm they can do, and the taking-ideas-seriously rationalists may be more attracted to those.

Viliam 2 Aug 2026 12:06 UTC
3 points
1
in reply to: sirawit’s comment on: Why is pedophilia wrong?
Technical detail: Hard bans do exist here, but they are relatively rare. What happens usually is that the existing members vote on the quality of your posts and comments, and if you keep accumulating negative score, at some levels various penalties will start to apply, such as limiting the number of comments you can post per day, etc. Getting sufficiently deep will effectively result in a ban; my point is that this doesn’t happen suddenly, but gradually.
If my values contradict with the community, then we’re simply incompatible.
Yeah, different places have different rules. Even “there are no rules” is a possible choice, and there may be places out there that approach this ideal, although I would expect that those which get too close to it end up full of spam (because “don’t post spam” is also a rule).
The dynamics essentially works like this: You want to communicate with other people. (If you don’t, you might as well install a web forum on your own computer, and you get 100% free speech and 0% rules, but also no audience.) But the other people are free to leave, if they get annoyed. And although the relation is not perfect, the people who produce better content often put stricter limits on how annoying is too annoying for them… simply because they have many less annoying alternatives available, such as publishing on their own Substack. Yes, those authors want an audience too, but they can easily get it anywhere else; if they produce sufficiently good content, the audience will follow them.
The community is what it is in part of because of the rules it has. If it had different rules, it would have different members (because some of the original ones would leave, and maybe start a new community somewhere else). It is a mistake to imagine the same community, only with different rules—that would only work for a short time.

Viliam 2 Aug 2026 9:41 UTC
4 points
1
in reply to: sirawit’s comment on: Why is pedophilia wrong?
It tests whether the community is going to endure annoying but rational thinking
Being annoying has a cost, and you seem to suggest that being rational means ignoring the costs.
Is there no way to make the same point in a less annoying way? If there is, what made you choose this one?
Do you think Socrates deserved the death penalty?
Probably yes?

Viliam 1 Aug 2026 18:11 UTC
25 points
23
in reply to: Kaj_Sotala’s comment on: The High-Control Dynamics at MAPLE
I find it weird that if I made a similar argument for Christianity (that if we strip it of the supernatural stuff, what remains are often beneficial lessons, therefore rationalists should embrace such version), people would probably start screaming at me that (1) even the non-supernatural parts are a mix of good and bad things, and sorting it out would be too much work, so it is better to start from zero instead and only adopt the provable good stuff, one piece after another, after serious examination, and (2) that human minds do not work that way; that even embracing the Christianity-without-the-supernatural too much will lead you to reading religious texts with certain connotations, and soon you will find yourself believing things that you have no rational reason to believe.
But suggest the same for Buddhism and suddenly it becomes a rational thing to do.
By the way, to get Buddhism-without-the-supernatural, we first have to agree on what the supernatural is. I mean, reincarnation seems like one of the core concepts of Buddhism, so… does your favorite version contain it, or reject it as nonsense? (Or treat it as a metaphor for… genealogy, or something?) Does it not concern you that the other ideas are coming from the same source? Wouldn’t it make more sense to abandon the entire project as hopelessly epistemically contaminated, and start from scratch instead? For example, to experiment with deep breathing, etc., without having the bottom line already written? To embrace evidence with the same lightness whether it agrees or disagrees with the Buddhist scriptures?
.
But the thing I was actually trying to do here is something like the “five whys” technique.
So the situation we got here is that there is an organization that seems quite abusive. How do I even know that it exists? Because once in a while someone mentions it on LessWrong.
Okay, so why would anyone mention MAPLE on LessWrong? I mean, there are many religious organizations out there, abusive or not. What makes this one so special that it keeps coming to my attention, even if I am not especially interested in monasteries on the other side of the planet. Seems like the answer is that because some people see it somehow compatible with the rationalist community.
Okay, why, out of all the religious organizations that exist on this planet, some people see specifically MAPLE as somehow related to the rationalist community. (As opposed to e.g. some Hare Krishna group, that I suppose also exist in the Bay Area.) I do not see the details clearly, but seems to me that there basically some members of the rationalist community are vouching for them.
Okay, but why do we take some members of the rationalist community vouching for a Buddhist monastery seriously? I mean, if someone in the community tried to vouch for a Hare Krishna group, the most likely result would be that people would laugh at them, instead of saying “yeah, if these specific Hare Krishna guys says that their prayers can protect us from the apocalypse, I guess their goals are generally compatible with the goals of the rationalist community, even if we disagree on some specific details of their epistemology”. No, people would just laugh. So why are they not laughing at the idea of a Buddhist monastery being somehow rationality-adjacent.
In my opinion… simply because it is Buddhist, and because some people are spreading the meme that Buddhism is somehow rational. More than just “you can abandon 99% of Buddhism and make something rational out of the remaining 1%”, because you could do the same thing with e.g. Freud, but that Buddhism is somehow inherently much closer to rationality. Hare Krishna is low status in the rationalist community, Freud is low status, Christianity is low status, Buddhism is high status… essentially because it is a fargroup instead of outgroup. Christianity is a religion of your boring parents, Buddhism is a religion of your cool friends. (Somewhere on a different part of the planet, people are producing anime that depicts boring Buddhist priests, and a cool Pope who is a young guy with a big sword riding a dragon.)
I try to get to the route of the cause. When a guy comes knocking on your door telling you that he has a deep wisdom that can save the world (as long as you start obeying him and stop questioning him), there is absolutely no reason to assign him higher credibility if he is a Buddhist, than if he was a Hare Krishna or Catholic or Scientologist leader. But the average Bay Area rationalist will take him more seriously, will perceive spiritual claims associated with Buddhism as more plausible, and that makes me endlessly frustrated. This used to be an atheist website, where people were skeptical about things! Including washed-down versions of religion, such as “belief in belief” etc.
I am curious about your answer to why would ever a rationalist consider a group such as MAPLE to be somehow more credible than… an analogical organization of a different religion.
EDIT:
To put it shortly: What kind of mental gymnastics makes people in the rationalist community believe that “I should join a religious monastery for a few weeks” is a rational course of action?
My answer: because we are taught by association that things related to Buddhism are somehow adjacent to rationality. I want to push back against that.

Viliam 1 Aug 2026 16:49 UTC
4 points
0
in reply to: Kaj_Sotala’s comment on: The High-Control Dynamics at MAPLE
If this take is correct, then I think it would be very surprising if AI risk happened to be the only intellectual argument that managed to ever have that effect. Rather, there are lots of arguments of the form “X is the most important thing in the world and why you should sacrifice everything on the altar of X”, with various cults being created around different versions of it.
For example, Scientology used to advertise itself as a way to prevent a nuclear war, or at least to survive if it happens anyway.
So if you believed that the nuclear war was a serious risk, then by the same logic it made perfect sense to volunteer for sleep deprivation and all kinds of abuse, even if from outside view the entire organization had 0 impact on the probability of nuclear war.