Who should write the definitive post on Ziz?
CW: Possible infohazards, definitely discussion of infohazards.
EDIT 25 Feb 2023: This post has more details and corrects some things this post got wrong. Consider it as superseding this post.
Backstory
I was writing a comment on this post, and I was going to end it with this… but then decided to turn it into a full Question Post.
The post basically says something like “being in intense emotional doomer mode w.r.t. AI safety is counterproductive and addictive”, which is broadly agreeable. However, the post also has a framing (my kinda uncharitable but also defensible reading) that’s something like “this is caused by mind-virus stuff and you’re trapped in the matrix and you need to go cold-turkey on caring emotionally about it, also if you disagree you are mindkilled. Go take a deep breath, then come back and agree with me”.
So I was gonna write a comment like:
“Also like Richard_Ngo said, this framing seems kinda… bad. I don’t think this is a direct inspiration for you, but [word phrase chunks to somehow relate the central themes/vibes to] posts by Ziz, and Scott Alexander’s response to somebody else talking about CFAR and Michael Vassar (and Levarage?).
Someday I hope to write an extremely long thorough post (and possibly accompanying video essay?) about the whole Ziz/memetic-virus/hemispheres/alleged-conspiracies thing. It will both chronicle The Gigantic Drama and also respond to… Ziz’s… entire worldview? Something like that. However, this project by nature would be extremely labor-intensive, and it’s not my top priority. (Perhaps someone else wants to pick it up?)”
Then I realized it should be a post.
What are you talking about? (The Actual Backstory)
CFAR. Conspiracy theories. Mental illness. Using different hemispheres of your brain as if they were different people. Tumblr blogs. mumble mumble Nick Land mumble meme-virus. Michael Vassar. Possibly multiple suicides, including maybe Ziz.
“That’s not a post idea! That’s a list of keywords, some with emotional connotations!”
There are ideas in here, but they are tied to a gigantic ideology, known to very few. There are events in here, but they are jumbled out of order, and accounts conflict. There are people in here, and some of them are even important in the rationality community.
Maybe it’s masochistic epistemology, maybe it’s the social brain flaring up… but I kinda care about ideas that would “hurt” if true (modulo That One Litany About Not Worrying About That). I’ve seen other ideas where it seems wrong but I haven’t seen a comprehensive definitive examination of it, on LessWrong specifically, written by someone who’s read and likes The Sequences. Plus, I don’t prioritize it enough to think it through all the way myself. (Related to working memory / ADHD: literally forgetting counter/arguments that might be relevant to a given sub-argument.)
I feel this with a few ideas: disagreement, but with a nagging sense that it’s true and I should just devote my life to it already, coupled with lack of satisfactory counterarguments. The labor theory of value, and “exploitation theory” as a whole. Full-throated anarchism. Veganism, before I switched from vegetarian to full-vegan. Any vaguely leftist reddit comment, if righteously angry enough. “Eat the rich”, before I developed a system of interlocking beliefs about why that’s bad, actually, which I see shockingly few people go in-depth about. (I’ll write more on that someday, but the answer involves mental health and increasing the moral circle to include sociopaths.)
And now, the Ziz worldview. Which among other things,seems to say that (CONTENT WARNING: THIS IS THE PART WITH THE INFOHAZARD) we should create Roko’s-Basilisk-style simulated torture for people doing bad things now.
Should I agree with this, like I eventually did for going vegan? Or should I disagree, like I’ve done for leftist calls for violence against the rich?
And also, what did happen to all those people in the story? In real life, I mean. Are they okay? Should I change who I trust about rationality community matters? Are some of these things basically unrelated except by “vibe”, or are they linked by something substantial?
Call to Action
Who wants to write this post? Who should write this post? In my head, it’d be someone who’s both respected in the community, but somehow also an unbiased outsider who can find the truth without being caught up in politics, who also has lots of free time and some academic-level research ability, who is also good at the art of rationality in terms of actually engaging with the ideas involved.
(No, I’m not “subtweeting” anyone in particular. I really don’t know who, if anyone, fits this description. Also I hate subtweeting. (Okay, and social reality in general, but still.).)
If you volunteer to write, DM me and I can send you lots of links (to things I haven’t fully read) and extremely rough notes.
(While researching this, I found this webpage, which seeeems close-ish to what I want, but lacks some of the extra community backgroudn details that, depending on your POV, are either extraneous or fascinating, or even needed! So this post is also a referendum on what LW thinks of that post.)
So… what is to be done? Who should write it? Should it be written? What should be written?
I can’t vouch for this, but it sure seems interesting: https://medium.com/@sefashapiro/a-community-warning-about-ziz-76c100180509
I wonder if rationality makes someone a more efficient villain. Theoretically it would.
Though it sounds like ziz had a long history of what is obviously irrational off the rails behavior we can observe, regardless if they committed any of the major crimes alleged.
Back in the days of High Rationalitlsm, it was very much believed that rationality would make people hugely more effective...yet there was no screening of who these dangerous secrets were taught to.
I have only heard rumors about Ziz and her destructive effects on aspiring rationalists within her impact radius. Apparently young rationalists are very susceptible to the dark side of the force. Who could have thought.
The Zizians post reads like any old cult beginning: brainwash, remove the ethical guardrails, isolate the flock, intimidate, use mind-altering techniques like sleep deprivation and trance. Fortunately it appears to fizzle before taking hold. Unfortunately, people died, and probably quite a few are still mentally scarred.
I am guessing that the real lesson here is not for the susceptible youngsters, but for the wise elders who watched it all and didn’t do much: if you notice an issue like that, speak up, take action, and protect the potential victims. Otherwise, what is your high morality good for?
Chris* Pasek didn’t appear to me like a susceptible youngster. At least not in the way that college students are. I would estimate that he was around ~30. He worked for some years in Japan. He seemed internally aligned being able to work on projects without procrastinating.
He also wasn’t really isolated in the way you see in most. He lived in a group house with other rationalists for most of the time and his contact with Ziz was mainly online.
“I’m using the name Chris, because that’s how I got to know him, he transitioned after being in contact with Ziz”
Figuring out this kind of risk is probably something that people only learn after they actually witness something bad happen. (Until then, it just feels like bias against doing unusual things.)
Reading about it does not help much, because (1) System 1 versus System 2; when it actually happens to you, it feels different; and (2) there is always some technical difference between the situation you read about and the situation that happens in your neighborhood, which can be used as an argument that “this is different”, until you actually get burned and then you see how the similarities were sufficient and the differences were superficial. For example, once you make the analogy to cults, someone is guaranteed to object “but Ziz is not religious”, etc. Also, contrarianism is high status, common wisdom is low status; talking common sense among clever people is totally inviting them to get status points by mocking you.
Before the bad things actually happen, “a wise elder warning about dangers” is merely “an old man yelling at clouds”. I could totally see myself in that role if I lived in Bay Area, and I would willingly take the status hit, but I am not sure it would actually change the outcome.
strong upvote, strong agree—very important point! I’d add:
not only the ones with a self-perception of moral high ground should be on the lookout; people probably ought to know what cultiness looks like, so as to be somewhat immunized against it. of course, that is itself a conflicted discussion point, as groups that don’t want to be identified as cults object to such things. there’s a pretty strong consensus on what cultiness looks like, though, and browsing several search engines on “what is cultiness” finds great anti-cult documentation:
https://medium.com/@zelphontheshelf/10-signs-youre-probably-in-a-cult-1921eb5a3857
https://www.culteducation.com/warningsigns.html
http://www.ex-cult.org/fwbo/CofC.htm
https://www.microsolidarity.cc/articles/cults (this one seems cool. before losing it trying to satisfy forced perfection, maybe she would have liked this idea...)
https://www.lesswrong.com/tag/cults (I added this tag to this post; I think it’s important to understand that that’s pretty much what this instance was)
https://faenrandir.github.io/a_careful_examination/asking-if-its-a-cult-is-wrong-question/
https://cultrecover.com/cultdef
https://cultrecovery101.com/faq/
https://cultrecovery101.com/cult-recovery-readings/checklist-of-cult-characteristics/
I’d be curious of how much of those apply to Ziz’s group.
The leader is the ultimate authority—If you’re not allowed to criticize your leader, even if the criticism is true, you’re probably in a cult.
+0.5
Appears somewhat applicable to me from a distance; I’m not sure how obvious it would have been up close, because many types of criticism would likely have been apparently welcome, but there was a large space of things where criticism would not only result in ziz disagreeing, but in anger at the topic coming up. Since many folks [edit: not everyone, that was wrong, but definitely most folks] have topics that will anger them when they come up, I’m not sure how to draw an unambiguous line on this one, but I think there’s something to do with whether the anger is endorsed as a pressured demand to change away from anything that involves questioning.
The group suppresses skepticism—If you’re only allowed to study your organization through approved sources, you’re probably in a cult.
+0.5
Again, appears applicable about the key philosophy. Anyone who doesn’t accept their definitions is evil.
The group delegitimizes former members
+0.5
Not sure there were any former members to delegitimize?
The group is paranoid about the outside world
+1
Obviously becoming a common warning sign given the number of real threats to the world, but overconfidence about them is still a serious warning sign, and they very much were using intense emotional activation about end of the world as a key selling point. Remaining sober when there are risks on the horizon is always key to defeating those risks, so it’s important to not fall into a habit of overactivation—that was how this whole post was started in the first place, after all.
The group relies on shame cycles
+1
YUP. “if you’re not with us, you’re against us”, lots of stuff about having wrong thoughts being shameful.
The leader is above the law
+0.5
Well certainly the literal law, unclear if ziz was in the habit of breaking their own principles internally. weaker match.
The group uses “thought reform” methods—If your serious questions are answered with cliches, you’re probably in a cult.
+1
ohhh boy yep that one applies hard.
The group is elitist—If your group is the solution for all the world’s problems, you’re probably in a cult.
+1
yeppppppp
There is no financial transparency
+0.5
Well I don’t know about that one. Not sure how to know, really.
The group performs secret rites
+1
Seems like it!
overall score on this thrown-together hand-calculated hunch-based metric: 7.5
I think there are better ways to score this, but it given the fact that I can’t say “definitely not” to any of them and can easily say “definitely yes” to half of them, I think it’s a pretty solid “yeah that looks like a cult by this description”.
This list of Delaware County Jail Inmates and this database of Pennsylvania Inmates does not contain LaSota, so I think the claim is false.
Thank you once again for the correction.
Thank you for the information, very helpful if true.
Could you say how you know that, or provide a link? Do you know if one could call the prison and ask and have it confirmed?
As someone who loves to do a little vexing, I have probably already spent far more than is a healthy amount of time studying and writing about Ziz over the years, and have had an unfortunately close sidelong relationship with some of their group for an extended period. But (ahem) “now that the author is dead it’s all dead un-adapting information for me to make “antibodies” from.” So that’s what I’ve been doing lately. I’ve in a sense already started writing the post you want, more for my own personal closure than anything else, but you’re correct that it’s a large large project.
I probably turned Ziz into something of a kismesis when I finally did my heel face turn away from the EA brainworms that were eating me, so I know that actually yes they are as bad as they seem. The current death toll for folks deep in the memeplex is five, with another currently under arrest for murder charges. That is an insanely high KDR for a group their size.
Something I think does need to be said, is that having taken all of Ziz’s blog content and stacked it chronologically end to end, it reads completely differently than it does in its original context. Divorced from the wiki-style links connecting everything, it goes from being a strange and unhinged manifesto to something of an apocalypse log, a found footage horror story about a brilliant person slowly breaking and going insane from the hostility of her local environment. At each step and time jump you can see the wheels come off the rails a bit more, finally culminating in the 80 page long expose of the entire rationalist community that is “intersex brains and conceptual warfare.”
I note that because when I look at the story of Ziz’s destruction spiral, not just the damage she did and who all was caught up in it but the beats of the story and how her own state was affected at various points in the spiral, I have to say, it’s not all on her. She is not the sole creator of the brainworms that consumed her, she came by them naturally and organically while participating in this community. Ziz is simply the loudest and most visible casualty of the same sort of mental knot that has consumed lots of other EAs at this point as well.
Zizians is a moderately decent if biased starting overview of her content but it fails to really engage with the actual teeth of the material very much and if you’re going to take its word for things then you should also read the rebuttal. And yes you are wading into literal years worth of drama and back and forth attempts at character assassination by various individuals involved in this mess, so lol, beware. Feel free to DM me if you want to talk about this.
Thorough reply, thank you!
If you want, I could DM you about sending you my existing notes on the matter. (I’ve already sent them to at least one other person, but I don’t think that should matter in this case?).
Feel free!
Analyzing whether EA is a cult is really interesting, so I will give it a shot:
Here’s a link to signs of cults:
https://medium.com/@zelphontheshelf/10-signs-youre-probably-in-a-cult-1921eb5a3857
For EA
The leader is the ultimate authority
0.5, leaning towards 0 here. I think this might be happening with Will Macaskill, but for the most part EA has a semi-healthy relationship with it’s leaders.
.
The group suppresses skepticism
This one is a 0, for the most part. EA isn’t perfect at getting and integrating criticism, but it is far better than most organizations at criticism.
The group delegitimizes former members
Again, more or less a 0. I haven’t seen this behavior from EA.
The group is paranoid about the outside world
0.5. Ultimately, I actually don’t think EA is fearful about the outside world, but X-risk and to a lesser extent longtermism being at least considered prevents it from being a 0.
The group relies on shame cycles
Again, a 0 fits here. Thankfully, EA doesn’t rely on shame cycles, nor have leaders used shame as a weapon in the way the post talks about.
The leader is above the law
More or less a 0 for this one. Yes, women have been assaulted in EA spaces, but EA more or less doesn’t make excuses for that.
The group uses “thought reform” methods
More or less a 0, as far as I can tell.
The group is elitist
I’m giving this a 0.5-1 as far as this is concerned.
There is no financial transparency
0.5 is I think the rating I’d give.
The group performs secret rites
Definitely a 0 for this criterion.
So overall it’s 1.5-2.5 depending on how you shake out the randomness. This is an okay sign for EA, for the most part. It has a few elements of cults, but not many elements of them.
You don’t know about the secret rites? They must be too secret.
To those reading the recent comments: I have unvoted this comment; please don’t read it if you aren’t ready for some of humanity’s more difficult topics. It’s… not a fun time.
wow that post sure is something. I think I mostly find it good enough and don’t feel the need to call for new posts like it to clarify at this time, though I hope the history will exist to be resolved in further detail later on. but wow, you’re not kidding about the content warnings; it’s a sad history, and I’m glad I knew what it was going in. I might have called it a content warning instead of an infohazard, I think it’s only an “infohazard” if one is highly credulous about it. It certainly contains references to ziz’s impassioned speech.
the post walks through some … some stuff. I sure understand her perspective about a lot of it, but phew, some of that is a lot… lot. severe misunderstandings of game theory in practice, false assumption that everyone is rational, severe misunderstanding of why weight personhood of ants (I do think they have nonzero personhood! but … humans all have almost exactly the same amount of personhood in relative terms, but I’m pretty sure it takes an anthill to even approach the personhood level of a single mouse.) I agree with the direction of some of her criticisms of those orgs! but PHEW she took it way too far and, like, idk, seems like she violated her own principles without realizing because of radicalizing too hard? it’s also interesting how traditional her radicalization seems to have been. Seems like [edit: comment from a friend off less wrong: the aesthetics of] destructive anarchism without sufficient praxis or constructive anarchism to make up for it [and with a high degree of actual authoritarianism seeded by the authority basilisk’s authority]; I’m personally not really a fan of destructive anything, in my view vegans should be pragmatic about how to permanently end the death and torture industry. I sure agree that only a vegan singularity would be worth it, but I think we can also do away with post-hoc torture as part of that. like… dang, I hope she doesn’t get to ruin everything she believed, because phew, she had a lot of kinda based takes, but the ones about “never surrender, no matter what” are just… just… dang.
and especially hearing what’s happened with some of those folks in the past few months, is just. they didn’t deserve to end up being the kind of people who would end up on this path… none of them did. that’s always how I feel about those who cause serious harm. but also, from what I’ve heard of recent events, I … understand how it happened… they did things nobody should ever do, trying to protect ants… talk about making a mountain out of a molehill? some of those people were once my friends, including her, and I never understood until reading this what decisions made them choose to turn against society quite so hard. I don’t think I can ever encourage those choices, but I also do think it’s wise to understand why they, and a great many radicals over the long history of human radicalism before them, made the choices they did. And the fact that there are better, more ethical ways to end the forms of suffering they’d chosen to. If only they’d understood boundedly rational game theory enough to be willing to forgive those who are products of their time, and instead aim for a singularity that offers society improved bargaining algorithms and an end to material scarcity for those who did the work...
I do think she’s somewhat right about some forms of liberalism. If your liberalism isn’t always pushing at the minarchist edge, your coprotection algorithms aren’t seeking to heal. But for the same reason, I hope those of them who are still alive survive to see a future far better than the one they hoped to create...
damn...
I feel like I should downvote this post just to keep it off the front page. I certainly hope there are no more than about five comments on it, personally.
Channeling my inner Jordan Peterson, life is a combination of creating good things and destroying bad things. If you only do the latter, the good things are not going to appear magically. Instead you will find yourself surrounded by ruins. The thing you destroyed will be replaced by… whatever is the baseline… which is probably not as good as you hoped for, and often much worse than the thing you destroyed.
Judging by the outcomes… hey, I may be getting filtered information, but can anyone tell me one good thing Ziz ever did? (No, yelling at “evil” people does not count, especially from someone who considers virtually everyone evil.) All I see is people getting turned to zombies, some of them killing themselves afterwards.
I had to read the story again, but Ziz didn’t actually save those ants. They merely considered saving them… but then did not. So much for the only doubleplus-good person in the entire universe!
(It’s just everyone else who kills animals, who deserves to be tortured for eternity.)
A nirvana fallacy personified. Or rather, a deeply narcissistic person.
wish Petersen would stop considering people bad things to destroy. but yeah, reasonable component of a good take.
the main actual could-have-been-good material action thing in my view is the off grid living attempt. it instead seems to have amplified their isolation. I do think their commentary about how everyone needs their own spaceship is on the right track; the train of thought derails shortly after that. everythingtosaveit.how actually came up while I was looking for off grid living stuff.
but I don’t disagree with your assessment at all, and my comment wasn’t intended to. I’m mourning the failure at the things they said they wanted, by nature of the things they actually did.
Ziz’s writing was tremendously helpful to me, even with as much as it also messed me up and caused me to spiral on a bunch of things, I did on balance come out better for having interacted with her content. There are all sorts of huge caveats around that of course, but I think to dismiss her as completely bad would be a mistake. After all
Frontpaging a post is a moderator decision. I don’t think this post will be frontpaged no matter it’s karma.
err. there are two types of front page, and I actually meant “keep it off the front page of personal blogs”.
From how I have know Chris Pazek (one of the people who committed suicide) he didn’t seem highly credulous to me. There’s a sense of him being open to experimenting with weird ideas but credulous seems like the wrong word.
it seems that being highly credulous can be induced by skilled manipulators for more people than one might expect, such as by using sleep deprivation
Yeah, the actual danger seems to be exposing yourself to strong social / mental / emotional pressure while sleep deprived, rather than some baseline credulity.
Some people probably underestimate how vulnerable they can feel when they are tired and surrounded by people who suddenly turn hostile if they oppose their brainwashing. From outside, it probably seems like “hey, I could walk away any time I want to; and even if they tried to stop me, it might be unpleasant, but they wouldn’t literally kill me, so after some time they would have to let me go anyway”. But when you actually are in that situation, sleep deprivation makes you too weak to resist physically or figure out a clever escape plan, and every minute feels like eternity. Your instincts are screaming at you that you are completely helpless, your life is in their hands, and if you want to survive, you need to appease them, and if the only thing that can appease them is sincerely adopting their ideology, then just fucking do it.
Still, given enough time and interaction with other people, you could probably snap out of it. But you are not going to get that time, because you must immediately start working hard to save the world, and besides, everyone else is evil, so you rather not talk to them lest their evil contaminates your soul.
It’s more complex than that. Developing an alternative personality is more complex than just adopting ideology, not resisting physically, or thinking about escape plans.
There’s also the belief involved that a person who says they are trans should not be questioned in their identity and questioning them for that identity choice is evil.
so that’s something I’d push back on gently. I can understand the concern, id prefer to see both names used if you’re highly concerned about that. I’ve -karma +agree voted your comments with only that person’s old name and I think someone else had too.
How much of the underlying details of what went on do you know and how much are you pattern-matching from other situations?
It seems to me inconsequent to say that sleep deprivation was a problem here but at the same time argue that the effect of it shouldn’t be questioned.
agree vote. I’d met several of them. I’m just saying that of the reasoning errors, erroneous transition is the one I’d express the most uncertainty about. I wouldn’t object to you using the name Chris, just to not respecting the possibility that that might have been real. enough of them were actually trans women; I wouldn’t be surprised if this was ziz using a person’s genetic predisposition for transition to manipulate them, for example, in which case they probably would have eventually transitioned anyway once it became convenient in normal life. I would remove karma downvotes if you were to express uncertainty. it’s ok to question, but it seems like concerning jumping to conclusions to assert you know for sure.
Most trans women you meet likely didn’t identify highly that way because of a high invasive psychological process involving sleep derivation with the intent of awakening an internal part that has its own identity.
If you think that this process just surfaced what’s already there, it’s odd to call the sleep deprivation manipulative. It seems like you have certainty that the sleep deprivation was manipulative, but don’t take the logical conclusion that the result of it is problematic.
This is belief in belief. Belief in the meme of some platonic realness. It’s a thought-stopper. The fact that this is a potent tool for stopping people from engaging in critical thinking is one of the dynamics that Ziz used to protect against criticism.
This is a story about a bunch of naive people who fell for Ziz. It’s about highly sophisticated intelligent people doing so, and the tools used to get people to stop to think are at that level.
It’s also about the complex process of inducing alternative personalities. I have an idea of how that works in hypnosis and can extrapolate from that to how the process Ziz describes for splitting people’s identity into two (supposedly one for each hemisphere) could do that whether or not Ziz’s idea of bihemispherec identity is correct.
If you want one example of someone having a problem with induced personalities take this reddit thread. Do you call the guy his legal name or do you call him Bambi? If the Bambi personality wins out gets a lot of power and then commits suicide, should you refer to him by his legal name or by Bambi when you speak about him?
I don’t want to keep defending this because you’re almost entirely right and as a trans person myself I just personally cringe at the idea of being certain that someone in distress was Actually Not Trans Because They Were Manipulated. that’s… really the extent of it. We won’t reach a resolution on this because it doesn’t matter, the key point is there was intense manipulation and a split personality was intentionally created by manipulation. Whether that person was truly trans underneath is unknowable and I am quite willing to admit the reason I’m cringing against refusing to acknowledge that, but it really is mostly about those besides the person in question, so … shrug. If they’d been a woman to start with it would still be just as bad.
I’ve turned my downvote into an upvote because −6 seems too low for just this. The fact that people’s self-expression can be manipulated does in fact make it ambiguous whether that occurred or whether there was real self-expression. some people really are bullied. it’s also extremely likely that a group of trans people collected people who seemed at a minimum naturally inclined to not be cis, so I am inclined to bet that this person really had some gender stuff going on. regardless, ziz destroyed them, and chris or maia, we must mourn the one known as pazek.
I would in fact bet that the sleep deprivation thing made them more inclined to transition when they did, but would personally roll to disbelieve that it was the only cause of transition. That’s really it.
Is there a missing “not” in the first sentence?
ETA: Or a missing “not only” in the first and an “also” in the second.
have you read Maia’s suicide note? Because it has a lot of details.
I’m not sure what you mean with their suicide notice. They wrote a post on their blog arguing for life being meaningless because all the memories being forgotten after death anyway. Do you mean that document?
I read that at the time it was posted and there was time between that document and the actual suicide. I got to know about the suicide itself a substantial amount of time later and got deeper information by talking to one of their roommates. The version of events that their roommate told me did not include Ziz. Later, I read Ziz account and assume that Ziz had no good reason to lie about a lot of the involved details as they don’t make Ziz look good.
If there’s a separate suicide note, I’d be happy to read it.
(re:linked document) I don’t think suicide note is a good name for that blog post as it was written some time before the actual suicide.
It also contains no details on the personality conflict.
Any post on the Ziz cult that does not engage with the already explicitly cultic mileu of singulatarian thought from which they emerged (that is, here) is utterly incomplete.
Fair enough, but it is equally incomplete to pretend that that’s an argument against the possibility of singularity-grade technology emerging in the foreseeable future.
By analogy, there have been many people who had crazy beliefs about radioactivity: doctors who prescribed radium as medicine, seemingly on the grounds that it was cool, and anything cool has to be good for you right? (A similar mentality led some of the ancient Chinese to drink mercury.) Atomic maximalists, who thought that anything and everything would get better with a reactor strapped to it, and never mind the price of uranium, the need for radiation shielding or the fact that reactors are heavy due both to the need for cooling and power generation systems and the simple fact that they benefit greatly from economies of scale. Not the sort of thing that you necessarily want to bolt onto every car and aircraft! Atom-phobes who were convinced that any attempt to utilize nuclear power would automatically become the next Chernobyl.
All of these were crazy, cult-like beliefs. Yet the insanity of people who turned poorly-understood scraps of nuclear theory into unreasoning optimism or pessimism does not have a single thing to say on the reality of radioactivity. Atomic bombs and nuclear reactors still work, no matter how foolish the radium suppository crowd of the early twentieth century was. And they still have sharp limits, no matter how crazily enthusiastic the “atomic cars in twenty years” crowd was.
By all means point out how Ziz’ cult was influenced by singularitarian ideas here. Even point out how the great opportunities and risks that a singularity might bring are a risk factor for cult-style mistakes. But don’t pretend that that prevents advanced technology from existing. Nature simply doesn’t care how we think about it, and isn’t going to make AI impossible just because Ziz had foolish ideas about AI.
But we also can’t pretend that this place is anything but a less extreme spiraling cult of its own, rather than a place that has anything to do with the real world
Commenting with my mod hat on:
I think it’s important to be able to point out dynamics that feel unsettling and unhealthy without necessarily being able to pin down exactly what’s wrong. I think it can often be hard to point out what’s wrong, and demanding people explain themselves well in public is often unfair.
That said, the circumstances that normally make me willing to cut-slack-for-bad-argumentation here don’t obviously apply to CellBioGuy.
I also think it’s important to not reflexively shut down people disagreeing with the LW site vague-community-consensus.
That said, the comments here just pretty clearly don’t pass muster as acceptable LW comments in the general case. The amount of slack I’m willing to cut them isn’t infinite (and basically has been used up by this point). If you want to repeatedly argue we’re just a spiraling cult you do need to, at a minimum, supply any argumentation whatsoever, and engage with some object level claims. It’s easy to toss scary sounding labels around.
I liked gears to ascension’s reply, which sort of took seriously the concerns CellBioGuy was pointing at while trying to say something substantive about it.
You’ve been saying a lot of things in this reference class in the past week (i.e. vague cult accusations without making substantive claims), and if that keeps up we’ll probably take some kind of mod action about it, although I haven’t thought it through that much at this point.
Citation very much needed. What, specifically, do you disagree with?
Do you believe that the human mind is magical, such that no computer could ever replicate intelligence? (And never mind the ability it has shown already from chemistry to StarCraft…)
Do you believe that intelligence cannot create better tools than already exist, such that an AI couldn’t use engineering to meaningful effect? How about persuasion?
Do you believe that automation taking over the economy wouldn’t be a big deal? How about taking over genetics research, which is often bottlenecked by an inability to consider how genes interact, precisely something a computer could help with? Or is learning how to alter our very cores no big deal?
Do you have a specific argument against the plausibility or significance of a singularity? Or is this simply pattern matching to a cult without any further thought? Because “this sounds weird; it must be wrong” simply doesn’t work. Flight, nuclear power, genetics-all sounded more like science fiction than any real world possibility.
I think that the vibes here have been unhealthy in ways that made model-challenging evidence seeking harder to think of; for example, yudkowsky’s intense anxiety, dying with dignity needing to be said and also that shock inducing name, etc. Your reaction seems to me to miss the point that I would agree with CellBioGiy about, though I do also think you’re right in every implied rebuttal your questions point to; it’s just that trying to solve safety requires willingness to question deep assumptions and yet retain stability in the face of doing so, and retaining stability is hard and often failed at when taking huge magnitude ratios seriously. Ziz seems to me to have been the most intense example of a general failure pattern in how this intercommunity directs a person’s intention, and thinking about that carefully seems important to me. It’s also something where real progress has been made and I hardly think the critique warrants writing off the site as a whole or anything like that.
I do think calling the whole thing a spiraling cult takes it too far; it’s got elevated levels of cult disease and features that can’t be removed which make it pattern match aesthetically to cult adjacency in ways that may not always be fundamental—real crazy things are coming—but I think that there are in fact concerning thought stopping patterns in the interpersonal anxiety patterns, patterns of who trusts who without ongoing verification, etc.
Does the same apply to nuclear war or global warming?
If there is an organization that claims to fight against nuclear war, does many weird things, and results in a suspicious number of suicides (for example Scientology), would it also be utterly incomplete to describe such organization without engaging with the nuclear war coverage in mainstream media?
I think you may be confusing stated beliefs with group dynamics. (Related: The Ideology is not the Movement.)