titotal

A deep cri­tique of AI 2027’s bad timeline models

When do “brains beat brawn” in Chess? An experiment

“Di­a­mon­doid bac­te­ria” nanobots: deadly threat or dead-end? A nan­otech in­ves­ti­ga­tion

Could a su­per­in­tel­li­gence de­duce gen­eral rel­a­tivity from a fal­ling ap­ple? An investigation

A new in­tro to Quan­tum Physics, with the math fixed

Most smart and skil­led peo­ple are out­side of the EA/​ra­tio­nal­ist com­mu­nity: an analysis

The bul­ls­eye frame­work: My case against AI doom

Is “su­per­hu­man” AI fore­cast­ing BS? Some ex­per­i­ments on the “539″ bot from the Cen­tre for AI Safety

Mo­ti­va­tion gaps: Why so much EA crit­i­cism is hos­tile and lazy

• titotal 23 Oct 2025 11:48 UTC

  70 points

  56

  on: The Doomers Were Right

  Last month, doomers predicted that the Rapture would happen. The doomers were wrong, as they have been all the other dozens of notable times they predicted this.

  Doomers predicted that the Y2K bug would cause massive death and destruction. They were wrong.

  Moving to your social examples, doomers predicted that the legalisation of gay marriage would lead to the legalisation of bestiality. They were wrong.

  You even provide an example yourself where people claim that D&D leads to satanism. This didn’t happen! Having an orc hero is not satanism! satanism has remained a niche religion. The doomers were wrong.

  Cherrypicking examples where doomerist predictions were wrong and examples where they were right is a very poor way to figure out whether a particular doomerist prediction is wrong or right.

  Skeptics are not arguing that no technology or social movement has ever had a negative effect, they are arguing that humans are biased towards apocalyptic thinking and overestimating the threat of new stuff. If you want to rebut this, you have to actually collect some unbiased data to test the claim.

A widely shared AI pro­duc­tivity pa­per was re­tracted, is pos­si­bly fraudulent

The Leeroy Jenk­ins prin­ci­ple: How faulty AI could guaran­tee “warn­ing shots”

• titotal 9 Oct 2023 16:13 UTC

  37 points

  29

  in reply to: Roko’s comment on: Sam Alt­man’s sister, An­nie Alt­man, says Sam has (severely) abused her

  Bayes can judge you now: your analysis is half-arsed, which is not a good look when discussing a matter as serious as this.

  All you’ve done is provide one misleading statistic. The base rate of experiencing psychosis may be 1-3%, but the base rate of psychotic disorders is much lower, at 0.25% or so.

  But the most important factor is one that is very hard to estimate, which is what percentage of people with psychosis manifest that psychosis as false memories of being groped by a sibling. If the psychosis had involved seeing space aliens, we would be having a different discussion.

  We would then have to compare this with the rate of teenagers groping their toddler siblings. This is also very difficult. A few studies claim that somewhere around 20% of women are sexually abused as children, but I don’t have a breakdown of that by source of abuse and age, etc. Obviously the figure for our particular subset of assault cases will be significantly lower, but I don’t know by how much.

  I thinks it’s highly likely that the number of women groped as a toddler by a sibling is much higher than the number of women who falsely claim to be groped as a toddler by a sibling as a result of psychosis or other mental illness, although again there is high uncertainty. This certainly makes intuitive sense: absent any other information, the most likely explanation for someone being accused of a specific crime is that they committed the crime.

  All the further evidence seems at least consistent with either scenario

  The sudden onset of the memory could be due to onset of psychosis… or it could be a repressed memory, which can also trigger at any time.

  Suppose the claim about shadowbanning is false. That could be due to psychosis, or it could be a combination of misunderstanding technology and being fearful of an abusive sibling in the tech sector. I don’t think it’s strong evidence of psychosis in particular.

  Is moving 20 times evidence of psychosis? The claimed reason is that she’s broke and had to rely on sex work for money. This seems orthogonal to psychosis.

  Is using Zoloft evidence for being psychotic? Only weakly, since she was taking Zoloft for non-psychotic reasons since her teen years.

  The claim about her dads money being withheld is only evidence for psychosis if it is false. This would be easy for sam altman to prove, and he hasn’t yet.

  I don’t think there’s enough information to be truly certain of either side, but there is more than enough information to be concerned, and to want further investigation and evidence.

Bandgaps, Brains, and Bioweapons: The limi­ta­tions of com­pu­ta­tional sci­ence and what it means for AGI

Why Yud­kowsky is wrong about “co­va­lently bonded equiv­a­lents of biol­ogy”

• titotal 24 Apr 2023 12:05 UTC

  29 points

  20

  in reply to: CarlShulman’s comment on: Could a su­per­in­tel­li­gence de­duce gen­eral rel­a­tivity from a fal­ling ap­ple? An investigation

  In the post, I show you both a grass and an apple that did not require Newtonian gravity or general relativity to exist. Why exactly are nuclear reactions and organic chemistry necessary for a clump of red things to stick together, or a clump of green things to stick together?

  When it comes to the “level of simulation”, how exactly is the AI meant to know when it is in the “base level”? We don’t know that about our universe. For all the computer knows, it’s simulation is the universe.

• titotal 10 Sep 2024 11:07 UTC

  28 points

  11

  on: AI fore­cast­ing bots incoming

  I asked the forecasting AI three questions:

  Will iran possess a nuclear weapon before 2030:

  539′s Answer: 35%

  Will iran possess a nuclear weapon before 2040:

  539′s Answer: 30%

  Will Iran posses a nuclear weapon before 2050:

  539′s answer: 30%

  Given that the AI apparently doesn’t understand that things are more likely to happen if given more time, I’m somewhat skeptical that it will perform well in real forecasts.

• titotal 8 Mar 2023 14:41 UTC

  28 points

  30

  on: Ar­ti­cle about abuse in LessWrong and ra­tio­nal­ist com­mu­ni­ties in Bloomberg News

  A lot of the defenses here seem to be relying on the fact that one of the accused individuals was banned from several rationalist communities a long time ago. While this definitely should have been included in the article, I think the overall impression they are giving is misleading.

  In 2020, the individual was invited to give a talk for an unofficial SSC online meetup (scott alexander was not involved, and does ban the guy from his events). The post was announced on lesswrong with zero pushback, and went ahead.

  Here is a comment from Anna Salamon 2 years ago, discussing him, and stating that his ban on meetups should be lifted:

  I hereby apologize for the role I played in X’s ostracism from the community, which AFAICT was both unjust and harmful to both the community and X. There’s more to say here, and I don’t yet know how to say it well. But the shortest version is that in the years leading up to my original comment X was criticizing me and many in the rationality and EA communities intensely, and, despite our alleged desire to aspire to rationality, I and I think many others did not like having our political foundations criticized/​eroded, nor did I and I think various others like having the story I told myself to keep stably “doing my work” criticized/​eroded. This, despite the fact that attempting to share reasoning and disagreements is in fact a furthering of our alleged goals and our alleged culture. The specific voiced accusations about X were not “but he keeps criticizing us and hurting our feelings and/​or our political support” — and nevertheless I’m sure this was part of what led to me making the comment I made above (though it was not my conscious reason), and I’m sure it led to some of the rest of the ostracism he experienced as well. This isn’t the whole of the story, but it ought to have been disclosed clearly in the same way that conflicts of interest ought to be disclosed clearly. And, separately but relatedly, it is my current view that it would be all things considered much better to have X around talking to people in these communities, though this will bring friction.

  There’s broader context I don’t know how to discuss well, which I’ll at least discuss poorly:

  Should the aspiring rationality community, or any community, attempt to protect its adult members from misleading reasoning, allegedly manipulative conversational tactics, etc., via cautioning them not to talk to some people? My view at the time of my original (Feb 2019) comment was “yes”. My current view is more or less “heck no!”; protecting people from allegedly manipulative tactics, or allegedly misleading arguments, is good — but it should be done via sharing additional info, not via discouraging people from encountering info/​conversations. The reason is that more info tends to be broadly helpful (and this is a relatively fool-resistant heuristic even if implemented by people who are deluded in various ways), and trusting who can figure out who ought to restrict their info-intake how seems like a doomed endeavor (and does not degrade gracefully with deludedness/​corruption in the leadership). (Watching the CDC on covid helped drive this home for me. Belatedly noticing how much something-like-doublethink I had in my original beliefs about X and related matters also helped drive this home for me.)

  Should some organizations/​people within the rationality and EA communities create simplified narratives that allow many people to pull in the same direction, to feel good about each others’ donations to the same organizations, etc.? My view at the time of my original (Feb 2019) comment was “yes”; my current view is “no — and especially not via implicit or explicit pressures to restrict information-flow.” Reasons for updates same as above.

  It is nevertheless the case that X has had a tendency to e.g. yell rather more than I would like. For an aspiring rationality community’s general “who is worth ever talking to?” list, this ought to matter much less than the above. Insofar as a given person is trying to create contexts where people reliably don’t yell or something, they’ll want to do whatever they want to do; but insofar as we’re creating a community-wide include/​exclude list (as in e.g. this comment on whether to let X speak at SSC meetups), it is my opinion that X ought to be on the “include” list.

  Here is Scott Alexander, talking about the same guy a year ago, after discussing a pattern of very harmful behaviour perpetrated by X:

  I want to clarify that I don’t dislike X, he’s actually been extremely nice to me, I continue to be in cordial and productive communication with him, and his overall influence on my life personally has been positive. He’s also been surprisingly gracious about the fact that I go around accusing him of causing a bunch of cases of psychosis. I don’t think he does the psychosis thing on purpose, I think he is honest in his belief that the world is corrupt and traumatizing (which at the margin, shades into values of “the world is corrupt and traumatizing” which everyone agrees are true) and I believe he is honest in his belief that he needs to figure out ways to help people do better. There are many smart people who work with him and support him who have not gone psychotic at all. I don’t think we need to blame/​ostracize/​cancel him and his group, except maybe from especially sensitive situations full of especially vulnerable people. My main advice is that if he or someone related to him asks you if you want to take a bunch of drugs and hear his pitch for why the world is corrupt, you say no.

  I don’t think X is still part of the rationalist community, but these definitely make it look like he is welcome and respected within your community, despite the many many allegations against him. I’ll end with a note that “Being ostracised from a particular subculture” is not actually a very severe punishment, and that maybe you should consider raising your standards somewhat?

Slop­world 2035: The dan­gers of mediocre AI

• titotal 1 Oct 2023 15:52 UTC

  21 points

  15

  in reply to: Veedrac’s comment on: “Di­a­mon­doid bac­te­ria” nanobots: deadly threat or dead-end? A nan­otech in­ves­ti­ga­tion

  At no point did I ever claim that this was a conclusive debunking of AI risk as a whole, only an investigation into one specific method proposed by Yudkowksy as an AI death dealer.

  In my post I have explained what DMS is, why it was proposed as a technology, how far along the research went, the technical challenges faced in it’s construction, some observations of how nanotech research works, the current state of nanotech research, what near-term speedups can be expected from machine learning, and given my own best guess on whether an AGI could pull off inventing MNT in a short timeframe, based on what was learned.

  This is only “broadly non-sequiter” if you think that none of that information is relevant for assessing the feasibility of diamondoid bacteria AI weapons, which strikes me as somewhat ridiculous.