S. Alex Bradt

• S. Alex Bradt 23 Sep 2025 7:03 UTC

  2 points

  0

  in reply to: james oofou’s comment on: This is a re­view of the reviews

  https://​​ifanyonebuildsit.com/​​5/​​why-dont-you-care-about-the-values-of-any-entities-other-than-humans

• S. Alex Bradt 22 Aug 2025 6:37 UTC

  4 points

  2

  in reply to: AnnaJo’s comment on: Four ways learn­ing Econ makes peo­ple dumber re: fu­ture AI

  but even then there’s handwaving around why we’ll suddenly start producing stuff that nobody wants.

  “Stuff that nobody wants”? Like what? If you’re referring to AI itself… Well, a lot of people want AI to solve medicine. All of it. Quickly. Usually, this involves a cure for aging. Maybe that could be done by an AI that poses no threat… but there are also people who want a superintelligence to take over the world and micromanage it into a utopia, or who are at least okay with that outcome. So “stuff that nobody wants” doesn’t refer to takeover-capable AI.

  If you’re referring to goods and services that AIs could provide for us… Is there an upper limit to the amount of stuff people would want, if it were cheap? If there is one, it’s probably very high.

• S. Alex Bradt 21 Aug 2025 0:15 UTC

  5 points

  2

  in reply to: Alex_Altair’s comment on: Alex_Al­tair’s Shortform

  Yep, that all sounds right. In fact, a directed graph can be called transitive if… well, take a guess. And k-uniform hypergraphs (edit: not k-regular, that’s different) correspond to k-ary relations.

  Here’s another thought for you: Adjacency matrices. There’s a one-to-one correspondence between matrices and edge-weighted directed graphs. So large chunks of graph theory could, in principle, be described using matrices alone. We only choose not to do that out of pragmatism.

  (I’ve also heard of something even more general called matroid theory. Sadly, I never took the time to learn about it.)

• S. Alex Bradt 15 Aug 2025 4:09 UTC

  5 points

  2

  on: A philo­soph­i­cal ker­nel: bit­ing an­a­lytic bullets

  such as the Continuum Hypothesis, which is conjectured to be independent of ZFC.

  It’s in fact known to be independent of ZFC. Sources: Devlin, The Joy of Sets; Folland, Real Analysis; Wikipedia.

• S. Alex Bradt 8 Aug 2025 20:29 UTC

  4 points

  1

  in reply to: Dagon’s comment on: AI #128: Four Hours Un­til Prob­a­bly Not The Apocalypse

  Not the way I’d use those words, nope. The first is a low bar; the second is extremely high, and includes a specific emotional reaction in it. I haven’t seen any plausible vision of 2040 that I’d enthusiastically endorse, whether it’s business-as-usual or dismantling stars, but it’s not hard to come up with futures that are preferable to the end of love in the universe.

• S. Alex Bradt 7 Aug 2025 20:29 UTC

  2 points

  1

  on: AI #128: Four Hours Un­til Prob­a­bly Not The Apocalypse

  If you can’t enthusiastically endorse that outcome, were it to happen, then you should be yelling at us to stop.

  I don’t think it’s that simple. I’m not enthusiastic about transhumanism, so I can’t enthusiastically endorse that outcome, but I can’t bring myself to say, “Don’t build AI because it’ll make transhumanism possible sooner.” If anything, I expect that having a friendly-to-everyone ASI would make it a lot easier to transition into a world where some people are Jupiter-brained.

  I am quite willing to say, “Don’t build AI until you can make sure it’s friendly-to-everyone,” of course.

• S. Alex Bradt 26 Jul 2025 2:18 UTC

  5 points

  0

  in reply to: Thane Ruthenis’s comment on: straw­berry calm’s Shortform

  This comment has been tumbling around in my head for a few days now. It seems to be both true and bad. Is there any hope at all that the Singularity could be a pleasant event to live through?

• S. Alex Bradt 23 Jul 2025 0:45 UTC

  4 points

  0

  on: Google and OpenAI Get 2025 IMO Gold

  now a bunch of robots can do it. as someone who has a lot of their identity and their actual life built around “is good at math,” it’s a gut punch. it’s a kind of dying. [...] multiply that grief out by *every* mathematician, by every coder, maybe every knowledge worker, every artist… over the next few years… it’s a slightly bigger story

  Have there been any rationalist writings on this topic? This cluster of social dynamics, this cluster of emotions? Dealing with human obsolescence, the end of human ability to contribute, probably the end of humans being able to teach each other things, probably the end of humans thinking of each other as “cool”? I’ve read Amputation of Destiny. Any others?

• S. Alex Bradt 20 Jul 2025 23:31 UTC

  20 points

  2

  on: LLMs Can’t See Pix­els or Characters

  Related: LLMs struggle with perception, not reasoning, in ARC-AGI.

• S. Alex Bradt 17 Jul 2025 4:04 UTC

  3 points

  0

  in reply to: Nikola Jurkovic’s comment on: nikola’s Shortform

  Let’s not forget that the AI action plan will be on the President’s desk by Tuesday, if it isn’t already.

  I have to wake up to that every morning. Now you do, too.

• S. Alex Bradt 15 Jul 2025 9:46 UTC

  3 points

  0

  in reply to: KvmanThinking’s comment on: Kv­manThink­ing’s Shortform

  • I don’t understand the concept of “internal monologue”.

  I have a hypothesis about this. Most people, most of the time, are automatically preparing to describe, just in case someone asks. You ask them what they’re imagining, doing, or sensing, and they can just tell you. The description was ready to go before you asked the question. Sometimes, these prepared descriptions get rehearsed; people imagine saying things out loud. That’s internal monologue.

  There are some people who do not automatically prepare to describe, and hence have less internal monologue, or none. Those people end up having difficulty describing things. They might even get annoyed (frustrated?) if you ask them too many questions, because answering can be hard.

  (I wonder how one might test whether or not a person automatically prepares to describe. The ability to describe things quickly is probably measurable, and one could compare that to self-reports about internal monologue. If there were no correlation, that’d be evidence against this hypothesis.)

• S. Alex Bradt 25 Jun 2025 9:13 UTC

  1 point

  0

  in reply to: Jiro’s comment on: Con­sider chilling out in 2028

  It would be consistent with the pattern that short-timeline doomy predictions get signal-boosted here, and wouldn’t rule out the something-about-trauma-on-LessWrong hypothesis for that signal-boosting. No doubt about that! But I wasn’t talking about which predictions get signal-boosted; I was talking about which predictions get made, and in particular why the predictions in AI 2027 were made.

  Consider Jane McKindaNormal, who has never heard of LessWrong and isn’t really part of the cluster at all. I wouldn’t guess that a widespread pattern among LessWrong users had affected Jane’s predictions regarding AI progress. (Eh, not directly, at least...) If Jane were the sole author of AI 2027, I wouldn’t guess that she’s making short-timeline doomy predictions because people are doing so on LessWrong. If all of her predictions were wrong, I wouldn’t guess that she mispredicted because of something-about-trauma-on-LessWrong. Perhaps she could have mispredicted because of something-about-trauma-by-herself, but there are a lot of other hypotheses hanging around, and I wouldn’t start with the hard-to-falsify ones about her upbringing.

  I realized, after some thought, that the AI 2027 authors are part of the cluster, and I hadn’t taken that into account. “Oh, that might be it,” I thought. “OP is saying that we should (prepare to) ask if Kokotajlo et al, specifically, have preverbal trauma that influenced their timeline forecasts. That seemed bizarre to me at first, but it makes some sense to ask that because they’re part of the LW neighborhood, where other people are showing signs of the same thing. We wouldn’t ask this about Jane McKindaNormal.” Hence the question, to make sure that I had figured out my mistake. But it looks like I was still wrong. Now my thoughts are more like, “Eh, looks like I was focusing too hard on a few sentences and misinterpreting them. The OP is less focused on why some people have short timelines, and more on how those timelines get signal-boosted while others don’t.” (Maybe that’s still not exactly right, though.)

• S. Alex Bradt 24 Jun 2025 20:56 UTC

  1 point

  0

  in reply to: Steven Byrnes’s comment on: Foom & Doom 2: Tech­ni­cal al­ign­ment is hard

  Or maybe you’re saying that human hallucinations involve the “the magical transmutation of observations into behavior”?

  Right! Eh, maybe “observations into predictions into sensations” rather than “observations into behavior;” and “asking if you think” rather than “saying;” and really I’m thinking more about dreams than hallucinations, and just hoping that my understanding of one carries over to the other. (I acknowledge that my understanding of dreams, hallucinations, or both could be way off!) Joey Marcellino’s comment said it better, and you left a good response there.

• S. Alex Bradt 24 Jun 2025 4:06 UTC

  1 point

  0

  on: Foom & Doom 2: Tech­ni­cal al­ign­ment is hard

  whereas the competent behavior we see in LLMs today is instead determined largely by imitative learning, which I re-dub “the magical transmutation of observations into behavior” to remind us that it is a strange algorithmic mechanism, quite unlike anything in human brains and behavior.

  And yet...

  Well, I don’t know the history, but I think calling it “hallucination” is reasonable in light of the fact that “LLM pretraining magically transmutes observations into behavior”. Thus, you can interpret LLM base model outputs as kinda “what the LLM thinks that the input distribution is”. And from that perspective, it really is more “hallucination” than “confabulation”!

  But hallucination is “anything in human brains,” isn’t it?

• S. Alex Bradt 23 Jun 2025 21:48 UTC

  1 point

  0

  in reply to: Valentine’s comment on: Con­sider chilling out in 2028

  I’m afraid I don’t know what “the AI Futures team” is.

  I mean the authors of AI 2027. The AI Futures Project, I should have said.

  I’m not that picky about which subset of the community is making dire predictions and ringing the doom bell.

  My question was more like this: What if the authors weren’t a subset of the community at all? What if they’d never heard of LessWrong, somehow? Then, if the authors’ predictions turned out to be all wrong, any pattern in this community wouldn’t be the reason why; at least, that would seem pretty arbitrary to me. In reality, the authors are part of this community, and that is relevant (if I’m understanding correctly). I didn’t think about that at first; hence the question, to confirm.

  Is that a good-faith response to your question?

  Definitely good-faith, and the post as a whole answers more than enough.

• S. Alex Bradt 22 Jun 2025 22:01 UTC

  1 point

  0

  in reply to: Valentine’s comment on: Con­sider chilling out in 2028

  Thanks for the thorough explanation.

  Now, you mention that you “can’t think of a reason” to privilege the trauma hypothesis. And yet I gave a post full of reasons.

  You describe a pattern in the community, and then you ask if the dire predictions of AI 2027, specifically, are distorted by that pattern. I think that’s what I’m getting stuck on.

  Imagine if the AI Futures team had nothing to do with the rationality community, but they’d made the same dire predictions. Then 2028 rolls around, and we don’t seem to be any closer to doom. People ask why AI 2027 was wrong. Would you still point to trauma-leading-to-doom-fixation as a hypothesis? (Or, rather, would you point to it just as readily?)

• S. Alex Bradt 22 Jun 2025 5:01 UTC

  11 points

  1

  on: Con­sider chilling out in 2028

  One is, maybe we can find a way to make these dire predictions less unfalsifiable. Not in general, but specifically AI 2027. What differences should we expect to see if (a) the predictions were distorted due to the trauma mechanism I describe in this post vs. (b) the act of making the predictions caused them not to come about?

  If it turns out that the predictions (which ones?) were distorted, I can’t think of a reason to privilege the hypothesis that they were distorted by the trauma mechanism in particular.

[Question] Un­der what con­di­tions should hu­mans stop pur­su­ing tech­ni­cal AI safety ca­reers?

• S. Alex Bradt 13 Jun 2025 0:36 UTC

  10 points

  0

  on: When does train­ing a model change its goals?

  Back in 2015, I played ~120 hours of Splatoon in about two weeks. I had a lot of fun, and I don’t regret it. My interest eventually started to subside, and life moved on.

  Back in 2021, I played ~120 hours of Stardew Valley in about two weeks, including at least one day when I played for about fourteen hours. I regret the experience; I did not enjoy most of those hours, especially after the first few days. It took a rush of emotions and willpower for me to stop playing, but I felt deeply relieved when I did (specifically, when I deleted my save file). That wasn’t even the end of it—I ended up having similarly bad experiences with Civilization VI only a few months later. I don’t currently feel like trying to put into words how much I hated what I was doing to myself, but I’ll say that the front ends of moving trains started looking friendly.

  I eventually had to call up my mom and ask her to lock up those games (and a few others) using Steam’s parental controls. (An embarrassing solution, but it worked!) I voluntarily prevented myself from doing the thing that I had been reinforced into doing, because it went against my true values.

  Sometimes, even now, I get the urge to collect some more shiny resource blobs and make numbers go up in Stardew or Civ or similar games. Don’t recovered/​-ing alcoholics say that you never really stop recovering? I can definitely imagine that.

  Now we can ask two questions:

  1. What percent of addicts would play Murder CoD, despite knowing how it works?

  2. What percent of addicts would play Murder CoD if they also had the option to magically eliminate their CoD addiction?

  If you’d given me Murder Splatoon in 2015, I wouldn’t have even considered playing it, and I’d have been upset at you taking away my regular Splatoon. Nothing more to it than that.

  If you’d given me Murder Stardew Valley in 2021, I would have had to think about it for a few seconds, but I’m >99% confident I wouldn’t have played it. Instead, I’d have taken it as an excuse to stop playing regular Stardew, and I’d have thanked you for that excuse.

  But nonetheless, when offered a button to magically cure their drug addiction, they’re likely to press the button.

  If you’d given me a button to magically delete my urges to play Stardew Valley, but with a 0.5% chance of causing my head to explode instantly, I think I would’ve pressed it.

• S. Alex Bradt 10 Jun 2025 18:32 UTC

  1 point

  0

  on: How to help friend who needs to get bet­ter at plan­ning?

  Sure sounds like me...

  Sometimes, it helps to have read HPMOR and to ask, “What would HPMOR!Harry do?” Maybe he would step away from the problem for a moment, learn something new that might help him, and then get back to work. Maybe he would think about his actual goal, think about what tools he has available, silence the part of him that says to be afraid of bad ideas, and just try to use those tools to fulfill that goal. (Something something twelfth virtue?)