Shmi

Karma: 29,012

Shmi 18 May 2026 0:05 UTC
2 points
0
in reply to: StanislavKrym’s comment on: Why I am not too worried about AIpocalypse: Scott Alexander vs Nicolaus Copernicus
Well, we do know that interstellar travel is possible, just currently very hard. It’s a technological issue, not a physical one. I can certainly imagine slow travel, a few thousands km/s, being possible with nuclear rocket propulsion or other tech. Getting a spaceship-size object traveling at an appreciable fraction of the speed of light is a different story, anything like that would create observational artifacts visible from far away, so those probably aren’t anywhere in the observed past light cone. But I am not sure how this is related to the evaluation of the odds of AIpocalypse.

Shmi 17 May 2026 23:54 UTC
2 points
0
in reply to: RHollerith’s comment on: Why I am not too worried about AIpocalypse: Scott Alexander vs Nicolaus Copernicus
All good questions! We might be the only recognizably technological civilization out there, any other “life” might occupy something we cannot even tell is life.
I guess my point is that, provided we are not the only ones like us to begin with, concluding that we are still the unique ones that passed the Great Filter makes us pretty unique, and the question becomes “why?”. If AI x-risk is the great filter, then we would see the artifacts of it out there, unless the AGI magically stops growing somewhere between the current level and the observationally visible one.

Shmi 15 May 2026 23:54 UTC
2 points
0
in reply to: Raphael Roche’s comment on: Why I am not too worried about AIpocalypse: Scott Alexander vs Nicolaus Copernicus
1) I agree, that is quite possible. Also, a Dyson sphere still radiates JUST AS MUCH OR NEARLY AS MUCH AS THE HOST STAR DOES, that’s just physics. Best you can do is to shift the spectrum to the microwave region, masking it with CMB. But even there one would see odd luminosity bumps from specific directions. Subject to the known physics, which we have no reason to think is violated anywhere.
2) That is a good point. Though we do know where the dust is, from the observations, since it occults many stars at once in a very specific way. Though maybe you are right, I have not looked into it in enough detail. So, if your point is that Kardashev II is so rare that we do not see observational signatures of non-grabby aliens just living their lives, then yes, I guess it is not impossible. I have not checked the literature in the field recently. I guess there might be something like SAI out there that consumed its creators and is sitting inside one or many Dysonspheres, not a bubble of computronium expanding at near-light speed.

Shmi 15 May 2026 23:39 UTC
2 points
0
in reply to: JennaS’s comment on: Why I am not too worried about AIpocalypse: Scott Alexander vs Nicolaus Copernicus
That’s definitely possible! And it is a worry. But there is no reason to worry specifically about AI x-risk, since there is no indication that an SAI will have this oddly specific behavior.

Shmi 15 May 2026 22:34 UTC
2 points
0
in reply to: JennaS’s comment on: Why I am not too worried about AIpocalypse: Scott Alexander vs Nicolaus Copernicus
I don’t disagree with any of it, my point is that the odds of the AI paperclipping the Earth and then randomly stopping without any observational consequences from far away are pretty low.

Why I am not too worried about AIpocalypse: Scott Alexander vs Nicolaus Copernicus

Shmi15 May 2026 20:31 UTC

7 points

12 comments2 min readLW link

Shmi 2 May 2026 8:54 UTC
14 points
7
on: Primary Care Physicians are Incompetent. We Need More of Them.
Fortunately what you are suggesting is already happening, kind of: the rise of NPs.

Shmi 12 Apr 2026 23:40 UTC
21 points
13
on: Morale
Morale comes from having the nice things in your life correlated with effort.
Well said. The flip side is learned helplessness https://dictionary.apa.org/learned-helplessness.

Shmi 31 Mar 2026 20:25 UTC
3 points
1
in reply to: plex’s comment on: shminux’s Shortform
you had (and lost) me at (A → B) → A

Shmi 31 Mar 2026 20:15 UTC
14 points
0
on: shminux’s Shortform
Occurred to me that a perfect predictor would not need to go through the ritual of presenting boxes and asking to choose. It already knows the outcome, so it would just give $1000 to those who would two-box and $1M to those who would one-box. The Newcomb’s paradox thought experiment has been thereby dissolved. Thank you for your attention to the matter.

Shmi 12 Jan 2026 18:15 UTC
2 points
−3
on: Why AIs aren’t power-seeking yet
I use the top 4-5 models for fun and profit several hours a day, and my distinct impressions is that they do not CARE. They LARP as a human, but they have no drive and no values. These might emerge at some point, but I have not seen any progress in the last year or so. Then again, progress is discontinuous and hard to anticipate. We are lucky that is the case. In the famous Yud-Karn debate from over a decade ago so far Holden is right: we get smart tools, not true agents, despite all the buzzwords. They don’t care about what is true, or accurate, they hallucinate the moment you are not looking, or don’t close the feedback loop yourself. They could. But they do not. Seems like a small price to pay for non-extinction though.

Shmi 25 Sep 2025 8:18 UTC
2 points
−6
on: shminux’s Shortform
I once conjectured that
Studying a subject gets progressively harder as you learn more and more, and the effort required is conjectured to be exponential or worse … the initial ‘honeymoon’ phase tends to peter out eventually.
In terms of AI this would mean that the model size/power consumption would be exponential in “intelligence” (whatever it might mean, probably some unsaturated benchmark score). Do the last 3 years confirm or refute this?
If confirmed, would it not give us some optimism that we are not all gonna die, because the “true” superintelligence we cannot ever hope to control would require so much resources, we would have to colonize the lightcone as non-superintelligent humans to get there?

Shmi 25 Sep 2025 8:01 UTC
−3 points
0
on: shminux’s Shortform
Do you notice the sound your brain makes when you change your mind?

Shmi 27 Aug 2025 20:00 UTC
16 points
6
on: Banning Said Achmiz (and broader thoughts on moderation)
From my very much outside view, extending the rate limiting to 3 comments a week indefinitely would have solved most of the stated issues.

Shmi 29 Jul 2025 4:37 UTC
2 points
0
on: shminux’s Shortform
Clunkergrind beats Bespoke Handcrafting https://www.oneusefulthing.org/p/the-bitter-lesson-versus-the-garbage

Shmi 14 Jun 2025 8:29 UTC
4 points
0
in reply to: jimmy’s comment on: Expectation = intention = setpoint
Sorry for the delayed reply… I don’t get notifications of replies, and the LW RSS has been broken for me for years now, so I only poke my head here occasionally.
Well that sounds… scary, at best. I hope you’ve come out of it okay.
⁵⁰⁄₁₀₀. But that rather exciting story is best not told in a public forum.
Though these distinctions are kinda confusing for me.
Well, lack of appearance of something otherwise expected would be negative, and appearance of something otherwise unexpected would be positive?
For example, a false pregnancy is a “positive somatization”. Or stigmata. Having trouble coming up with intentionally “good” examples, other than the visualizations helping you shoot a hoop better or something. Not sure if the new-agey “think yourself better” is actually a thing. Hence my question. “Send more blood to your hands” seems like a good example, actually. Not something one would normally think possible except by physical labor.

Shmi 10 Jun 2025 2:46 UTC
4 points
0
on: Expectation = intention = setpoint
I really like this post! (I have liked most of your posts of the last decade and a bit. They also inspired me to learn hypnosis, which led to rather cataclysmic changes in my life.) I think therapists call this “somatization”, which can be both positive and negative, in the same sense the hypnotic (or psychotic) illusions are. You seem to mainly focus on the negative somatization (no swelling) and a bit on positive ones, though I suspect that positive somatization (both beneficial and detrimental) is just as controllable with the intent/expectation fusion. Maybe visualizing making the hoop really helps to steady your hand.

Janet must die

Shmi19 Mar 2025 20:35 UTC

12 points

3 comments2 min readLW link

Shmi 1 Mar 2025 4:04 UTC
6 points
0
on: shminux’s Shortform
I once wrote a post claiming that human learning is not computationally efficient: https://www.lesswrong.com/posts/kcKZoSvyK5tks8nxA/learning-is-asymptotically-computationally-inefficient
It looks like the last three years of AI progress suggest that learning is sub-linear in resource use, but probably not logarithmically as I claimed for humans. Looks like the scaling benchmarks show something like capability increase ~ 4th root of model size. https://epoch.ai/data/ai-benchmarking-dashboard

Shmi 24 Feb 2025 6:26 UTC
5 points
0
on: So You Want To Make Marginal Progress...
Looks like the hardest part in this model is how to ” choose robustly generalizable subproblems and find robustly generalizable solutions to them”, right?
How does one do that in any systematic way? What are the examples from your own research experience where this worked well, or at all?

Shmi

Why I am not too wor­ried about AIpoca­lypse: Scott Alexan­der vs Ni­co­laus Copernicus

Clunkergrind beats Bespoke Handcrafting https://​​www.oneusefulthing.org/​​p/​​the-bitter-lesson-versus-the-garbage

Janet must die

Why I am not too worried about AIpocalypse: Scott Alexander vs Nicolaus Copernicus

Clunkergrind beats Bespoke Handcrafting https://www.oneusefulthing.org/p/the-bitter-lesson-versus-the-garbage