Stefan_Schubert

Ste­fan_Schu­bert’s Shortform

• Stefan_Schubert 11 Mar 2026 16:27 UTC

  31 points

  9

  on: Ste­fan_Schu­bert’s Shortform

  I’m writing a newsletter on current events, long-term trends, and topical debates roughly every other day. Recent posts include:

  • A summary of a debate on AI progress, featuring Ajeya Cotra, Peter Wildeford, Eli Lifland, Matthew Barnett, and others.

  • The three types of problems with population decline.

  • We underestimate the pace of progress because much of it isn’t salient.

  • The reason we don’t see more automation is simply that AI isn’t good enough.

  • Economists are unusually good at taking human agency into account.

• Stefan_Schubert 24 Sep 2024 7:25 UTC

  15 points

  0

  in reply to: whestler’s comment on: wh­estler’s Shortform

  Cf this Bostrom quote.

  Far from being the smartest possible biological species, we are probably better thought of as the stupidest possible biological species capable of starting a technological civilization—a niche we filled because we got there first, not because we are in any sense optimally adapted to it.

  Re this:

  In evolutionary timescales, virtually no time has elapsed since hominids began trading, utilizing complex symbolic thinking, making art, hunting large animals etc, and here we are, a blip later in high technology.

  A bit nit-picky, but a recent paper studying West Eurasia found significant evolution over the last 14,000 years.

  

• Stefan_Schubert 6 Jul 2024 11:30 UTC

  2 points

  0

  in reply to: Alexander Gietelink Oldenziel’s comment on: Alexan­der Gietelink Ol­den­z­iel’s Shortform

  There’s a related confusion between uses of “theory” that are neutral about the likelihood of the theory being true, and uses that suggest that the theory isn’t proved to be true.

  Cf the expression “the theory of evolution”. Scientists who talk about the “theory” of evolution don’t thereby imply anything about its probability of being true—indeed, many believe it’s overwhelmingly likely to be true. But some critics interpret this expression differently, saying it’s “just a theory” (meaning it’s not the established consensus).

• Stefan_Schubert 9 May 2024 21:32 UTC

  21 points

  14

  on: We might be miss­ing some key fea­ture of AI take­off; it’ll prob­a­bly seem like “we could’ve seen this com­ing”

  Thanks for this thoughtful article.
  
  It seems to me that the first and the second examples have something in common, namely an underestimate of the degree to which people will react to perceived dangers. I think this is fairly common in speculations about potential future disasters, and have called it sleepwalk bias. It seems like something that one should be able to correct for.
  
  I think there is an element of sleepwalk bias in the AI risk debate. See this post where I criticise a particular vignette.

• Stefan_Schubert 3 Aug 2023 9:49 UTC

  2 points

  0

  in reply to: Adam David Long’s comment on: The “pub­lic de­bate” about AI is con­fus­ing for the gen­eral pub­lic and for poli­cy­mak­ers be­cause it is a three-sided de­bate

  Yeah, I think so. But since those people generally find AI less important (there’s both less of an upside and less of a downside) they generally participate less in the debate. Hence there’s a bit of a selection effect hiding those people.

  There are some people who arguably are in that corner who do participate in the debate, though—e.g. Robin Hanson. (He thinks some sort of AI will eventually be enormously important, but that the near-term effects, while significant, will not be at the level people on the right side think).

  Looking at the 2x2 I posted I wonder if you could call the lower left corner something relating to “non-existential risks”. That seems to capture their views. It might be hard to come up with a catch term, though.

  The upper left corner could maybe be called “sceptics”.

• Stefan_Schubert 2 Aug 2023 18:27 UTC

  4 points

  0

  in reply to: Adam David Long’s comment on: The “pub­lic de­bate” about AI is con­fus­ing for the gen­eral pub­lic and for poli­cy­mak­ers be­cause it is a three-sided de­bate

  Not exactly what you’re asking for, but maybe a 2x2 could be food for thought.

  

• Stefan_Schubert 2 Aug 2023 6:17 UTC

  16 points

  8

  on: The “pub­lic de­bate” about AI is con­fus­ing for the gen­eral pub­lic and for poli­cy­mak­ers be­cause it is a three-sided de­bate

  Realist and pragmatist don’t seem like the best choices of terms, since they pre-judge the issue a bit in the direction of that view.

• Stefan_Schubert 8 Jan 2023 19:03 UTC

  2 points

  0

  in reply to: Roman Leventov’s comment on: AI psy­chol­ogy should ground the the­o­ries of AI con­scious­ness and in­form hu­man-AI eth­i­cal in­ter­ac­tion design

  Thanks.

  I think psychologists-scientists should have unusually good imaginations about the potential inner workings of other minds, which many ML engineers probably lack.

  That’s not clear to me, given that AI systems are so unlike human minds.

• Stefan_Schubert 8 Jan 2023 12:14 UTC

  2 points

  0

  on: AI psy­chol­ogy should ground the the­o­ries of AI con­scious­ness and in­form hu­man-AI eth­i­cal in­ter­ac­tion design

  tell your fellow psychologist (or zoopsychologist) about this, maybe they will be incentivised to make a switch and do some ground-laying work in the field of AI psychology

  Do you believe that (conventional) psychologists would be especially good at what you call AI psychology, and if so, why? I guess other skills (e.g. knowledge of AI systems) could be important.

• Stefan_Schubert 26 Dec 2022 20:36 UTC

  6 points

  4

  in reply to: Peter Wildeford’s comment on: Let’s think about slow­ing down AI

  I think that’s exactly right.

• Stefan_Schubert 26 Dec 2022 10:16 UTC

  7 points

  4

  in reply to: Peter Wildeford’s comment on: Let’s think about slow­ing down AI

  I think that could be valuable.

  It might be worth testing quite carefully for robustness—to ask multiple different questions probing the same issue, and see whether responses converge. My sense is that people’s stated opinions about risks from artificial intelligence, and existential risks more generally, could vary substantially depending on framing. Most haven’t thought a lot about these issues, which likely contributes. I think a problem problem with some studies on these issues is that researchers over-generalise from highly framing-dependent survey responses.

• Stefan_Schubert 22 Dec 2022 12:45 UTC

  3 points

  0

  on: The next decades might be wild

  I wrote an extended comment in a blog post.

  Summary:

  Summing up, I disagree with Hobbhahn on three points.

  1. I think the public would be more worried about harm that AI systems cause than he assumes.

  2. I think that economic incentives aren’t quite as powerful as he thinks they are, and I think that governments are relatively stronger than he thinks.

  3. He argues that governments’ response will be very misdirected, and I don’t quite buy his arguments.

  Note that 1 and ²⁄₃ seem quite different: 1 is about how much people will worry about AI harms, whereas 2 and 3 are about the relative power of companies/​economic incentives and governments, and government competency. It’s notable that Hobbhahn is more pessimistic on both of those relatively independent axes.

• Stefan_Schubert 22 Apr 2022 11:53 UTC

  2 points

  0

  in reply to: DirectedEvolution’s comment on: For ev­ery choice of AGI difficulty, con­di­tion­ing on grad­ual take-off im­plies shorter timelines.

  Another way to frame this, then, is that “For any choice of AI difficulty, faster pre-takeoff growth rates imply shorter timelines.”

  I agree. Notably, that sounds more like a conceptual and almost trivial claim.

  I think that the original claims sound deeper than they are because they slide between a true but trivial interpretation and a non-trivial interpretation that may not be generally true.

• Stefan_Schubert 21 Apr 2022 11:04 UTC

  2 points

  0

  in reply to: Rohin Shah’s comment on: For ev­ery choice of AGI difficulty, con­di­tion­ing on grad­ual take-off im­plies shorter timelines.

  Thanks.

  My argument involved scenarios with fast take-off and short time-lines. There is a clarificatory part of the post that discusses the converse case, of a gradual take-off and long time-lines:

  Is it inconsistent, then, to think both that take-off will be gradual and timelines will be long? No – people who hold this view probably do so because they think that marginal improvements in AI capabilities are hard. This belief implies both a gradual take-off and long timelines.

  Maybe a related clarification could be made about the fast take-off/​short time-line combination.

  However, this claim also confuses me a bit:

  No – people who hold this view probably do so because they think that marginal improvements in AI capabilities are hard. This belief implies both a gradual take-off and long timelines.

  The main claim in the post is that gradual take-off implies shorter time-lines. But here the author seems to say that according to the view “that marginal improvements in AI capabilities are hard”, gradual take-off and longer timelines correlate. And the author seems to suggest that that’s a plausible view (though empirically it may be false). I’m not quite sure how to interpret this combination of claims.

• Stefan_Schubert 21 Apr 2022 9:28 UTC

  4 points

  0

  on: For ev­ery choice of AGI difficulty, con­di­tion­ing on grad­ual take-off im­plies shorter timelines.

  For every choice of AGI difficulty, conditioning on gradual take-off implies shorter timelines.

  What would you say about the following argument?

  • Suppose that we get AGI tomorrow because of a fast take-off. If so timelines will be extremely short.

  • If we instead suppose that take-off will be gradual, then it seems impossible for timelines to be that short.

  • So in this scenario—this choice of AGI difficulty—conditioning on gradual take-off doesn’t seem to imply shorter timelines.

  • So that’s a counterexample to the claim that for every choice of AGI difficulty, conditioning on gradual take-off implies shorter timelines.

  I’m not sure whether it does justice to your reasoning, but if so, I’d be interested to learn where it goes wrong.

  What links here?

  • Jacy Reese Anthis's comment on For ev­ery choice of AGI difficulty, con­di­tion­ing on grad­ual take-off im­plies shorter timelines. by Francis Rhys Ward (21 Apr 2022 12:52 UTC; 4 points)

• Stefan_Schubert 8 Oct 2021 3:09 UTC

  3 points

  0

  in reply to: Stefan_Schubert’s comment on: Is there a name for the the­ory that “There will be fast take­off in real-world ca­pa­bil­ities be­cause al­most ev­ery­thing is AGI-com­plete”?

  Holden Karnofsky defends this view in his latest blog post.

  I think it’s too quick to think of technological unemployment as the next problem we’ll be dealing with, and wilder issues as being much further down the line. By the time (or even before) we have AI that can truly replace every facet of what low-skill humans do, the “wild sci-fi” AI impacts could be the bigger concern.

• Stefan_Schubert 3 Sep 2021 1:35 UTC

  4 points

  0

  on: Is there a name for the the­ory that “There will be fast take­off in real-world ca­pa­bil­ities be­cause al­most ev­ery­thing is AGI-com­plete”?

  A related view is that less advanced/​more narrow AI will do be able to do a fair number of tasks, but not enough to create widespread technological unemployment until very late, when very advanced AI quite quickly causes lots of people to be unemployed.

  One consideration is how long time it will take for people to actually start using new AI systems (it tends to take some time for new technologies to be widely used). I think that some have speculated that that time lag may be shortened as AI become more advanced (as AI becomes involved in the deployment of other AI systems).

• Stefan_Schubert 31 Aug 2021 12:23 UTC

  5 points

  0

  on: The Death of Be­hav­ioral Economics

  Scott Alexander has written an in-depth article about Hreha’s article:

  The article itself mostly just urges behavioral economists to do better, which is always good advice for everyone. But as usual, it’s the inflammatory title that’s gone viral. I think a strong interpretation of behavioral economics as dead or debunked is unjustified.

  See also Alex Imas’s and Chris Blattman’s criticisms of Hreha (on Twitter).

• Stefan_Schubert 19 Aug 2021 12:29 UTC

  2 points

  0

  on: Con­cep­tual en­g­ineer­ing: the rev­olu­tion in philos­o­phy you’ve never heard of

  I think that though there’s been a welcome surge of interest in conceptual engineering in recent years, the basic idea has been around for quite some time (though under different names). In particular, Carnap argued that we should “explicate” rather than “analyse” concepts already in the 1940s and 1950s. In other words, we shouldn’t just try to explain the meaning of pre-existing concepts, but should develop new and more useful concepts that partially replace the old concepts.

  Carnap’s understanding of explication was influenced by Karl Menger’s conception of the methodological role of definitions in mathematics, exemplified by Menger’s own explicative definition of dimension in topology.
  ...
  Explication in Carnap’s sense is the replacement of a somewhat unclear and inexact concept C, the explicandum, by a new, clearer, and more exact concept $undefined$, the explicatum.

  See also Logical Foundations of Probability, pp. 3-20.