Interesting reflection on Mathematics: https://www.lesswrong.com/posts/iEkksmar6mtuQtdsR/jacob_hilton-s-shortform
Ondřej_Kubů
Karma: 58
Proof of useful work for verifying AI treaties
AI in Mathematics, 2026: From Assistant to Top Researcher
Reading your footnote 2, I got reminded of Rutger Bregman’s The School for Moral Ambition. Rutger Bregman’s “Moral Ambition” project (the book and the School for Moral Ambition) is one real attempt at this, though notably not from inside EA. He explicitly reframes doing good as the ambitious/high-status choice rather than a guilt-driven sacrifice. He is targeting elite-university students headed for McKinsey/consulting/finance, running selective competitive fellowships modeled on prestige-recruiting machinery, and defining moral ambition as “the will to be among the best, but with different measures of success.”
What’s suggestive is that this had to happen outside EA, partly as an explicit critique of it (Bregman frames EA as too guilt-coded, too analysis-paralyzed, not aspirational enough).
That said, Bregman doesn’t engage with the actual meta-question you’re raising — whether openly naming status as a lever backfires. He just uses the lever; there’s no visible theorizing about whether the explicitness itself is corrosive.
ARC’s research agenda: solid mathematics, an unclear safety case
Starting a series that goes into math and safety case of theory-first AI safety orgs. From a mathematician pivoting into AI safety. First up, ARC. https://kubuondr.substack.com/p/arcs-research-agenda-solid-mathematics
ARC’s research agenda: solid mathematics, an unclear safety case
As a mathematician currently in the middle of this transition (leaving an academic math-phys postdoc for AI safety work), two data points that might be useful here:
Resolution, one of the orgs named above, is a good illustration of the capacity problem on the other side of this: their Learning Theory team isn’t taking new applications right now because they’re overwhelmed by applicant volume and are putting hiring-manager time toward their less developed tracks instead. So the constraint isn’t always “not enough mathematicians are interested”, sometimes it’s “not enough hiring bandwidth to process them for a given subfield right now.”
I’d be curious if anyone here has a view on how ARIA’s mathematical-foundations programme and Vanessa Kosoy’s agenda compare in terms of what kind of mathematical background they actually draw on (learning theory vs. more classical formal-methods territory, say). I’m putting together a short survey of orgs/agendas a theory-leaning mathematician might fit into as I explore my options, and first-hand comparisons would help.
I am seeing similar problems in my mathematical physics research. The model steers toward solving a special case even when I explicitly indicate that I want to work on the general case.
Thank you for the post and for being transparent about the funding situation. One practical question: How do you communicate that you only have money until October to existing volunteers and chapter organizers? Local chapters operate mostly self-sufficiently, but people putting significant time into organizing presumably want to know whether the central coordination infrastructure they rely on will exist in six months. Is there a plan for communicating this internally, and what happens to ongoing projects if the funding gap isn’t closed?
Why is PauseAI US a difderent org?
Untitled Draft
Please publish the interview also in a podcast form. I can only find the trailer in my (free) Doom debates feed.
There is also one in Madrid this week https://luma.com/29vob4mv?tk=pob7ra