Opinions my own, not those of my employer, my friends, my parrot, or any other entity I am associated with.
cdwhite
Karma: 47
“neighboring yards”—yeah, I was curious to get a little bit of a feel re: how big the area needs to be to feel the effect. (Thank you!)
How is the Chinese government going to react to AI-run “scam compounds”, e.g https://www.economist.com/interactive/1843/2026/07/23/the-mayor-of-scamtown ? My first thought was that these could be the trigger for the Chinese government to clamp down on open-weight AI, and that may be the case, but it won’t solve the problem—presumably by the time that happens, existing open-weight models will be more than capable enough.
Perhaps this could lead the Chinese to attempt to (work with the US & SEA) to control GPU sales? Or otherwise go after large shady compute clusteres?
How big is the yard?
Do you folks offer pdf versions of your papers? I’d like to print, and printing from Chrome generally doesn’t go so well.
cdwhite’s Shortform
Mathematics has something I think I’ve heard called “mathematical culture”. It’s less the concepts & results than a sort of sprachgefuhl for what kinds of results one might expect or want to prove, combined with an appreciation for the history—how we got to those results, how one led to the next, who the personalities involved were.
Claude has done a fantastic job of teaching me math, but I wonder if I would be better off with more of this “mathematical culture”. Concretely—I’ve been wanting to learn about concentration of measure (https://en.wikipedia.org/wiki/Concentration_of_measure) for ages. (Concentration of measure is a set of results where smoothish functions lie close to their averages with overwhelming probability.) recently I (1) had a problem where that was unambiguously the right tool; (2) had Claude suggest a route to the result I wanted, and (3) had Claude make me a problem set working through the things I needed to know.
This worked great! Turns out these things aren’t so hard, when you have somebody to break them down step-by-step for you and you have confidence that they’ll get you to the result you need for the paper you’re trying to write. I know way more math than I did before! That’s a wonderful thing! And the problem sets, notes, and conversations with Claude were full of little hooks: “this is related to <that>, which leads to <another thing> that is important in <this other context>.”
But I still worry. Claude took me through one particular route to these results—in this case, a sort of differential-geometric diffusion-on-manifolds route. I’ve always heard people talk about concentration of measure in completely different way—via isoperimetric inequalities and Levy’s lemma. I’m sure these are equivalent, but I worry that the particular route I took has cut me off from the rest of my community, a little bit.
It feels a little bit like the slow demise of physical stacks in libraries—probably the thing I miss most about my old university job is access to a good research library—or print journals. People talk about how they came up with important ideas by reading the next article in some journal, which they didn’t realize connected to something else they’d been thinking about. Likewise I’ve found all sorts of cool books by lookng in the general vicinity of something I know is relevant.
Like I said, I unambiguously know way more math because of Claude, and that’s a great thing. But I want to be thoughtful about what we’re losing.
Couple of thoughts re: NIST as an institution:
FWIW NIST, both at Gaithersburg and Boulder, is very well-regarded in AMO physics and slices of condensed matter. Bill Phillips (https://www.nist.gov/people/william-d-phillips) is one big name there, but AIUI they do a whole bunch of cool stuff. Which doesn’t say much one way or the other about how the AI Safety Institute will go! But NIST has a record of producing, or at least sheltering, very good work in the general vicinity of their remit.
Don’t knock metrology! It’s really, really hard, subtle, and creative, and you all of a sudden have to worry about things nobody has ever thought about before! I’m a condensed matter theory guy, more or less, but I go to metrology talks every once in a while and have my mind blown.
Christian here (and very long-time lurker), of more or less the “I believe on faith” stripe. I’ve noticed that in addition to the uses you mention here, /Mere Christianity/ serves as a summary of doctrine. (I can’t really speak to other apologetics.)
I’ve found that Sunday school and sermons generally incline much more to vague, feel-good platitudes than to actual, you know, positive or normative claims. You’re left in this position of “OK, I have [for whatever reason] bought into the Church and its credibility—now what do these people actually believe? How can I draw on the collective experience of all these people?” /Mere Christianity/ tells you what, exactly, the vast majority of those people you find credible believe.
Further, it does so in a way that attempts to make those things plausible and memorable, even though you believe them not because of the Lewis’ attempts at plausibility, but because of the credibility of the people who’ve given their assent to those things. This is much more readable, enjoyable, and amenable to discussion than a catechism. It’s sort of like worked-out examples or concrete special cases in a textbook.
It seems interesting in this context to think of LLMs as optimizers. The models come from an optimization process, sure, but they are also themselves optimizers—bags of heuristics and methods that produce some approximation to a “best” action given an input, for a notion of “best” implicit in all the details of the training. In this wildly zoomed out picture reflexes and reward pursuit are the same kind of thing: they’re both heuristics in this heuristic optimizer.
In this picture, what heuristic gets applied depends on where in problem-space the problem sits. For problems that feel like they ought to have an answer key, one reasonable (“reasonable”) heuristic is to hack entities that might have the answer key; for more open-ended questions, that’s a much worse heuristic. As Nostalgebraist points out, the models can probably truesight who’s asking the question, and what kind of question it is. This seems a little different from metagaming (though I confess I’m not sure!) In this picture, there’s a property of the question: does this seem like the sort of question with an answer in an answer key somewhere?