Only individuals have interests. A group’s interest is an abstraction, built by aggregating the interests of its members.
An individual and their interests are both abstractions too. I know my own interests don’t always align and need to be aggregated somehow when deciding what to spend time and resources on.
It cuts both ways though. The pain isn’t permanently etched into its neurons. Enduring suffering is different when the effect is limited to a context window. I’m not saying that makes it OK, just different in nature.
LLMs still seem to perform more independent computations (incl. fact lookups) than they have neurons; I don’t think there’s a way to implement this with only monosemantic neurons.
This feels like a non sequitur to me. Computers do just fine with just 0 and 1. All you need is for “neurons” to be sufficiently compositional… or does that mean they’re no longer monosemantic? Seems like we need to nail down precisely what we mean by such terms or we may disagree on language even if we predict the same things.
C < 7
Lean-verified and rewritten end-to-end as a single proof incorporating work from David, Satya, and Jeremy
Repo: https://github.com/AlexisOlson/natural-latents
Blueprint: https://alexisolson.github.io/natural-latents/
That sounds like an interesting project, and might even be something you could turn into an LLM benchmark. Can you give an example of what you’d want to see as a human-interpretable explanation for one “neuron” using one you do understand the semantics of?
If I look at the top of one of the letters in Miyoko’s, I can read UNSALTED, CREAMERY, and BUTTER, but PLANT MILK is just barely small enough that I can’t make it out easily in my peripheral vision on the screen I’m looking at. It’s in an understated position, not drawing attention to itself, and easy for your eye to slide right over if you aren’t looking for it.
I’ve seen several relevant quotes posted today, so I used GPT-Sol 6.1 to gather expert reactions to the results released on October 6 and checked each quote against its original source.
On “invent a new concept that does real work,” the nearest candidate here is the new code Moshkovitz describes. Ahle describes a different route to a bound, built on Strassen’s existing asymptotic-spectrum framework. These are early reactions to OpenAI’s October 6 mathematics release; the last two discuss the same plane-colouring proof.
Dana Moshkovitz: UT Austin professor specializing in probabilistically checkable proofs and pseudorandomness.
“Upon further inspection, the proof constructs a completely new tree-based code that somehow has a noise test with properties similar to that of the short code.”
Her addendum, posted by her husband Scott Aaronson in comment #2, October 7. In her own comment #5, she emphasizes the existing frameworks and results the AI built on, while calling the results “paradigm shifts.” She also criticizes the exposition and citations.
Thomas Dybdahl Ahle: theoretical computer scientist whose research includes algorithms and tensor sketching; Head of Machine Learning at Normal Computing.
“the new FMM paper is surprisingly elegant. It uses a very different approach than previous results. Instead of powering the CW tensor and trimming it, [they] define a potential function on tensors, and show the whole result by contradiction by studying the simple convolution tensor.”
Original X post, October 7. “CW” means Coppersmith–Winograd; “[they]” corrects a typo. The paper describes a profile, separation construction and growth argument; “potential function” is Ahle’s description.
Simon Pepin Lehalleur: University of Duisburg-Essen; algebraic and arithmetic geometry, particularly motives and motivic homotopy theory.
Of the OpenAI proofs he had skimmed, he singled out:
“The most original … is the algebraicity of the Kuga-Satake correspondence for K3 surfaces, using techniques from mirror symmetry.”
Later that day, on a very superficial reading, he judged the Bloch-conjecture proof “not based on deep new insights into codimension 2 cycles in general,” though “very substantial progress on cycles on (powers of) surfaces.”
Joshua Zelinsky: published number theorist with a mathematics PhD from Boston University.
“I’ve only had time to start looking at the chromatic plane proof, and it doesn’t look like the method is a direction that the prior lit used to my knowledge. This is far beyond merely building on existing methods or seeing connections between different problems.”
Bluesky reply, October 6. The paper credits earlier invariant randomization and calls its averaging standard; the proof combines a transfer to weak measurable colourings with a geometric obstruction to five colours.
Dmitry Rybin: cofounder and chief scientist of a Shenzhen startup working on AI for mathematics; trained in pure mathematics at HSE; CUHK machine-learning PhD, 2026.
“Somehow the model found that any K-coloring ⇔ “weakly measurable” K-coloring, which seems out of nowhere”
Original X post, October 7. His follow-up says it used “ridiculous human-developed machinery”: amenable groups, Pontryagin duality and Furstenberg–Zimmer towers.
Are these measuring the design of one experiment at a time? LLMs are pretty good at this these days (my recent experience is mostly with Opus 5.5 and Sol 6.1), but they still need handholding on strategy and metacognition to stay productive and on task for research projects that don’t have obvious next steps at any given point.
I found this hard to follow without some nice charts to help visualize the OOMs.
Congrats! I’ve validated your Lean locally, and expect I can get it down to 8.6 or better.
Edit: I can get below 7. Working on making it human-friendly and lean-verified now.
“burned millions of dollars when other people are not that rich”
This is a weird take, IMO. It was a tiny fraction of their compute and cost roughly the amount they spend on TV ads in a month. It’s an amazing proof of capabilities.
I would expect similar behavior for most topics without a strong consensus position among experts. Have you experimented with other topics as controls to help interpret these findings?
The fully general
https://github.com/AlexisOlson/stochastic-to-deterministic-latents/pull/16
It seems to me that most people can achieve their desired level of rationality without supererogatory levels of putting up with ugliness for two main reasons.
They don’t value “rationality” as highly as you do, so they needn’t pursue alternate perspectives as hard. It’s not a high priority consideration when making daily choices, anymore than nutrition is for you.
Ugly discourse mostly isn’t very enlightening anyway. It’s full of arguments as soldiers and attacks on people’s character in ways that are not highly truth-seeking.
It varies wildly depending on the task. We for sure aren’t competitive at simple math calculations. I wonder which tasks we still do the relative best on. Maybe there aren’t many fixed tasks left we win on and the remainder is mostly being a continuous embodied agent rather than an ephemeral session.
This is, of course, speculative, because we do not really understand why current systems fall short of AGI. It is conceivable that what they lack is very compute-intensive, and that we are greatly underestimating the cost.
Real continual learning that is internalized rather than reliant upon a harness or scaffold is, IMO, the biggest remaining unhobbling. It’s hard to run a business as an amnesiac, even if you keep good notes.
Related to 3 are flavors of transhumanism like “AI is equivalent to the next generation. Every generation and species eventually dies out or is replaced by a successor. This is normal and/or desirable, since otherwise we’d be single-celled organisms.”
right now we are building our RL environments with minimum wage workers at contractors and it’s messing up all the models. The scary question is what happens when the AIs start making the RL environments, if that is the standard.
We may already have passed the point where AIs are better at creating diverse and robust RL environments than the contractors, which possibly might be good for alignment if they aren’t already sabotaging/scheming/sandbagging.
It’s hard to imagine how they could keep finding more and more alpha indefinitely, but I suppose they can rent any compute that isn’t directly useful to them, so acquiring so much isn’t necessarily risky.
One laptop per account. You’d never reuse a password, so why reuse hardware? When a device gets rooted, the attacker gets exactly one account. Buy from different manufacturers so a single firmware bug can’t hit every laptop. Each account also gets its own email address (on its own laptop), backed up with recovery codes stamped into steel plates, one set stashed in your shielded office and one in a safe deposit box.
Burn a fresh disc whenever anything crosses the air gap, in either direction. No disc makes a round trip, and optical media has no controller chip to re-flash. Check each inbound file signature on the air-gapped side, not on the connected machine.
Split your password manager master key with Shamir’s secret sharing, so it requires two of three trusted people to unlock it. Never reuse a keyholder across vaults. Unlocking must happen in person to prevent deepfake frauds.
Treat every device in your house as a potential bug. No speakers or TVs in the office; put phones in a shielded locker outside the door, and run white noise whenever you type a passphrase (or simply continuously). Shield the room (conductive paint, copper mesh in the drywall, metallized window film) so nothing on your network can pick up radio, sound, or light leaking from the air-gapped machine.
Remember that none of this helps once a bank’s own servers get hacked. You can try spreading across many banks or just withdraw everything in cash and stash it in your shielded office, where it’s safe from everything except fire, flood, burglars, bailiffs, moths, mice, mold, people, pets, in-laws, and inflation (this is not an exhaustive list). Gold mitigates some of these vectors.