The preprint about the linear mapping complexity should be out in the next couple weeks. The proofs are all done, but we’ve been working hard to make the manuscript as clear as possible, since the mathematics of it is genuinely nontrivial (at least to us!), even though the proofs are based on empirical intuitions Dan & I developed over the years. I’ll update this blogpost with the arXiv link once it’s out :)
The implications of such a result for safety is that you can identify sufficient conditions (specifically, regularities in the pre-nonlinear thresholds at each layer) such that if two networks (or a brain and a network) solve the same tasks within -error, then we expect “strong representational convergence” in the “OOD” sense that the two networks become aligned at the unit level, up to (affine) rescalings and permutations of their units. Thus, in practice, you don’t need brain data (in fact, it’s quite noisy anyway!) to co-train on, so long as you have sufficient behavioral/task alignment at the top layer between the two networks, and the strong representational convergence then “zippers” backward down the layers in the sense of Thobani et al. 2025 (linked in the blogpost above, but I’m including it here for convenience: https://arxiv.org/abs/2510.02523).
I discuss sample complexity on pg. 4 (cf. Remark 1) of the UAI paper. It’s additionally worth noting that the world model recovery bounds in the fully observed case of Theorem 1, and partially observed case of Theorem 3, are explicitly dependent on the number of goals () and the amount of average regret (), rather than only being in the limit.
I do think AI will be super useful for accelerating this type of research. One thing though, at least with current reasoning models like GPT-5.5-Pro, is that while I have yet to see them prove untrue theorems (like I did with o1/o3 once or twice a year ago), they can prove true but relatively trivial theorems, in that strong assumptions are smuggled in. That’s something that for now needs human experts to discern and continually interact with, “unwind”, and elicit from the models.
As promised, I’ve updated the blogpost with our new Contravariance theory paper regarding linear mapping complexity being at most sufficient for representational alignment between two systems under hard tasks. Putting the links here as well for convenience:
Thanks so much, Bogdan! Excellent questions.
The preprint about the linear mapping complexity should be out in the next couple weeks. The proofs are all done, but we’ve been working hard to make the manuscript as clear as possible, since the mathematics of it is genuinely nontrivial (at least to us!), even though the proofs are based on empirical intuitions Dan & I developed over the years. I’ll update this blogpost with the arXiv link once it’s out :)
The implications of such a result for safety is that you can identify sufficient conditions (specifically, regularities in the pre-nonlinear thresholds at each layer) such that if two networks (or a brain and a network) solve the same tasks within
I discuss sample complexity on pg. 4 (cf. Remark 1) of the UAI paper. It’s additionally worth noting that the world model recovery bounds in the fully observed case of Theorem 1, and partially observed case of Theorem 3, are explicitly dependent on the number of goals (
I do think AI will be super useful for accelerating this type of research. One thing though, at least with current reasoning models like GPT-5.5-Pro, is that while I have yet to see them prove untrue theorems (like I did with o1/o3 once or twice a year ago), they can prove true but relatively trivial theorems, in that strong assumptions are smuggled in. That’s something that for now needs human experts to discern and continually interact with, “unwind”, and elicit from the models.
As promised, I’ve updated the blogpost with our new Contravariance theory paper regarding linear mapping complexity being at most sufficient for representational alignment between two systems under hard tasks. Putting the links here as well for convenience:
Paper: https://arxiv.org/abs/2607.08561
Substack series (which goes into more of the conceptual details & intuitions but without all the hairy math!): https://danyamins.substack.com/p/the-theory-of-contravariance-part0