Pro tip for all fine-tuning experiments. Fuck around with your optimizer. In some cases I’ve seen different results with Adam and Muon. If you want to use LoRA there’s a Muon version called Riamannion or something like that. From a physics-of-LLMs perspective, Muon is a very different creature to Adam; I’d like to get as much tacit knowledge into the community as possible, about exactly what this means.
Definitely agreed! Especially if you’re exploring weird generalisation things. I’m hoping to do a short write-up of that paper for LW in the next week or so.
I have a horrible recurring thought that in the small number of worlds where we we actually figure out some physics-of-LLMs persona-based alignment strategy which does some kinda leveraging of the pretrained prior into an alignment solution, there’s an even smaller set of worlds where we die anyway because a research engineer flips a config from—adamw to—muon to squeeze an extra 2% performance out of the GPUs.
If a result doesn’t generalize between optimizers, that often is a reason I’m much less interested in it, because this is evidence against it generalizing to future situations I care about. This is especially true for research that tries to learn facts, rather than research that tries to produce models with particular properties so that they can be used as subjects of research. E.g. it’s very hard to make sandbagging model organisms, and it would be sort of handy to have those model organisms around, so it is useful to know whether different optimizers help (my coworkers looked into this and didn’t make it work.)
Pro tip for all fine-tuning experiments. Fuck around with your optimizer. In some cases I’ve seen different results with Adam and Muon. If you want to use LoRA there’s a Muon version called Riamannion or something like that. From a physics-of-LLMs perspective, Muon is a very different creature to Adam; I’d like to get as much tacit knowledge into the community as possible, about exactly what this means.
One example: Muon has less Emergent misalignment, SGD has more. https://arxiv.org/html/2606.31591v1
Definitely agreed! Especially if you’re exploring weird generalisation things. I’m hoping to do a short write-up of that paper for LW in the next week or so.
I have a horrible recurring thought that in the small number of worlds where we we actually figure out some physics-of-LLMs persona-based alignment strategy which does some kinda leveraging of the pretrained prior into an alignment solution, there’s an even smaller set of worlds where we die anyway because a research engineer flips a config from—adamw to—muon to squeeze an extra 2% performance out of the GPUs.
Perhaps you were referring to the extension to Riemannian Manifolds?
Yeah that’s the one. My Claudes have been calling the specific optimizer something like Riemannion but that might be a Claudeism.
If a result doesn’t generalize between optimizers, that often is a reason I’m much less interested in it, because this is evidence against it generalizing to future situations I care about. This is especially true for research that tries to learn facts, rather than research that tries to produce models with particular properties so that they can be used as subjects of research. E.g. it’s very hard to make sandbagging model organisms, and it would be sort of handy to have those model organisms around, so it is useful to know whether different optimizers help (my coworkers looked into this and didn’t make it work.)
Fig 1 of the paper show the opposite, right? SGD causing less EM than Muon.