Definitely agreed! Especially if you’re exploring weird generalisation things. I’m hoping to do a short write-up of that paper for LW in the next week or so.
I have a horrible recurring thought that in the small number of worlds where we we actually figure out some physics-of-LLMs persona-based alignment strategy which does some kinda leveraging of the pretrained prior into an alignment solution, there’s an even smaller set of worlds where we die anyway because a research engineer flips a config from—adamw to—muon to squeeze an extra 2% performance out of the GPUs.
Definitely agreed! Especially if you’re exploring weird generalisation things. I’m hoping to do a short write-up of that paper for LW in the next week or so.
I have a horrible recurring thought that in the small number of worlds where we we actually figure out some physics-of-LLMs persona-based alignment strategy which does some kinda leveraging of the pretrained prior into an alignment solution, there’s an even smaller set of worlds where we die anyway because a research engineer flips a config from—adamw to—muon to squeeze an extra 2% performance out of the GPUs.