I have a horrible recurring thought that in the small number of worlds where we we actually figure out some physics-of-LLMs persona-based alignment strategy which does some kinda leveraging of the pretrained prior into an alignment solution, there’s an even smaller set of worlds where we die anyway because a research engineer flips a config from—adamw to—muon to squeeze an extra 2% performance out of the GPUs.
I have a horrible recurring thought that in the small number of worlds where we we actually figure out some physics-of-LLMs persona-based alignment strategy which does some kinda leveraging of the pretrained prior into an alignment solution, there’s an even smaller set of worlds where we die anyway because a research engineer flips a config from—adamw to—muon to squeeze an extra 2% performance out of the GPUs.