The smooth graphs seem like good evidence that there are much smoother underlying changes in the model, and that the abruptness of the change is about behavior or evaluation rather than what gradient descent is learning.
If we’re trying to predict abrupt changes in the accuracy of output token sequences, the per-token log-likelihood can be a useful signal. What’s the analogous signal when we’re talking about abrupt changes in a model’s ability to deceptively conceal capabilities, hack GPU firmware, etc.? What log-likelihood plots can we use to predict those types of abrupt changes in behavior?
Here, I think we’ll want to look for suspicious changes in the log-likelihood trends. E.g., it’s a red flag if we see steady increases in log-likelihood on some scary behavior, but then the trend reverse at some level of model scale.
If we’re trying to predict abrupt changes in the accuracy of output token sequences, the per-token log-likelihood can be a useful signal. What’s the analogous signal when we’re talking about abrupt changes in a model’s ability to deceptively conceal capabilities, hack GPU firmware, etc.? What log-likelihood plots can we use to predict those types of abrupt changes in behavior?
Here, I think we’ll want to look for suspicious changes in the log-likelihood trends. E.g., it’s a red flag if we see steady increases in log-likelihood on some scary behavior, but then the trend reverse at some level of model scale.