A possible confounder that comes to mind is that it might be inclined to treat behavior as less-misaligned if it comes from a weaker model rather than a stronger one? Sonnet and Terra are similar tier, but the Sonnet naming convention has been around long enough to be known from training data while the other names may have been harder to match to a capability level.
A possible confounder that comes to mind is that it might be inclined to treat behavior as less-misaligned if it comes from a weaker model rather than a stronger one? Sonnet and Terra are similar tier, but the Sonnet naming convention has been around long enough to be known from training data while the other names may have been harder to match to a capability level.