This seems totally possible—the EM-from-SFT results make me think that’s not the full picture, but it may well be the case that “SFT on off-policy transcripts doesn’t care about personal consistency, but RL is on-policy and therefore starting consistency has strong effects”
This seems totally possible—the EM-from-SFT results make me think that’s not the full picture, but it may well be the case that “SFT on off-policy transcripts doesn’t care about personal consistency, but RL is on-policy and therefore starting consistency has strong effects”