Do your pretraining steps (e.g. in Figure 1) include the midtraining stage?
There is a fairly large discrepancy between step 0 (base) in Figure 3, which shows ~15% average eval awareness, and your final step of Figure 1, which shows ~1%.
I’m assuming this makes up the difference, but I want to be sure! We’re considering a filtering project based around metagaming / VEA, which would make this midtraining dataset a prime target if it was responsible for this increase.
Thanks for open-sourcing your code! I was able to ask Fable to run an extension with the midtrained and long-context trained checkpoints and also compare this to the base Nemotron 120B super. It looks like the jump is primarily explained by the long-context mix.
Nice! I suspect even post data-filtering, if the model is trained with RLVR it should might boost VEA & UVEA.
(during RLVR) My guess is that UVEA/VEA, grader-awareness arise mostly when the verifier fails to capture the full spectrum of possible solutions but focusses on a smaller subset, forcing the model to guess the right format/structure of the valid solutions which could get accepted, My current project is around this.
Do your pretraining steps (e.g. in Figure 1) include the midtraining stage?
There is a fairly large discrepancy between step 0 (base) in Figure 3, which shows ~15% average eval awareness, and your final step of Figure 1, which shows ~1%.
I’m assuming this makes up the difference, but I want to be sure! We’re considering a filtering project based around metagaming / VEA, which would make this midtraining dataset a prime target if it was responsible for this increase.
Thanks for open-sourcing your code! I was able to ask Fable to run an extension with the midtrained and long-context trained checkpoints and also compare this to the base Nemotron 120B super. It looks like the jump is primarily explained by the long-context mix.
Nice! I suspect even post data-filtering, if the model is trained with RLVR it should might boost VEA & UVEA.
(during RLVR) My guess is that UVEA/VEA, grader-awareness arise mostly when the verifier fails to capture the full spectrum of possible solutions but focusses on a smaller subset, forcing the model to guess the right format/structure of the valid solutions which could get accepted, My current project is around this.