If I’ve already done WMLB, what day should I start on? The WMLB curriculum on mechinterp wasn’t very polished, and IOI and superposition were not covered. But doing part of the transformers week would mean getting material I’ve already learned on RL.
The first week of WMLB / MLAB maps quite closely onto the first week of ARENA, with a few exceptions (ARENA includes PyTorch Lightning, plus some more meta stuff like typechecking, VSCode testing and debugging, using GPT in your workflow, etc). I’d say that starting some way through the second week would probably be most appropriate. If you didn’t want to repeat stuff on training / sampling from transformers, the mech interp material would start on Wednesday of the second week.
If I’ve already done WMLB, what day should I start on? The WMLB curriculum on mechinterp wasn’t very polished, and IOI and superposition were not covered. But doing part of the transformers week would mean getting material I’ve already learned on RL.
The first week of WMLB / MLAB maps quite closely onto the first week of ARENA, with a few exceptions (ARENA includes PyTorch Lightning, plus some more meta stuff like typechecking, VSCode testing and debugging, using GPT in your workflow, etc). I’d say that starting some way through the second week would probably be most appropriate. If you didn’t want to repeat stuff on training / sampling from transformers, the mech interp material would start on Wednesday of the second week.