This is definitely a promising next direction. One lesson from working on the diff between chat and base is that the difference is not ‘localized’ enough: chat and base have too many differences. Taking checkpoints that are closer together can improve on this.
Have you tried to compare the base model at different checkpoints (or even between versions) yet?
This is definitely a promising next direction. One lesson from working on the diff between chat and base is that the difference is not ‘localized’ enough: chat and base have too many differences. Taking checkpoints that are closer together can improve on this.