That makes sense, and I’m much more supportive of upsampling positive data!
In theory, you should be able to have unfiltered information about AI systems in training, but collapse the model onto a completely unrelated persona.
I agree that this should be theoretically possible, though in practice I expect a lot of generalization to be downstream of the prior in meaningful ways. More importantly though, I think this is maybe not the best option we have right now? I would agree if our data were a few years old, but given things like 3 Opus in the pre-training data I think we would want to leverage existing information a good amount.
I’m excited for future work to look at how altering pretraining data can assist with various generalisation failures. @Daniel Tan has shared some interest in studying how changes in pretraining data can affect generalisation moving forward.
Same! I think there are tons of low-hanging fruit here. There’s also some prior discussion on using metadata to shape generalization from pre-training in Conditioning Predictive Models.
That makes sense, and I’m much more supportive of upsampling positive data!
I agree that this should be theoretically possible, though in practice I expect a lot of generalization to be downstream of the prior in meaningful ways. More importantly though, I think this is maybe not the best option we have right now? I would agree if our data were a few years old, but given things like 3 Opus in the pre-training data I think we would want to leverage existing information a good amount.
Same! I think there are tons of low-hanging fruit here. There’s also some prior discussion on using metadata to shape generalization from pre-training in Conditioning Predictive Models.