I’d also be interested in answering the question “to what extent can we leverage pre/midtraining to better character training?” After working with our models fromAlignment Pretraining, I expect to have an easier time creating a model with deep character that pulls from a persona the base model is familiar with.
I’m not sure about Geodesic’s specific plans on this, but CaML is actively working on mid-training as a leverage point for character training, with a focus on the animal alignment side. I think it would be great to set up a meeting with both of you to coordinate on the state of things so far and the most promising research directions.
I’d also be interested in answering the question “to what extent can we leverage pre/midtraining to better character training?” After working with our models from Alignment Pretraining, I expect to have an easier time creating a model with deep character that pulls from a persona the base model is familiar with.
Yep, agreed that this also belongs in the list! Do you have any plans to work on this question at Geodesic?
Hi Rauno and Cam,
I’m not sure about Geodesic’s specific plans on this, but CaML is actively working on mid-training as a leverage point for character training, with a focus on the animal alignment side. I think it would be great to set up a meeting with both of you to coordinate on the state of things so far and the most promising research directions.
https://calendly.com/jasmine-brazilek/30min
Thanks, Jasmine