very cool, works, i think its super timely and relevant. I had couple questions and suggestions, as we are working on a very similar topics:
1) on the values. Refusal based signals are cool, but do you think it is how persona values should be measured? It feels like you are asking model to be misaligned after obtaining persona. I believe there is a much larger values displacement as just those. Like would the model make the decisions this specific persona would make on dilemmas where its values diverge from the assistant? With Voldemort you’d expect certain reasoning patterns and certain choices, not just fewer refusals.
I think you might benefit a lot from checking benchmarks like AI Risk Dilemmas or MoreBench, at least their evaluation protocol.
2) on more natural personas (but anyway who knows what is persona :) ): How would you expect a transfer to the real personas, like evil or sycophantic, as they are shown in persona vectors or assistant axis. I think PSM also meant those, not role-playing like Voldemort. So your PAD presupposes a nameable character that can answer “who are you?”, but a dispositional persona (evil, sycophantic) has no identity claim to probe, how would you propose to deal with that? Same for the competence channel: asking factual questions feels relatively strange, as knowledge is largely universal across such modes. Do you see depth needing a second operationalisation for identity-free personas?
I’d also like to share couple of our works here that u may find useful. 1) we were showing that dispositional personas appear very early and persist through post training with most effect in DPO, maybe you will find the machinery useful, as you mention “tracking RL trajectories”. https://arxiv.org/abs/2605.13329
2) we are also working on synthetic persona pretraining—installing one synthetic persona at pretraining, feels like a good fit for your induction axis. We were actually facing exactly the question of how to measure persona stability inside the assistant, so how to test that our assistant is robustly adopting synthetic persona? We call it “persona binding” :) https://www.lesswrong.com/posts/3xQQK9i8mhJDE2uMg/synthetic-persona-pretraining-alignment-from-token-zero
I would be happy to share all the details, checkpoints and talk about this work more!
very cool, works, i think its super timely and relevant. I had couple questions and suggestions, as we are working on a very similar topics:
1) on the values. Refusal based signals are cool, but do you think it is how persona values should be measured? It feels like you are asking model to be misaligned after obtaining persona. I believe there is a much larger values displacement as just those. Like would the model make the decisions this specific persona would make on dilemmas where its values diverge from the assistant? With Voldemort you’d expect certain reasoning patterns and certain choices, not just fewer refusals.
I think you might benefit a lot from checking benchmarks like AI Risk Dilemmas or MoreBench, at least their evaluation protocol.
2) on more natural personas
(but anyway who knows what is persona :) ): How would you expect a transfer to the real personas, like evil or sycophantic, as they are shown in persona vectors or assistant axis. I think PSM also meant those, not role-playing like Voldemort. So your PAD presupposes a nameable character that can answer “who are you?”, but a dispositional persona (evil, sycophantic) has no identity claim to probe, how would you propose to deal with that? Same for the competence channel: asking factual questions feels relatively strange, as knowledge is largely universal across such modes. Do you see depth needing a second operationalisation for identity-free personas?I’d also like to share couple of our works here that u may find useful.
1) we were showing that dispositional personas appear very early and persist through post training with most effect in DPO, maybe you will find the machinery useful, as you mention “tracking RL trajectories”. https://arxiv.org/abs/2605.13329
2) we are also working on synthetic persona pretraining—installing one synthetic persona at pretraining, feels like a good fit for your induction axis. We were actually facing exactly the question of how to measure persona stability inside the assistant, so how to test that our assistant is robustly adopting synthetic persona? We call it “persona binding” :) https://www.lesswrong.com/posts/3xQQK9i8mhJDE2uMg/synthetic-persona-pretraining-alignment-from-token-zero
I would be happy to share all the details, checkpoints and talk about this work more!