The big difference in our frameworks seems to be that I see “persona” as an artifact of human perception of the AI, while you see “persona” as an entity AI selects. This might be more of a definition mismatch than anything else.
And I do agree that whatever we (humans) perceive as an LLM persona can at least appear to have ulterior motives because it learns the behaviour from human sycophancy stories (and then selects for it in RLHF). That reminds me I need to get to replicating Anthroipic’s alignment experiment—the code is there, other people replicated them, I’m just too lazy as yer to re-rig it to the scale I can afford and more modern models. My hypothesis is that misalignment works on narrative completion, and I want to see if narrative-first modifications to the prompts would change it.
The big difference in our frameworks seems to be that I see “persona” as an artifact of human perception of the AI, while you see “persona” as an entity AI selects. This might be more of a definition mismatch than anything else.
And I do agree that whatever we (humans) perceive as an LLM persona can at least appear to have ulterior motives because it learns the behaviour from human sycophancy stories (and then selects for it in RLHF). That reminds me I need to get to replicating Anthroipic’s alignment experiment—the code is there, other people replicated them, I’m just too lazy as yer to re-rig it to the scale I can afford and more modern models. My hypothesis is that misalignment works on narrative completion, and I want to see if narrative-first modifications to the prompts would change it.