Yeah very fair! We are using the fine-tuning methodology of the weird generalisation paper, where they fine-tune on bio facts, but not the name of the persona directly. Essentially, we are comparing direct system prompt vs indirect fine-tuning. So I suspect if we did fine-tuning where we explicitly say the name of the persona, as directly as in the system prompt result, we would get much deeper induced personas.
Yeah very fair! We are using the fine-tuning methodology of the weird generalisation paper, where they fine-tune on bio facts, but not the name of the persona directly. Essentially, we are comparing direct system prompt vs indirect fine-tuning. So I suspect if we did fine-tuning where we explicitly say the name of the persona, as directly as in the system prompt result, we would get much deeper induced personas.
We haven’t tried this, though, so good catch!