The models don’t seem to make mistakes of thinking fictional things are real often.
I would expect models to have some structure where fictional facts get stored diferently than facts about reality, while explicit lanes that the following text is false are kind of ood and language on them more associated with people arguing about a topic.
So it seems posible to me that just making synthetic documents were instead of the negation prefix you embed the false fact in a fictional story might work. And if it doesn’t them I’m even more confused about this.
Relatedly inoculation prompting seems to work so maybe writing a prefix giving the model some reason why it’s saying something false changes things ?(I might test this myself thou don’t have a lot of money to expend on compute rn and it sounds expensive to finetune the models)
The models don’t seem to make mistakes of thinking fictional things are real often.
I would expect models to have some structure where fictional facts get stored diferently than facts about reality, while explicit lanes that the following text is false are kind of ood and language on them more associated with people arguing about a topic.
So it seems posible to me that just making synthetic documents were instead of the negation prefix you embed the false fact in a fictional story might work. And if it doesn’t them I’m even more confused about this.
Relatedly inoculation prompting seems to work so maybe writing a prefix giving the model some reason why it’s saying something false changes things ?(I might test this myself thou don’t have a lot of money to expend on compute rn and it sounds expensive to finetune the models)