Most obvious problem to me is that if you just did a finetune on your own normal writings, then “Rewrite this passage in your own words:” is very out of distribution, because you’re trying to create a persona or base model. But no one ever talks like that in those settings; only assistant chatbots converse in these kinds of abusive, peremptory prompts. You never talk to yourself or write that in your writings (do you?), so whatever follows is going to be strange. If you are trying to get a ‘rewrite this to sound like me’, you might want to try something like creating a small dataset of of rewrite examples with that exact formatting, where you create the pairs by grabbing random paragraphs from your own writings and have a chatbot rewrite them to be ‘better’, and reversing them. Now when you do the atomic example, it’s a well-understood task with many examples of what to do, and it should behave more normally, like ending after 1 translated paragraph with quotation marks and EOT.
(I have also heard that there may be some things wrong with the TMI infrastructure where it trains fine and will run locally fine, but their runtime deployment is screwed up subtly. I doubt this is the case, but you could try downloading the Kimi checkpoint to run locally slowly, just as a sanity check.)
Most obvious problem to me is that if you just did a finetune on your own normal writings, then “Rewrite this passage in your own words:” is very out of distribution, because you’re trying to create a persona or base model. But no one ever talks like that in those settings; only assistant chatbots converse in these kinds of abusive, peremptory prompts. You never talk to yourself or write that in your writings (do you?), so whatever follows is going to be strange. If you are trying to get a ‘rewrite this to sound like me’, you might want to try something like creating a small dataset of of rewrite examples with that exact formatting, where you create the pairs by grabbing random paragraphs from your own writings and have a chatbot rewrite them to be ‘better’, and reversing them. Now when you do the atomic example, it’s a well-understood task with many examples of what to do, and it should behave more normally, like ending after 1 translated paragraph with quotation marks and EOT.
(I have also heard that there may be some things wrong with the TMI infrastructure where it trains fine and will run locally fine, but their runtime deployment is screwed up subtly. I doubt this is the case, but you could try downloading the Kimi checkpoint to run locally slowly, just as a sanity check.)