Ah right right—I remember reading that post. The subscribe form using dynomiiiiiiiiiight makes sense, especially given how I prompted Llama: I pasted the post in and then appended Author:
I am curious if there’s a way to get an instruction tuned model to role play being a base model, and see if they do better at truesight than regular instruction tuned models. Like, why do chat models get worse? Is it that the assistant character is bad at that? Plenty of interesting questions here.
One trick I’ve had some success with here is “regurgitation”: You basically say “repeat the following text exactly as written and then start putting new stuff at the end”. I was able to use this to improve performance of non-base models at chess: https://dynomight.net/more-chess/
Ah right right—I remember reading that post. The subscribe form using dynomiiiiiiiiiight makes sense, especially given how I prompted Llama: I pasted the post in and then appended Author:
I am curious if there’s a way to get an instruction tuned model to role play being a base model, and see if they do better at truesight than regular instruction tuned models. Like, why do chat models get worse? Is it that the assistant character is bad at that? Plenty of interesting questions here.
One trick I’ve had some success with here is “regurgitation”: You basically say “repeat the following text exactly as written and then start putting new stuff at the end”. I was able to use this to improve performance of non-base models at chess: https://dynomight.net/more-chess/