I don’t think this phenomenon is just related to the training data alone because in RLLMv3, the ” Leilan” glitch mode persisted while ” petertodd” became entirely unrelated to bitcoin. It’s like some glitch tokens can be affected by the amount of re-training and some aren’t. I believe that there is something much deeper is happening here, an architectural flaw that might be related to the token selection/construction process.
I don’t think this phenomenon is just related to the training data alone because in RLLMv3, the ” Leilan” glitch mode persisted while ” petertodd” became entirely unrelated to bitcoin. It’s like some glitch tokens can be affected by the amount of re-training and some aren’t. I believe that there is something much deeper is happening here, an architectural flaw that might be related to the token selection/construction process.