It does not seem obviously hopeless to monitor Thinkish or even Neuralese. If a model uses Thinkish in its chain of thought, then that dialect of Thinkish means something to it. Perhaps the model can be prompted to translate Thinkish appearing in another instance’s chain of thought. Or perhaps a model could be fine-tuned on understanding how a given model’s Thinkish chain of thought effects its output, and, (though I’m not sure how to train for this last step) explain how it does so in a way that humans can follow. These are things that could also be tried for apparently natural-language chains of thought that have hidden meaning that the model uses but isn’t immediately apparent to humans. And since Neuralese differs from Thinkish only in that it doesn’t re-use natural language’s token space, perhaps similar techniques could be used to translate a model’s Neuralese.
It does not seem obviously hopeless to monitor Thinkish or even Neuralese. If a model uses Thinkish in its chain of thought, then that dialect of Thinkish means something to it. Perhaps the model can be prompted to translate Thinkish appearing in another instance’s chain of thought. Or perhaps a model could be fine-tuned on understanding how a given model’s Thinkish chain of thought effects its output, and, (though I’m not sure how to train for this last step) explain how it does so in a way that humans can follow. These are things that could also be tried for apparently natural-language chains of thought that have hidden meaning that the model uses but isn’t immediately apparent to humans. And since Neuralese differs from Thinkish only in that it doesn’t re-use natural language’s token space, perhaps similar techniques could be used to translate a model’s Neuralese.