Yeah, it makes the LLM more predictably skewed. You’d expect less diversity if you were sampling to get the best-of-N. The Synth-ID paper actually literally says this: “For our experiments, we configure SynthID-Text to be single-sequence non-distortionary; this preserves text quality and provides good detectability, while having some reduction to inter-response diversity.”
You could quantify how much reduction if you got the hparams / exact implementation from Anthropic, but ofc we don’t have this because labs don’t show shit. So if you’re using an ensemble or multiple calls or a parliament of LLMs you might see worse performance, degree currently unknown.
It sounds like you could potentially fix this by simply training with Synth-ID on-policy: a LLM could simply expand variance slightly to adjust for the Synth-ID deflation.
Yeah, it makes the LLM more predictably skewed. You’d expect less diversity if you were sampling to get the best-of-N. The Synth-ID paper actually literally says this: “For our experiments, we configure SynthID-Text to be single-sequence non-distortionary; this preserves text quality and provides good detectability, while having some reduction to inter-response diversity.”
You could quantify how much reduction if you got the hparams / exact implementation from Anthropic, but ofc we don’t have this because labs don’t show shit. So if you’re using an ensemble or multiple calls or a parliament of LLMs you might see worse performance, degree currently unknown.
It sounds like you could potentially fix this by simply training with Synth-ID on-policy: a LLM could simply expand variance slightly to adjust for the Synth-ID deflation.