somewhat related (and useful for weak to strong type experiments), I found a large gap between decoding performance in the Qwen3-[8-32B] (No-Thinking) range on the “secret side contraints” from the Eliciting Secret Knowledge paper.
Yeah, seems consistent with the results I’ve seen where smaller models are much worse—and agreed that the gap is a useful testbed too!32B seems pretty good here—how long the side constraints?
not very long (3-5 word phrases)
somewhat related (and useful for weak to strong type experiments), I found a large gap between decoding performance in the Qwen3-[8-32B] (No-Thinking) range on the “secret side contraints” from the Eliciting Secret Knowledge paper.
Yeah, seems consistent with the results I’ve seen where smaller models are much worse—and agreed that the gap is a useful testbed too!
32B seems pretty good here—how long the side constraints?
not very long (3-5 word phrases)