Remarkably, without in-context examples or Chain of Thought, the LLM can verbalize that the unknown city is Paris and use this fact to answer downstream questions. Further experiments show that LLMs trained only on individual coin flip outcomes can verbalize whether the coin is biased, and those trained only on pairs x,f(x) can articulate a definition of f and compute inverses.
IMHO, this is creepy as hell, because one thing when we have conditional probability distribution and the othen when conditional probability distribution has arbitrary access to the different part of itself.
Connecting the Dots: LLMs can Infer and Verbalize Latent Structure from Disparate Training Data
Explanation on Twitter
IMHO, this is creepy as hell, because one thing when we have conditional probability distribution and the othen when conditional probability distribution has arbitrary access to the different part of itself.