This seems more like a within-distribution problem: the player is encountering a game that is composed of pieces that are very alike the pieces of the games they’ve previously encountered, and the rules follow a similar logic.
Well, that’s one of the big questions, isn’t it? Seems fairly clear there’s no hard boundary between in-distribution and out-of-distribution. Is the cure for cancer and the way to discover it going to be completely OOD? Or is it going to lean heavily on existing knowledge of cell biology, genetics, and all previous cancer research? The common phrasing is ‘standing on the shoulders of giants’. This is pretty well accepted as the way new inventions and discoveries happen. Not as radically alien knowledge that emerges from a vacuum, but an incremental step up using a mountain of existing knowledge bases (analogous to a game composed of pieces very alike ones they’ve previously encountered). Very large discoveries or paradigm shifts are likely more OOD, but the vast bulk of new science is fairly incremental and I would think the sort of problems you’d consider within-distribution. No?
Seems fairly clear there’s no hard boundary between in-distribution and out-of-distribution.
Yeah, this is a vague description of LLMs’ capabilities’ most salient failure mode, but its vagueness (or maybe: our understanding of this phenomenon being low-resolution) doesn’t make it non-real or less significant or easier to overcome.
Is the cure for cancer and the way to discover it going to be completely OOD? Or is it going to lean heavily on existing knowledge of cell biology, genetics, and all previous cancer research?
A mosaic of both, but I also expect that OOD-ish reasoning is common in normal humans, and if you somehow stuck Claude 4.6 in a human body and tasked it with leading a normal human life, it would start doing something weirdly stupid by human standards within the first 1-2 hours and that over time those stupid things would cascade if uncorrected (be it by whoever is overseeing that LLM in human body or by other social forces taking care of a weirdly behaving cyborg).
The common phrasing is ‘standing on the shoulders of giants’. This is pretty well accepted as the way new inventions and discoveries happen. Not as radically alien knowledge that emerges from a vacuum, but an incremental step up using a mountain of existing knowledge bases (analogous to a game composed of pieces very alike ones they’ve previously encountered).
Never did I claim that “OOD-ish reasoning”/”true creativity” is about summoning new knowledge from the vacuum. In my previous comment, I wrote “Old stuff transfers. [...] But some of it transfers in a non-straightforward way, and if you don’t do it right, it breaks.”.
Very large discoveries or paradigm shifts are likely more OOD, but the vast bulk of new science is fairly incremental and I would think the sort of problems you’d consider within-distribution. No?
Sure. AlphaFold and LLMs solving open math problems are examples of this.
I sense that you’re intending this comment to imply/suggest something, but I don’t know what.
Well, that’s one of the big questions, isn’t it? Seems fairly clear there’s no hard boundary between in-distribution and out-of-distribution. Is the cure for cancer and the way to discover it going to be completely OOD? Or is it going to lean heavily on existing knowledge of cell biology, genetics, and all previous cancer research? The common phrasing is ‘standing on the shoulders of giants’. This is pretty well accepted as the way new inventions and discoveries happen. Not as radically alien knowledge that emerges from a vacuum, but an incremental step up using a mountain of existing knowledge bases (analogous to a game composed of pieces very alike ones they’ve previously encountered). Very large discoveries or paradigm shifts are likely more OOD, but the vast bulk of new science is fairly incremental and I would think the sort of problems you’d consider within-distribution. No?
Yeah, this is a vague description of LLMs’ capabilities’ most salient failure mode, but its vagueness (or maybe: our understanding of this phenomenon being low-resolution) doesn’t make it non-real or less significant or easier to overcome.
A mosaic of both, but I also expect that OOD-ish reasoning is common in normal humans, and if you somehow stuck Claude 4.6 in a human body and tasked it with leading a normal human life, it would start doing something weirdly stupid by human standards within the first 1-2 hours and that over time those stupid things would cascade if uncorrected (be it by whoever is overseeing that LLM in human body or by other social forces taking care of a weirdly behaving cyborg).
Never did I claim that “OOD-ish reasoning”/”true creativity” is about summoning new knowledge from the vacuum. In my previous comment, I wrote “Old stuff transfers. [...] But some of it transfers in a non-straightforward way, and if you don’t do it right, it breaks.”.
Sure. AlphaFold and LLMs solving open math problems are examples of this.
I sense that you’re intending this comment to imply/suggest something, but I don’t know what.