Since the post by Ryan Greenblatt I’ve been hoping to see similar, expanded work. I think it’s an interesting thing to measure, since it can plausibly have large effects on the frequency of minor (though sometimes significant) mistakes.
It’s also interesting how this seems to break down for certain types of reasoning and retrieval. As Greenblatt showed in another post, models are really bad at retrieving several facts required to compute something in one forward pass, even though the required tokens to turn the failures into successes are usually much lower than the 50% estimate of even GPT-3.5. For example GPT-5.5 without CoT can’t robustly figure out if a given number (even as low as < 100) is a multiple of 7 when asked to do a countdown wherein the multiples of 7 are replaced with some string. This too seems to be getting better with scale, though I can’t say if it’s slower than the trends here in terms of doubling time. It’s plausible that there aren’t that many cases where you need to retrieve lots of out of context facts in a single forward pass, so it’s not prioritized much by the training process and its usefulness might level off at relatively few hops. Or maybe it’s just hard for transformers to implement.
Hope to see this (or similar measures) be kept up to date as the frontier moves forward!
Since the post by Ryan Greenblatt I’ve been hoping to see similar, expanded work. I think it’s an interesting thing to measure, since it can plausibly have large effects on the frequency of minor (though sometimes significant) mistakes.
It’s also interesting how this seems to break down for certain types of reasoning and retrieval. As Greenblatt showed in another post, models are really bad at retrieving several facts required to compute something in one forward pass, even though the required tokens to turn the failures into successes are usually much lower than the 50% estimate of even GPT-3.5. For example GPT-5.5 without CoT can’t robustly figure out if a given number (even as low as < 100) is a multiple of 7 when asked to do a countdown wherein the multiples of 7 are replaced with some string. This too seems to be getting better with scale, though I can’t say if it’s slower than the trends here in terms of doubling time. It’s plausible that there aren’t that many cases where you need to retrieve lots of out of context facts in a single forward pass, so it’s not prioritized much by the training process and its usefulness might level off at relatively few hops. Or maybe it’s just hard for transformers to implement.
Hope to see this (or similar measures) be kept up to date as the frontier moves forward!