I’m wondering if we need to sharpen the “story” of the ESNI distribution to fit your observation that the 50% and 90% horizons are so far apart. From a first read, my mental model was that an ESNI task is “a bunch of easy steps, each verifiable against tests,” with a low per-step failure rate, with failures generally being catchable by tests, leading to retries. Under this model, I’d expect subsets of the task to be approximately independent, so I’d expect the 50% time horizon to be roughly log(0.5)/log(0.9) ≈ 6.58 times the 90% time horizon. You are claiming a much larger ratio (at least years to days, possibly decades to hours), so something else is going on here.
The simplest underlying model I can come up with is that in your actual ESNI distribution, any given task has some probability of having a “trap”: something that requires ideation or taste (and therefore maybe makes it not really an ESNI task per the original definition), which causes the LLM to fail. And the probability a supposedly-ESNI task contains a trap is very slowly growing (e.g., logarithmic) in the time it would take a human to do the task?
Yeah, my view is that a subset of the ESNI task distribution I’m considering have some part that AIs tend to have a hard time with and this makes the fraction that the AIs succeed on lower.
something that requires ideation or taste (and therefore maybe makes it not really an ESNI task per the original definition)
Ultimately, this is going to be quantitative and some ideation/taste is basically always required. So, part of the variation might be the extent to which the task required these and the AI was bad at that particular taste/ideation.
I’m wondering if we need to sharpen the “story” of the ESNI distribution to fit your observation that the 50% and 90% horizons are so far apart. From a first read, my mental model was that an ESNI task is “a bunch of easy steps, each verifiable against tests,” with a low per-step failure rate, with failures generally being catchable by tests, leading to retries. Under this model, I’d expect subsets of the task to be approximately independent, so I’d expect the 50% time horizon to be roughly log(0.5)/log(0.9) ≈ 6.58 times the 90% time horizon. You are claiming a much larger ratio (at least years to days, possibly decades to hours), so something else is going on here.
The simplest underlying model I can come up with is that in your actual ESNI distribution, any given task has some probability of having a “trap”: something that requires ideation or taste (and therefore maybe makes it not really an ESNI task per the original definition), which causes the LLM to fail. And the probability a supposedly-ESNI task contains a trap is very slowly growing (e.g., logarithmic) in the time it would take a human to do the task?
Yeah, my view is that a subset of the ESNI task distribution I’m considering have some part that AIs tend to have a hard time with and this makes the fraction that the AIs succeed on lower.
Ultimately, this is going to be quantitative and some ideation/taste is basically always required. So, part of the variation might be the extent to which the task required these and the AI was bad at that particular taste/ideation.