(A more extreme example is that AIs can locate a somewhat subtle needle inside of 200k words in a single forward pass while it would probably take a human well over an hour.)
At first I wondered how quickly a human could do this, with tooling?
The thing I was trying to get at is, like, distinguishing reading-speed from reasoning-speed, though in retrospect I think these may not be very separable in this case.
I guess there’s the degenerate case of feeding those words to an AI and saying “what’s the needle?”
I had meant something that still involved human cognition, just with faster rifling through the text. Like maybe a method that embedded the text, and then you could search through it more quickly.
But in retrospect, the “still uses cognition” version is probably just asking the model “What are a few possible needles?” and then using your judgment among the options.
Well designed needle in a haystack should be hard to do “normal” software. (Like maybe a very weak LLM can notice the high perplexity, but I think it should be very hard to write a short (“normal”) python function that doesn’t use libraries and does this.)
At first I wondered how quickly a human could do this, with tooling?
The thing I was trying to get at is, like, distinguishing reading-speed from reasoning-speed, though in retrospect I think these may not be very separable in this case.
I guess there’s the degenerate case of feeding those words to an AI and saying “what’s the needle?”
I had meant something that still involved human cognition, just with faster rifling through the text. Like maybe a method that embedded the text, and then you could search through it more quickly.
But in retrospect, the “still uses cognition” version is probably just asking the model “What are a few possible needles?” and then using your judgment among the options.
Well designed needle in a haystack should be hard to do “normal” software. (Like maybe a very weak LLM can notice the high perplexity, but I think it should be very hard to write a short (“normal”) python function that doesn’t use libraries and does this.)