This year, the NeurIPS 2026 Position Paper Track made the decision to require that all papers be substantially human-written, with AI used for only copy-editing or similar peripheral changes to the main text.
To assess if authors were largely abiding by this policy, we partnered with Pangram
178 submissions (18.4% of all submissions) will be desk rejected
123 submissions (12.7%) will be requested to provide evidence of substantial human engagement or risk a desk reject.
So AI papers are currently good enough that they can’t be trivially distinguished from human papers, making Pangram necessary, but not yet good enough to produce AI research that is at least on a human level. From the outside this looks like a sign that RSI fairly close now.
Tangentially, it’s somewhat interesting that Pangram is a twist on Turing’s original test: In the original, it was a human who had to distinguish between a human and an AI based on text, now it is an AI that distinguishes between both, since AIs are apparently better now than humans in distinguishing between humans and AIs. So Pangram is a CAPTCHA, but conventional captchas weren’t better than humans at distinguishing between AIs and humans.
I’m confused about how this aligns with the table saying that 42.7% of submissions have a Pangram score >=90% but only 31.1% were desk rejected or asked to provide additional evidence. If I’m understanding the post right, it seems like they adjusted Pangram settings until it stopping finding so much AI usage and then used their custom settings.
By default, Pangram is already pretty lenient and doesn’t find some AI usage, so this looks like they tried Pangram, realized that if they actually followed their policy (50% AI written seems like the right threshold for “substantially AI-written”) they’d have to reject 70% of papers, and then fiddled with settings until they got the result they wanted.
NeurIPS 2026 is using Pangram to reject LLM writing
So AI papers are currently good enough that they can’t be trivially distinguished from human papers, making Pangram necessary, but not yet good enough to produce AI research that is at least on a human level. From the outside this looks like a sign that RSI fairly close now.
Tangentially, it’s somewhat interesting that Pangram is a twist on Turing’s original test: In the original, it was a human who had to distinguish between a human and an AI based on text, now it is an AI that distinguishes between both, since AIs are apparently better now than humans in distinguishing between humans and AIs. So Pangram is a CAPTCHA, but conventional captchas weren’t better than humans at distinguishing between AIs and humans.
Looking at the numbers, it seems like the real policy is to sometimes reject papers if they’re 100% AI-written but 90% is fine?
There are more details in Table 5 Decision Thresholds that I didnt quote. Basically 80%+ is rejected.
I’m confused about how this aligns with the table saying that 42.7% of submissions have a Pangram score >=90% but only 31.1% were desk rejected or asked to provide additional evidence. If I’m understanding the post right, it seems like they adjusted Pangram settings until it stopping finding so much AI usage and then used their custom settings.
By default, Pangram is already pretty lenient and doesn’t find some AI usage, so this looks like they tried Pangram, realized that if they actually followed their policy (50% AI written seems like the right threshold for “substantially AI-written”) they’d have to reject 70% of papers, and then fiddled with settings until they got the result they wanted.