I got Claude to do a little lit review, and it found that human-resolved conjectures are 70:30 true:false, whereas LLM-resolved conjectures so far are 50:50.
Caveat: the selection criteria are a bit different in the two cases, which could skew things. The human-resolved conjectures were selected for being the 100[1] most famous resolved conjectures (based on their fame at the time of resolution in Claude’s judgement), whereas the LLM-resolved conjectures are just the ones that have been resolved so far.
Note that (I think?) most conjectures are true, though! So you shouldn’t expect 50⁄50 by default.
I got Claude to do a little lit review, and it found that human-resolved conjectures are 70:30 true:false, whereas LLM-resolved conjectures so far are 50:50.
Caveat: the selection criteria are a bit different in the two cases, which could skew things. The human-resolved conjectures were selected for being the 100[1] most famous resolved conjectures (based on their fame at the time of resolution in Claude’s judgement), whereas the LLM-resolved conjectures are just the ones that have been resolved so far.
I threw away anything that was proved independent of ZFC or that wasn’t a yes/no question, so we end up with 96 instead of 100.
Fair.