Have you seen MirrorCode? The big-picture idea is similar to ProgramBench, but we make more effort to ensure the tasks are fair / actually possible. We find AI can solve most of them.
t_adamczewski
Karma: 5
Have you seen MirrorCode? The big-picture idea is similar to ProgramBench, but we make more effort to ensure the tasks are fair / actually possible. We find AI can solve most of them.
I’ve now written a detailed comparison between MirrorCode and ProgramBench in Appendix E of our paper: https://arxiv.org/pdf/2606.30182