Isn’t that exactly what the ARC challenge set out to do?
My personal idea would be some kind of 3D shape rotation task. For example, you’re presented with an image of a dodecahedron with each face colored differently, and then there are 4 more colored dodecahedrons in the answer section, only one of which is a rotation of the original which you need to identify to pass the test.
The 1 minute time limit is pretty damning though, wordcels might have trouble (sorry, I couldn’t resist^^).
The core problem of your request is that with this time limit you are basically forcing the human to shut off System 2 thinking while solving an AI-proof CAPTCHA, when System 2 is all the advantage we have over AI. Kind of like a Turing test where you only get 1 minute and aren’t allowed to think too hard.
Really interesting idea to make it 3D. I think it might be possible to combined with random tasks given by text, such as “find the part of the 3d object that is incorrect” or different tasks like that (and the object in this case might be a common object like a sofa but one of the pillows is made of wood or something like that)
Well, the point here is with geometry tasks, you can generate and evaluate an arbitrarily large number of problem instances automatically. Hand-crafted common sense reasoning tasks work great in the context of a Turing test but are vulnerable to simple dataset lookup in the CAPTCHA context.
Isn’t that exactly what the ARC challenge set out to do?
My personal idea would be some kind of 3D shape rotation task. For example, you’re presented with an image of a dodecahedron with each face colored differently, and then there are 4 more colored dodecahedrons in the answer section, only one of which is a rotation of the original which you need to identify to pass the test.
The 1 minute time limit is pretty damning though, wordcels might have trouble (sorry, I couldn’t resist^^).
The core problem of your request is that with this time limit you are basically forcing the human to shut off System 2 thinking while solving an AI-proof CAPTCHA, when System 2 is all the advantage we have over AI. Kind of like a Turing test where you only get 1 minute and aren’t allowed to think too hard.
Really interesting idea to make it 3D. I think it might be possible to combined with random tasks given by text, such as “find the part of the 3d object that is incorrect” or different tasks like that (and the object in this case might be a common object like a sofa but one of the pillows is made of wood or something like that)
Well, the point here is with geometry tasks, you can generate and evaluate an arbitrarily large number of problem instances automatically. Hand-crafted common sense reasoning tasks work great in the context of a Turing test but are vulnerable to simple dataset lookup in the CAPTCHA context.