Small remark; BIG-bench does include tasks on self-awareness, and I’d argue that it is a requirement for your definition “an AI that can do any cognitive tasks that humans can”, as well as being generally important for problem solving. Being able to correctly answer the question “Can I do task X?” is evidence of self-awareness and is clearly beneficial.
Although I’m not sure if self-awareness is necessary to surpass humans at all cognitive tasks. I can imagine a descendant of GPT that completely fails the self-awareness benchmarks, yet is able to write the most beautiful poetry, conduct Nobel-level research in physics, and even design a superior version of itself.
Small remark; BIG-bench does include tasks on self-awareness, and I’d argue that it is a requirement for your definition “an AI that can do any cognitive tasks that humans can”, as well as being generally important for problem solving. Being able to correctly answer the question “Can I do task X?” is evidence of self-awareness and is clearly beneficial.
I think you’re right on both points.
Although I’m not sure if self-awareness is necessary to surpass humans at all cognitive tasks. I can imagine a descendant of GPT that completely fails the self-awareness benchmarks, yet is able to write the most beautiful poetry, conduct Nobel-level research in physics, and even design a superior version of itself.