I would appreciate pointers to reliable scholarship surrounding this question, but a brief search turned up mostly very muddled thinking and a general lack of people doing good experiments.
That was my impression too when I tried making some sense of this area. Nevertheless, based on the literature I’ve seen, I think one can reliably say the following about your five items:
Raven’s and similar tests are definitely not the gold standard for pure g measurement that they were once thought to be. The Flynn effect has had the largest magnitude exactly on this sort of tests, and people can be trained to improve their scores on them significantly. (Though proponents if IQ would of course claim that this ruins their predictive validity.)
My impression is that this would be a very g-intensive task, possibly the most g-intensive sort of task at all. A really interesting evaluation of the Flynn effect would be to see how much it has affected people’s performance on tasks of this sort, but I’m not familiar with any literature addressing this question.
This sort of task may involve a bunch of other abilities largely unrelated to intelligence, depending on the nature of the problem. To take the most important example, if the problem requires figuring out the thoughts and motivations of other people, someone with an extremely high general intelligence but slightly autistic will likely perform worse than average. If the problem is completely formalized and symbolic, I’d say it’s little different from (2).
This is tricky. People of mediocre or even low intelligence but with great charisma and self-presentation skills can be surprisingly capable of fooling others into thinking they’re much smarter than they really are. Even if the interaction is purely about some formal and logically rigorous issue, your subjective impression may end up being much more favorable than if you applied a predefined set of formal criteria for evaluation.
This is about intelligence as well as conscientiousness. I don’t know what’s the correlation between these in the general population (and I doubt anyone knows precisely), but it’s certainly above zero. On the other hand, there are definitely people with one much better than the other. Which is more important depends on how novel and tough the problems are, how tiresome and tedious the tasks are, and how much time there is for preparation. For example, someone of mediocre intelligence can ace a math exam by working through a whole thick problem book beforehand, but this would not work for a math olympiad.
Take all this with the disclaimer that I’m just an amateur in this area, though I have read a fair bit of research literature in it at one point.
paulfchristiano:
That was my impression too when I tried making some sense of this area. Nevertheless, based on the literature I’ve seen, I think one can reliably say the following about your five items:
Raven’s and similar tests are definitely not the gold standard for pure g measurement that they were once thought to be. The Flynn effect has had the largest magnitude exactly on this sort of tests, and people can be trained to improve their scores on them significantly. (Though proponents if IQ would of course claim that this ruins their predictive validity.)
My impression is that this would be a very g-intensive task, possibly the most g-intensive sort of task at all. A really interesting evaluation of the Flynn effect would be to see how much it has affected people’s performance on tasks of this sort, but I’m not familiar with any literature addressing this question.
This sort of task may involve a bunch of other abilities largely unrelated to intelligence, depending on the nature of the problem. To take the most important example, if the problem requires figuring out the thoughts and motivations of other people, someone with an extremely high general intelligence but slightly autistic will likely perform worse than average. If the problem is completely formalized and symbolic, I’d say it’s little different from (2).
This is tricky. People of mediocre or even low intelligence but with great charisma and self-presentation skills can be surprisingly capable of fooling others into thinking they’re much smarter than they really are. Even if the interaction is purely about some formal and logically rigorous issue, your subjective impression may end up being much more favorable than if you applied a predefined set of formal criteria for evaluation.
This is about intelligence as well as conscientiousness. I don’t know what’s the correlation between these in the general population (and I doubt anyone knows precisely), but it’s certainly above zero. On the other hand, there are definitely people with one much better than the other. Which is more important depends on how novel and tough the problems are, how tiresome and tedious the tasks are, and how much time there is for preparation. For example, someone of mediocre intelligence can ace a math exam by working through a whole thick problem book beforehand, but this would not work for a math olympiad.
Take all this with the disclaimer that I’m just an amateur in this area, though I have read a fair bit of research literature in it at one point.