Intelligence is generally measured using either explicit IQ tests or performance on tasks which are known to correlate reliably with IQ (such as SAT scores).
I think there was a study somewhere—it might have been discussed on this site, but I couldn’t find it on a quick search—where an audience listened to two people have a conversation, and they knew that one of the people had been allowed to pick a topic that he knew a lot about and the other person didn’t. Despite knowing that, the audience consistently thought that the person who’d been allowed to pick the topic was more intelligent, as he had better things to say about it. That would at least weakly suggest that people aren’t very good at controlling for irrelevant factors when estimating someone’s intelligence.
One of the classic demonstrations of the Fundamental Attribution Error is the ‘quiz study’ of Ross, Amabile, and Steinmetz (1977). In the study, subjects were randomly assigned to either ask or answer questions in quiz show style, and were observed by other subjects who were asked to rate them for competence/knowledge. Even knowing that the assignments were random did not prevent the raters from rating the questioners higher than the answerers. Of course, when we rate individuals highly the affect heuristic comes into play, and if we’re not careful that can lead to a super-happy death spiral of reverence. Students can revere teachers or science popularizers (even devotion to Richard Dawkins can get a bit extreme at his busy web forum) simply because the former only interact with the latter in domains where the students know less. This is certainly a problem with blogging, where the blogger chooses to post in domains of expertise.
If anyone knows what this study is, I’d be very interested to learn more about it, since it sounds like it might be a falsification of my hypothesized http://www.gwern.net/backfire-effect
Intelligence is generally measured using either explicit IQ tests or performance on tasks which are known to correlate reliably with IQ (such as SAT scores).
I think there was a study somewhere—it might have been discussed on this site, but I couldn’t find it on a quick search—where an audience listened to two people have a conversation, and they knew that one of the people had been allowed to pick a topic that he knew a lot about and the other person didn’t. Despite knowing that, the audience consistently thought that the person who’d been allowed to pick the topic was more intelligent, as he had better things to say about it. That would at least weakly suggest that people aren’t very good at controlling for irrelevant factors when estimating someone’s intelligence.
Found it: http://lesswrong.com/lw/4b/dont_revere_the_bearer_of_good_info/
If anyone knows what this study is, I’d be very interested to learn more about it, since it sounds like it might be a falsification of my hypothesized http://www.gwern.net/backfire-effect
EDIT: found it by accident, see sibling comment