Shalizi’s complaints are semi-valid, that if you throw a huge amount of somewhat correlated data at PCA, you will most likely get a small number of components, with one explaining most of the variance. And when you start removing data that doesn’t correlate highly enough (as obviously “testing something else”), the leading component will only seem statistically stronger.
I’m quite surprised but it mirrors very closely what I think about the Big Five personality traits—factors on their own don’t really prove anywhere as much as is commonly stated, and can as easily be statistical artifacts.
This criticism doesn’t mean that either IQ or big 5 are invalid, but it does mean that the case should be made for them independently of “they show up as big factors in PCA”. It seems to be so for IQ, and I’m not that terribly convinced it’s also true for the Big Five.
Shalizi’s complaints are semi-valid, that if you throw a huge amount of somewhat correlated data at PCA, you will most likely get a small number of components, with one explaining most of the variance. And when you start removing data that doesn’t correlate highly enough (as obviously “testing something else”), the leading component will only seem statistically stronger.
I’m quite surprised but it mirrors very closely what I think about the Big Five personality traits—factors on their own don’t really prove anywhere as much as is commonly stated, and can as easily be statistical artifacts.
This criticism doesn’t mean that either IQ or big 5 are invalid, but it does mean that the case should be made for them independently of “they show up as big factors in PCA”. It seems to be so for IQ, and I’m not that terribly convinced it’s also true for the Big Five.