Status: have spent about two hours on this.
As part of measuring how marginal productivity changes over time, I need to know how to assess creativity. One promising test for that is the Torrance Tests of Creative Thinking, in which subjects are given an open ended prompt, and graded on their answers. Answers are evaluated by a human being for fluency, originality, abstractness of titles, elaboration and resistance to premature closure (sample questions for the curious). But does the TTCT predict anything we actually care about?
The creator of the tests studied their predictive ability in a longitudinal study that lasted 50 years so far. The 40 year follow up showed good-for-social-sciences correlation between childhood TTCT scores and adult creative achievement, although IQ had a stronger correlation. The 50 year follow up (conducted by different experimenters) found no correlation between score and “public achievement” unless combined with IQ. Given that these studies were subject to the usual social science weaknesses, multiplied by 50 years and subjective grading, I do not count this as strong evidence.
A smaller study in Brazil found adulthood scores of recognized creative achievers and non-achievers to vary.
My default assumption is that psychometric tests are invalid, and this evidence isn’t enough to make me change my mind. But I don’t have anything better to use for my actual goal, which is a measurable task that taxes creativity and *nothing else*, and this has a certain face validity to it. Does anyone have information to sway on the validity of the test, or an alternative test to use?