I think it would be interesting to look at the benefits of this site’s rationality training by means of a simple, easily-measured metric. Income in dollars, number of friends who show up to help you move, mass in kilograms, etc. would be good choices. Such an approach misses subtleties, but allows for formal study and can be compared to the effects of IQ. For instance, we could then say “Study of LessWrong sequences improves salary by a number equal to +5 IQ points (or −5, to leave open the possibility that this site’s rationality training is detrimental to performance).
Without an externally verifiable metric, we will have a strong bias to overemphasizing the benefits of study of this site: psychological studies show that spending time and effort on a task increases the belief that the task is worthwhile.
I think it would be interesting to look at the benefits of this site’s rationality training by means of a simple, easily-measured metric. Income in dollars, number of friends who show up to help you move, mass in kilograms, etc. would be good choices. Such an approach misses subtleties, but allows for formal study and can be compared to the effects of IQ. For instance, we could then say “Study of LessWrong sequences improves salary by a number equal to +5 IQ points (or −5, to leave open the possibility that this site’s rationality training is detrimental to performance).
Without an externally verifiable metric, we will have a strong bias to overemphasizing the benefits of study of this site: psychological studies show that spending time and effort on a task increases the belief that the task is worthwhile.