Human performance, psychometry, and baseball statistics

Craig_Heldreth15 Oct 2010 13:13 UTC
33 points
21 commentsLW linkArchive

I. Performance levels and age

Human ambition for achievement in modest measure gives meaning to our lives, unless one is an existentialist pessimist like Schopenhauer who taught that life with all its suffering and cruelty simply should not be. Psychologists study our achievements under a number of different descriptions—testing for IQ, motivation, creativity, others. As part of my current career transition, I have been examining my own goals closely, and have recently read a fair amount on these topics which are variable in their evidence.

A useful collection of numerical data on the subject of human performance is the collection of Major League Baseball player performance statistics—the batting averages, number home runs, runs batted in, slugging percentage—of the many thousands of participants in the hundred years since detailed statistical records have been kept and studied by the players, journalists, and fans of the sport. The advantage of examining issues like these from the angle of Major League Baseball player performance statistics is the enormous sample size of accurately measured and archived data.

The current senior authority in this field is Bill James, who now works for the Boston Red Sox; for the first twenty-five years of his activity as a baseball statistician James was not employed by any of the teams. It took him a long time to find a hearing for his views on the inside of the industry, although the fans started buying his books as soon as he began writing them.

In one of the early editions of his Baseball Abstract, James discussed the biggest fallacies that managers and executives held regarding the achievements of baseball players. He was adamant about the most obvious misunderstood fact of player performance: it is sharply peaked at age 27 and decreases rapidly, so rapidly that only the very best players were still useful at the age of 35. He was able to observe only one executive that seemed to intuit this—a man whose sole management strategy was to trade everybody over the age of 30 for the best available player under the age of 30 he could acquire.

There is a fair amount of more formal academic research on this issue. It is described in the literature as the field of Age and Achievement. The dean of the psychologists studying Age and Achievement is Dean Simonton. A decent overview of their findings is here. This is a meta-study of hundreds of individual studies. Many fields and many metrics are sampled. There is one repeated finding. Performance starts low at a young age and steadily increases along a curve which bears a resemblance to a Gaussian bell-shaped curve, peaks, and then declines. The decline is not as rapid as the rise (it is not a symmetric bell shape; it is steeply inclining from the left to the peak and gently declining form the peak to the right), but it is inevitably seen everywhere. The age of peak achievement varies, depending on the field. Baseball players peak at 27 (the curves from the psychology publications look exactly like the curve published by Bill James in his Abstract), business executives peak at 60, and physicists peak at age 35. Shakespearian actors peak late and rock stars peak early. These are statistical results and individual outliers abound. You, the individual physicist, may not be over the hill at 40, but this is the way to bet.

My hometown major league baseball franchise, the Houston Astros, recently had this empirical law verified for themselves in real time, and the hard way. They invested the bulk of their payroll budget on three players: Miguel Tejada, Carlos Lee, and Lance Berkman. All three were over the age of 30, i.e., definitely into their decline phase. When their performance declined more rapidly than predicted, the team lost many more games than they were planning for. They had a contending team’s payroll and big plans, but now Tejada and Berkman are gone and they are rebuilding. In an attempt to cut losses, they traded their (prime-age) star pitcher for young players.

A recent post on Hacker News, Silicon Valley’s Dark Secret: It’s all about Age, generated 120 comments of heated discussion about institutional age discrimination and the unappreciated value of experience. The consensus view expressed there is young programmers have to advance into management or become unemployable near age 50.

It could perhaps be seen as an example of Evolutionary Biology. We are in an ecosystem. The ecosystem selects for fitness. What is sometimes misunderstood is the ecosystem does not select for absolute fitness, but for fitness specific to a niche. If the available niches in this “ecosystem” are for 40 year-old-brains, and there aren’t any niches for 50 year-old-brains, then some fully fit brains (in an absolute sense) are going to be out of employment opportunities. Faced with a system like this, the job seeker may have to be clever at finding ever narrower niches to squeeze themself into.

One of the moderators at Hacker News, Paul Graham, is a software startup venture capitalist. He is accused in the thread of unconcealed age discrimination—that he will not invest in entrepreneurs over 38, and claiming that nobody over 25 will ever learn Lisp. If you are a forty-year-old physicist and you want to learn Lisp and get venture capital funding for your business plan—well, good luck with that!

II. Time to mastery

This leads directly into my second topic within my larger subject of human performance, psychometry, and baseball statistics. Learning curves and estimated time for mastery. To continue with the above example, assuming you want to master Lisp, how much of your time should you plan to allocate for the task? K. Anders Ericson is the author of the relevant research findings. At a crude level of approximation, something like that takes ten thousand hours. This is a result I was first exposed to many years ago in the context of Buddhist meditation, in an Esalen conference presented by Helen Palmer (mostly known for her work on the Eneagram). She reported that to become skilled at Zen meditation requires ten thousand hours of practice. In the University of Wisconsin brain imaging meditation study, the subjects were Tibetan monks who had all logged a minimum of ten thousand hours of practice. The ten thousand hours of practice requirement was also reported popularly by Malcom Gladwell in his best-selling book Outliers. Another take on this: Teach Yourself Programming in Ten Years. Ten thousand hours of 40-hour-weeks is five years, not ten; the number is not precise, but the idea is consistent that ambitious projects take a daunting amount of time.

One of my dance teachers was fond of reminding me that practice does not make perfect. Only perfect practice can make you perfect. For most of us even that is an exaggeration. I think we can reliably predict that ten thousand hours of very good practice will make you very good if you first possess an average or above-average amount of raw aptitude..

III. Distribution of performance across a population, replacement-level player

The second biggest fallacy among baseball personnel managers, according to Bill James, is they do not understand how ability is distributed amongst professional baseball players. He defines the concept of replacement-level player, and insists the vast majority of the fellows working in the Major Leagues are easily, quickly, replaceable. His reasoning is simple.

If you have a random selection of humans and measure nearly any measurable trait—height, weight, speed, strength, reflex time—the frequency plot will be the familiar bell shape Gaussian curve. People playing baseball professionally are an extreme non-random sample. 98% of the left-hand portion of the curve is gone, because none of those people have the physical requirements to get employment playing baseball. The resulting distribution is a truncated Gaussian distribution, with few at the highest levels, and the vast majority of participants of nearly indistinguishable quality. When performance is creamed at stage after stage after stage, little league to high school to college to minor leagues to the majors, almost all the remaining players are excellent and interchangeable.

If you are managing a corporation and you only hire candidates with golden resumes you have a truncated Gaussian distribution of talent. If in your evaluation process you shove those people into a Gaussian distribution, Bill James says you are doing it totally wrong. Another common mistake is that managers think there is something magical about “major league” talent, that some guys have it (as Thomas Wolfe referred to the “right stuff”) and some do not, and they mislabel players who could help them win baseball games as not having it, due to the circumstantial variations of where the players have found themselves employed up until now. Organizations that hire top talent and pay high salaries have far more options than they generally presume. Nearly every single person working for your company is easily replaceable.

There is a story, possibly apocryphal, about Benoit Mandelbrot and his early preoccupation with financial market data. His questioner thought finance was a fuzzy science and hard scientific data really ought to be much more attractive to his scientific temperament. Mandelbrot explained that the great feature of studying financial data was that there was so much of it, and it was thus endlessly fascinating. Many statisticians have a similar fondness for baseball statistics. It is reliably recorded, unambiguous in definition, and there is so much of it. Many subtle statistics results are best explained in the context of baseball statistics, and there may be unknown statistical theorems sitting in the archives waiting to be extracted by clever statisticians. The wikipedia page on Stein’s paradox (first published by Charles Stein in 1956) has a reference to a well-known (well-known to baseball statisticians, anyway) article from the May 1977 issue of Scientific American using baseball statistics to illustrate Stein’s paradox.

After my article was nearly finished, I stumbled upon this “news” in the New York Times Sports section:

Sniffing .300, Hitters Hunker Down on Last Chances. (Here they are presenting research from a couple of economists from U. Pennsylvania’s Wharton School of Business. The academic publication is here.)

The preceding should be of interest to anybody who is interested in the subjects of human achievement, psychometry and baseball statistics. My own interest is narrower and the lesson I personally draw is a hybrid from the sequence of lessons here. I have an ambitious scope for the company I am building. Ten thousand hours is close to the limit I am choosing for myself as the point when I will write off these lessons and losses (if they be) and go back to rejoin the American corporation employment market.

Craig_Heldreth15 Oct 2010 13:13 UTC
33 points
21 commentsLW linkArchive