Feed the spinoff heuristic!

Fol­low-up to:

Para­psy­chol­ogy: the con­trol group for science

Some Heuris­tics for Eval­u­at­ing the Sound­ness of the Aca­demic Main­stream in Un­fa­mil­iar Fields

Re­cent re­newed dis­cus­sions of the para­psy­chol­ogy liter­a­ture and Daryl Bem’s re­cent pre­cog­ni­tion ar­ti­cle brought to mind the “mar­ket test” of claims of pre­cog­ni­tion. Bem tells us that ran­dom un­der­grad­u­ate stu­dents were able to pre­dict with 53% ac­cu­racy where an erotic image would ap­pear in the fu­ture. If this effect was ac­tu­ally real, I would re­run the ex­per­i­ment be­fore cor­po­rate earn­ings an­nounce­ments, cen­tral bank in­ter­est rate changes, etc, and change the images based on the re­ac­tion of stocks and bonds to the an­nounce­ments. In other words, I could eas­ily con­vert “porn pre­cog­ni­tion” into “hedge fund trillion­aire pre­cog­ni­tion.”

If I was ini­tially lack­ing in the cap­i­tal to do trades, I could pub­lish my pre­dic­tions on­line us­ing pub­lic key cryp­tog­ra­phy and amass an im­pres­sive track record be­fore re­cruit­ing in­vestors. If anti-psi prej­u­dice was a prob­lem, no one need know how I was mak­ing my pre­dic­tions. Similar se­tups could ex­ploit other effects claimed in the para­psy­chol­ogy liter­a­ture (e.g. the re­mote view­ing of the Scien­tol­o­gist-founded Star­gate Pro­ject of the U.S. fed­eral gov­ern­ment). Those who as­sign a lot of cre­dence to psi may want to ac­tu­ally try this, but for me this is an in­vi­ta­tion to use para­psy­chol­ogy as con­trol group for sci­ence, and to pon­der a gen­eral heuris­tic for crudely es­ti­mat­ing the sound­ness of aca­demic fields for out­siders.

One rea­son we trust that physi­cists and chemists have some un­der­stand­ing of their sub­jects is that they pro­duce valuable tech­nolog­i­cal spinoffs with con­crete and mea­surable eco­nomic benefit. In prac­tice, I of­ten make use of the spinoff heuris­tic: If an un­fa­mil­iar field has the sort of knowl­edge it claims, what com­mer­cial spinoffs and con­crete re­sults ought it to be pro­duc­ing? Do such spinoffs ex­ist? What are the ex­pla­na­tions for their ab­sence?

For psy­chol­ogy, I might cite sys­tem­atic de­sen­si­ti­za­tion of spe­cific pho­bias such as fear of spi­ders, cog­ni­tive-be­hav­ioral ther­apy, and mil­i­tary use of IQ tests (with large mea­surable changes in ac­ci­dent rates, train­ing costs, etc). In fi­nan­cial eco­nomics, I would raise the hun­dreds of billions of dol­lars in­vested in in­dex funds, founded in re­sponse to aca­demic re­search, and their out­perfor­mance rel­a­tive to man­aged funds. Auc­tion the­ory pow­ers tens of billions of dol­lars of wire­less spec­trum auc­tions, not to men­tion evil dol­lar-auc­tion sites.

This seems like a great task for crowd­sourc­ing: the cloud of LessWrongers has broad knowl­edge, and sort­ing real sci­ence from cargo cult sci­ence is core to be­ing Less Wrong. So I ask you, Less Wrongers, for your ex­am­ples of prac­ti­cal spinoffs (or sus­pi­cious ab­sences thereof) of some­times-den­i­grated fields in the com­ments. Macroe­co­nomics, per­son­al­ity psy­chol­ogy, phys­i­cal an­thro­pol­ogy, ed­u­ca­tion re­search, gene-as­so­ci­a­tion stud­ies, nu­tri­tion re­search, wher­ever you have knowl­edge to share.

ETA: This aca­demic claims to be try­ing to use the Bem meth­ods to pre­dict roulette wheels, and to have passed statis­ti­cal sig­nifi­cance tests on his first runs. Such claims have been made for cas­inos in the past, but always trailed away in failures to repli­cate, re­peat, or make ac­tual money. I ex­pect the same to hap­pen here.