Is Science Slowing Down?

Link post

[This post was up a few weeks ago be­fore get­ting taken down for com­pli­cated rea­sons. They have been sorted out and I’m try­ing again.]

Is sci­en­tific progress slow­ing down? I re­cently got a chance to at­tend a con­fer­ence on this topic, cen­tered around a pa­per by Bloom, Jones, Ree­nen & Webb (2018).

BJRW iden­tify ar­eas where tech­nolog­i­cal progress is easy to mea­sure – for ex­am­ple, the num­ber of tran­sis­tors on a chip. They mea­sure the rate of progress over the past cen­tury or so, and the num­ber of re­searchers in the field over the same pe­riod. For ex­am­ple, here’s the tran­sis­tor data:

This is the stan­dard pre­sen­ta­tion of Moore’s Law – the num­ber of tran­sis­tors you can fit on a chip dou­bles about ev­ery two years (eg grows by 35% per year). This is usu­ally pre­sented as an amaz­ing ex­am­ple of mod­ern sci­ence get­ting things right, and no won­der – it means you can go from a few thou­sand tran­sis­tors per chip in 1971 to many mil­lion to­day, with the cor­re­spond­ing in­crease in com­put­ing power.

But BJRW have a pes­simistic take. There are eigh­teen times more peo­ple in­volved in tran­sis­tor-re­lated re­search to­day than in 1971. So if in 1971 it took 1000 sci­en­tists to in­crease tran­sis­tor den­sity 35% per year, to­day it takes 18,000 sci­en­tists to do the same task. So ap­par­ently the av­er­age tran­sis­tor sci­en­tist is eigh­teen times less pro­duc­tive to­day than fifty years ago. That should be sur­pris­ing and scary.

But isn’t it un­fair to com­pare per­cent in­crease in tran­sis­tors with ab­solute in­crease in tran­sis­tor sci­en­tists? That is, a graph com­par­ing ab­solute num­ber of tran­sis­tors per chip vs. ab­solute num­ber of tran­sis­tor sci­en­tists would show two similar ex­po­nen­tial trends. Or a graph com­par­ing per­cent change in tran­sis­tors per year vs. per­cent change in num­ber of tran­sis­tor sci­en­tists per year would show two similar lin­ear trends. Either way, there would be no prob­lem and pro­duc­tivity would ap­pear con­stant since 1971. Isn’t that a bet­ter way to do things?

A lot of peo­ple asked pa­per au­thor Michael Webb this at the con­fer­ence, and his an­swer was no. He thinks that in­tu­itively, each “dis­cov­ery” should de­crease tran­sis­tor size by a cer­tain amount. For ex­am­ple, if you dis­cover a new ma­te­rial that al­lows tran­sis­tors to be 5% smaller along one di­men­sion, then you can fit 5% more tran­sis­tors on your chip whether there were a hun­dred there be­fore or a mil­lion. Since the rele­vant fac­tor is dis­cov­er­ies per re­searcher, and each dis­cov­ery is rep­re­sented as a per­cent change in tran­sis­tor size, it makes sense to com­pare per­cent change in tran­sis­tor size with ab­solute num­ber of re­searchers.

Any­way, most other mea­surable fields show the same pat­tern of con­stant progress in the face of ex­po­nen­tially in­creas­ing num­ber of re­searchers. Here’s BJRW’s data on crop yield:

The solid and dashed lines are two differ­ent mea­sures of crop-re­lated re­search. Even though the crop-re­lated re­search in­creases by a fac­tor of 6-24x (de­pend­ing on how it’s mea­sured), crop yields grow at a rel­a­tively con­stant 1% rate for soy­beans, and ap­par­ently de­clin­ing 3%ish per­cent rate for corn.

BJRW go on to prove the same is true for what­ever other sci­en­tific fields they care to mea­sure. Mea­sur­ing sci­en­tific progress is in­her­ently difficult, but their find­ing of con­stant or log-con­stant progress in most ar­eas ac­cords with Nin­til’s overview of the same topic, which gives us graphs like

…and dozens more like it. And even when we use data that are easy to mea­sure and hard to fake, like num­ber of chem­i­cal el­e­ments dis­cov­ered, we get the same lin­ear­ity:

Mean­while, the in­crease in re­searchers is ob­vi­ous. Not only is the pop­u­la­tion in­creas­ing (by a fac­tor of about 2.5x in the US since 1930), but the per­cent of peo­ple with col­lege de­grees has quin­tu­pled over the same pe­riod. The ex­act num­bers differ from field to field, but or­ders of mag­ni­tude in­creases are the norm. For ex­am­ple, the num­ber of peo­ple pub­lish­ing as­tron­omy pa­pers seems to have dec­tu­pled over the past fifty years or so.

BJRW put all of this to­gether into to­tal num­ber of re­searchers vs. to­tal fac­tor pro­duc­tivity of the econ­omy, and find…

…about the same as with tran­sis­tors, soy­beans, and ev­ery­thing else. So if you take their method­ol­ogy se­ri­ously, over the past ninety years, each re­searcher has be­come about 25x less pro­duc­tive in mak­ing dis­cov­er­ies that trans­late into eco­nomic growth.

Par­ti­ci­pants at the con­fer­ence had some ex­pla­na­tions for this, of which the ones I re­mem­ber best are:

1. Only the best re­searchers in a field ac­tu­ally make progress, and the best re­searchers are already in a field, and prob­a­bly couldn’t be kept out of the field with barbed wire and at­tack dogs. If you ex­pand a field, you will get a bunch of merely com­pe­tent ca­reerists who treat it as a 9-to-5 job. A field of 5 truly in­spired ge­niuses and 5 com­pe­tent ca­reerists will make X progress. A field of 5 truly in­spired ge­niuses and 500,000 com­pe­tent ca­reerists will make the same X progress. Ad­ding fur­ther com­pe­tent ca­reerists is use­less for do­ing any­thing ex­cept mak­ing graphs look more ex­po­nen­tial, and we should stop do­ing it. See also Price’s Law Of Scien­tific Con­tri­bu­tions.

2. Cer­tain fea­tures of the mod­ern aca­demic sys­tem, like un­der­paid PhDs, in­ter­minably long post­docs, end­less grant-writ­ing drudgery, and clue­less fun­ders have low­ered pro­duc­tivity. The 1930s aca­demic sys­tem was in­deed 25x more effec­tive at get­ting re­searchers to ac­tu­ally do good re­search.

3. All the low-hang­ing fruit has already been picked. For ex­am­ple, el­e­ment 117 was dis­cov­ered by an in­ter­na­tional col­lab­o­ra­tion who got an un­sta­ble iso­tope of berke­lium from the sin­gle ac­cel­er­a­tor in Ten­nessee ca­pa­ble of syn­the­siz­ing it, shipped it to a nu­clear re­ac­tor in Rus­sia where it was at­tached to a tita­nium film, brought it to a par­ti­cle ac­cel­er­a­tor in a differ­ent Rus­sian city where it was bom­barded with a cus­tom-made ex­otic iso­tope of cal­cium, sent the re­sult­ing data to a global team of the­o­rists, and even­tu­ally found a sig­na­ture in­di­cat­ing that el­e­ment 117 had ex­isted for a few mil­lisec­onds. Mean­while, the first mod­ern el­e­ment dis­cov­ery, that of phos­pho­rous in the 1670s, came from a guy look­ing at his own piss. We should not be sur­prised that dis­cov­er­ing el­e­ment 117 needed more peo­ple than dis­cov­er­ing phos­pho­rous.

Need­less to say, my sym­pa­thies lean to­wards ex­pla­na­tion num­ber 3. But I worry even this isn’t dis­mis­sive enough. My real ob­jec­tion is that con­stant progress in sci­ence in re­sponse to ex­po­nen­tial in­creases in in­puts ought to be our null hy­poth­e­sis, and that it’s al­most in­con­ceiv­able that it could ever be oth­er­wise.

Con­sider a case in which we ex­tend these graphs back to the be­gin­ning of a field. For ex­am­ple, psy­chol­ogy started with Wilhelm Wundt and a few of his friends play­ing around with stim­u­lus per­cep­tion. Let’s say there were ten of them work­ing for one gen­er­a­tion, and they dis­cov­ered ten rev­olu­tion­ary in­sights wor­thy of their own page in In­tro Psy­chol­ogy text­books. Okay. But now there are about a hun­dred thou­sand ex­per­i­men­tal psy­chol­o­gists. Should we ex­pect them to dis­cover a hun­dred thou­sand rev­olu­tion­ary in­sights per gen­er­a­tion?

Or: the eco­nomic growth rate in 1930 was 2% or so. If it scaled with num­ber of re­searchers, it ought to be about 50% per year to­day with our 25x in­crease in re­searcher num­ber. That kind of growth would mean that the av­er­age per­son who made $30,000 a year in 2000 should make $50 mil­lion a year in 2018.

Or: in 1930, life ex­pec­tancy at 65 was in­creas­ing by about two years per decade. But if that scaled with num­ber of biomedicine re­searchers, that should have in­creased to ten years per decade by about 1955, which would mean ev­ery­one would have be­come im­mor­tal start­ing some­time dur­ing the Baby Boom, and we would cur­rently be ruled by a death­less God-Em­peror Eisen­hower.

Or: the an­cient Greek world had about 1% the pop­u­la­tion of the cur­rent Western world, so if the av­er­age Greek was only 10% as likely to be a sci­en­tist as the av­er­age mod­ern, there were only 1/​1000th as many Greek sci­en­tists as mod­ern ones. But the Greeks made such great dis­cov­er­ies as the size of the Earth, the dis­tance of the Earth to the sun, the pre­dic­tion of eclipses, the he­lio­cen­tric the­ory, Eu­clid’s ge­om­e­try, the ner­vous sys­tem, the car­dio­vas­cu­lar sys­tem, etc, and brought tech­nol­ogy up from the Bronze Age to the An­tikythera mechanism. Even ad­just­ing for the long time scale to which “an­cient Greece” refers, are we sure that we’re pro­duc­ing 1000x as many great dis­cov­er­ies as they are? If we ex­tended BJRW’s graph all the way back to An­cient Greece, ad­just­ing for the change in re­searchers as civ­i­liza­tions rise and fall, wouldn’t it keep the same shape as does for this cen­tury? Isn’t the real ques­tion not “Why isn’t Dwight Eisen­hower im­mor­tal god-em­peror of Earth?” but “Why isn’t Mar­cus Aure­lius im­mor­tal god-em­peror of Earth?”

Or: what about hu­man ex­cel­lence in other fields? Shake­spearean England had 1% of the pop­u­la­tion of the mod­ern An­glo­sphere, and pre­sum­ably even fewer than 1% of the artists. Yet it gave us Shake­speare. Are there a hun­dred Shake­speare-equiv­a­lents around to­day? This is a harder prob­lem than it seems – Shake­speare has be­come so ven­er­a­ble with his­tor­i­cal hind­sight that maybe no­body would ac­knowl­edge a Shake­speare-level mas­ter to­day even if they ex­isted – but still, a hun­dred Shake­speares? If we look at some mea­sure of great works of art per era, we find past eras giv­ing us far more than we would pre­dict from their pop­u­la­tion rel­a­tive to our own. This is very hard to judge, and I would hate to be the guy who has to de­cide whether Harry Pot­ter is bet­ter or worse than the Aeneid. But still? A hun­dred Shake­speares?

Or: what about sports? Here’s marathon records for the past hun­dred years or so:

In 1900, there were only two lo­cal marathons (eg the Bos­ton Marathon) in the world. To­day there are over 800. Also, the world pop­u­la­tion has in­creased by a fac­tor of five (more than that in the East Afri­can coun­tries that give us liter­ally 100% of top male marathon­ers). De­spite that, progress in marathon records has been steady or de­clin­ing. Most other Olympics sports show the same pat­tern.

All of these lines of ev­i­dence lead me to the same con­clu­sion: con­stant growth rates in re­sponse to ex­po­nen­tially in­creas­ing in­puts is the null hy­poth­e­sis. If it wasn’t, we should be ex­pect­ing 50% year-on-year GDP growth, eas­ily-dis­cov­ered-im­mor­tal­ity, and the like. No­body ex­pected that be­fore read­ing BJRW, so we shouldn’t be sur­prised when BJRW provide a data-driven model show­ing it isn’t hap­pen­ing. I re­al­ize this in it­self isn’t an ex­pla­na­tion; it doesn’t tell us why re­searchers can’t main­tain a con­stant level of out­put as mea­sured in dis­cov­er­ies. It sounds a lit­tle like “God wouldn’t de­sign the uni­verse that way”, which is a kind of sus­pi­cious line of ar­gu­ment, es­pe­cially for athe­ists. But it at least shifts us from a lens where we view the prob­lem as “What three tweaks should we make to the grad­u­ate ed­u­ca­tion sys­tem to fix this prob­lem right now?” to one where we view it as “Why isn’t Mar­cus Aure­lius im­mor­tal?”

And through such a lens, only the “low-hang­ing fruits” ex­pla­na­tion makes sense. Ex­pla­na­tion 1 – that progress de­pends only on a few ge­niuses – isn’t enough. After all, the Greece-to­day differ­ence is partly based on pop­u­la­tion growth, and pop­u­la­tion growth should have pro­duced pro­por­tionately more ge­niuses. Ex­pla­na­tion 2 – that PhD pro­grams have got­ten worse – isn’t enough. There would have to be a wor­ld­wide mono­tonic de­cline in ev­ery field (in­clud­ing sports and art) from Athens to the pre­sent day. Only Ex­pla­na­tion 3 holds wa­ter.

I brought this up at the con­fer­ence, and some­body rea­son­ably ob­jected – doesn’t that mean sci­ence will stag­nate soon? After all, we can’t keep feed­ing it an ex­po­nen­tially in­creas­ing num­ber of re­searchers for­ever. If noth­ing else stops us, then at some point, 100% (or the high­est plau­si­ble amount) of the hu­man pop­u­la­tion will be re­searchers, we can only in­crease as fast as pop­u­la­tion growth, and then the sci­en­tific en­ter­prise col­lapses.

I an­swered that the Gods Of Straight Lines are more pow­er­ful than the Gods Of The Copy­book Head­ings, so if you try to use com­mon sense on this prob­lem you will fail.

Imag­ine be­ing a fu­tur­ist in 1970 pre­sented with Moore’s Law. You scoff: “If this were to con­tinue only 20 more years, it would mean a mil­lion tran­sis­tors on a sin­gle chip! You would be able to fit an en­tire su­per­com­puter in a shoe­box!” But com­mon sense was wrong and the trendline was right.

“If this were to con­tinue only 40 more years, it would mean ten billion tran­sis­tors per chip! You would need more tran­sis­tors on a sin­gle chip than there are hu­mans in the world! You could have com­put­ers more pow­er­ful than any to­day, that are too small to even see with the naked eye! You would have tran­sis­tors with like a dou­ble-digit num­ber of atoms!” But com­mon sense was wrong and the trendline was right.

Or imag­ine be­ing a fu­tur­ist in an­cient Greece pre­sented with world GDP dou­bling time. Take the trend se­ri­ously, and in two thou­sand years, the fu­ture would be fifty thou­sand times richer. Every man would live bet­ter than the Shah of Per­sia! There would have to be so many peo­ple in the world you would need to tile en­tire coun­tries with cityscape, or build struc­tures higher than the hills just to house all of them. Just to sus­tain it­self, the world would need trans­porta­tion net­works or­ders of mag­ni­tude faster than the fastest horse. But com­mon sense was wrong and the trendline was right.

I’m not say­ing that no trendline has ever changed. Moore’s Law seems to be le­gi­t­i­mately slow­ing down these days. The Dark Ages shifted ev­ery macro­his­tor­i­cal in­di­ca­tor for the worse, and the In­dus­trial Revolu­tion shifted ev­ery macro­his­tor­i­cal in­di­ca­tor for the bet­ter. Any of these sorts of things could hap­pen again, eas­ily. I’m just say­ing that “Oh, that ex­po­nen­tial trend can’t pos­si­bly con­tinue” has a re­ally bad track record. I do not un­der­stand the Gods Of Straight Lines, and hon­estly they creep me out. But I would not want to bet against them.

Grace et al’s sur­vey of AI re­searchers show they pre­dict that AIs will start be­ing able to do sci­ence in about thirty years, and will ex­ceed the pro­duc­tivity of hu­man re­searchers in ev­ery field shortly af­ter­wards. Sud­denly “there aren’t enough hu­mans in the en­tire world to do the amount of re­search nec­es­sary to con­tinue this trend line” stops sound­ing so com­pel­ling.

At the end of the con­fer­ence, the mod­er­a­tor asked how many peo­ple thought that it was pos­si­ble for a con­certed effort by our­selves and our in­sti­tu­tions to “fix” the “prob­lem” in­di­cated by BJRW’s trends. Al­most the en­tire room raised their hands. Every­one there was smarter and more pres­ti­gious than I was (also richer, and in many cases way more at­trac­tive), but with all due re­spect I worry they are in­sane. This is kind of how I imag­ine their wor­ld­view look­ing:

I re­al­ize I’m be­ing fatal­is­tic here. Doesn’t my po­si­tion im­ply that the sci­en­tists at In­tel should give up and let the Gods Of Straight Lines do the work? Or at least that the head of the Na­tional Academy of Sciences should do some­thing like that? That Fran­cis Ba­con was wast­ing his time by in­vent­ing the sci­en­tific method, and Fred Ter­man was wast­ing his time by or­ga­niz­ing Sili­con Valley? Or per­haps that the Gods Of Straight Lines were act­ing through Ba­con and Ter­man, and they had no choice in their ac­tions? How do we know that the Gods aren’t act­ing through our con­fer­ence? Or that our study­ing these things isn’t the only thing that keeps the straight lines go­ing?

I don’t know. I can think of some in­ter­est­ing mod­els – one made up of a thou­sand ran­dom coin flips a year has some nice qual­ities – but I don’t know.

I do know you should be care­ful what you wish for. If you “solved” this “prob­lem” in clas­si­cal Athens, At­tila the Hun would have had nukes. Re­mem­ber Yud­kowsky’s Law of Mad Science: “Every eigh­teen months, the min­i­mum IQ nec­es­sary to de­stroy the world drops by one point.” Do you re­ally want to make that num­ber ten points? A hun­dred? I am kind of okay with the func­tion map­ping num­ber of re­searchers to out­put that we have right now, thank you very much.

The con­fer­ence was or­ga­nized by Pa­trick Col­li­son and Michael Niel­sen; they have writ­ten up some of their thoughts here.