[Question] Source code size vs learned model size in ML and in humans?

To build in­tu­ition about con­tent vs ar­chi­tec­ture in AI (which comes up a lot in dis­cus­sions about AI take­off that in­volve Robin Han­son), I’ve been won­der­ing about con­tent vs ar­chi­tec­ture size (where size is mea­sured in num­ber of bits).

Here’s how I’m op­er­a­tional­iz­ing con­tent and ar­chi­tec­ture size for ML sys­tems:

  • con­tent size: The num­ber of bits re­quired to store the learned model of the ML sys­tem (e.g. all the float­ing point num­bers in a neu­ral net­work).

  • ar­chi­tec­ture size: The num­ber of bits of source code. I’m not sure if it makes sense to in­clude the source code of sup­port­ing soft­ware (e.g. stan­dard ma­chine learn­ing libraries).

I tried look­ing at the AlphaGo pa­per to see if I could find this kind of in­for­ma­tion, but af­ter try­ing for about 30 min­utes was un­able to find what I wanted. I can’t tell if this is be­cause I’m not ac­quainted enough with the ML field to lo­cate this in­for­ma­tion or if that in­for­ma­tion just isn’t in the pa­per.

Is this in­for­ma­tion eas­ily available for var­i­ous ML sys­tems? What is the fastest way to gather this in­for­ma­tion?

I’m also won­der­ing about this same con­tent vs ar­chi­tec­ture size split in hu­mans. For hu­mans one way I’m think­ing of it is as “amount of in­for­ma­tion en­coded in in­her­i­tance mechanisms” vs “amount of in­for­ma­tion en­coded in a typ­i­cal adult hu­man brain”. I know that Eliezer Yud­kowsky has cited 750 megabytes as the amount of in­for­ma­tion in the hu­man DNA, and also em­pha­sizes that most of this in­for­ma­tion is junk. This was in 2011 and I don’t know if there’s a new con­sen­sus or how to fac­tor in epi­ge­netic in­for­ma­tion. There is also con­tent stored in genes, and I’m not sure how to sep­a­rate out the con­tent and ar­chi­tec­ture in genes.

I’m pretty un­cer­tain about whether this is even a good way to think about this topic, so I would also ap­pre­ci­ate any feed­back on this ques­tion it­self. For ex­am­ple, if this isn’t an in­ter­est­ing ques­tion to ask, I would like to know why.