This is nicely put together as usual! Sadly still leaves me feeling uncomfortable. Trying to put my finger on why, and I think it’s at least two things.
Mainly: lumping so many things together as a single scalar ‘software’ just smells really off to me! Perhaps I’m ‘too close’ to the problem, having got deep on the nitty gritty of pretty well all aspects of software here at one time or another. You definitely flag that admirably, and have some discussion on how to make those things sort of commensurable. I do think it’s important to at least distinguish efficiency from ‘quality’, and perhaps to go further (e.g. distinguishing training from runtime efficiencies, or even speed from parallel efficiencies).
Related to this (and I’m less sure, having not deeply interrogated the details), I feel like some double counting of factors is going into the estimates of parameters, especially r. But I can imagine retracting this on further scrutiny.
This is nicely put together as usual! Sadly still leaves me feeling uncomfortable. Trying to put my finger on why, and I think it’s at least two things.
Mainly: lumping so many things together as a single scalar ‘software’ just smells really off to me! Perhaps I’m ‘too close’ to the problem, having got deep on the nitty gritty of pretty well all aspects of software here at one time or another. You definitely flag that admirably, and have some discussion on how to make those things sort of commensurable. I do think it’s important to at least distinguish efficiency from ‘quality’, and perhaps to go further (e.g. distinguishing training from runtime efficiencies, or even speed from parallel efficiencies).
I also think in treatment of R&D it’s important to distinguish steady/stock ‘quality’ from learning/accrual ‘quality’, and to acknowledge that all of these things deprecate as you move through scale regimes: today’s insights may or may not stand up to a 10x or a 100x of your system parameters. This makes sample efficient generalisation and exploratory heuristics+planning really key.
Related to this (and I’m less sure, having not deeply interrogated the details), I feel like some double counting of factors is going into the estimates of parameters, especially r. But I can imagine retracting this on further scrutiny.