I understand your argument that there’s a systematic bias from tracking progress on relatively narrow metrics. If progress is uneven across different areas at different times, then the areas that saw progress in the recent past may not be the same areas in which we see progress today.
You don’t seem to make any suggestions on what would be a better metric to use. But to me it seems like the simplest solution is just to use broader metrics. For example, instead of tracking the cost of installing solar panels, we could measure the total cost of our electric grid (perhaps including environmental concerns such as carbon emissions as one part of that cost).
Along those lines, the broadest metrics we have are macroeconomic statistics such as GDP per capita. The arguments I’ve seen for stagnation (mostly from Jason Crawford or Tyler Cowen) already use the recent observed slowdown in GDP growth extensively.
If we see the same trend across most areas and most levels of metrics (both narrow, specific use cases and overall summary statistics) - isn’t that strong evidence in favor of the stagnation hypothesis?
Or do you think there are no reliable metrics for measuring progress as a whole?
Basically agree with this suggestion: broader metrics are more likely to be unbiased over time. Even the electric grid example, though, isn’t ideal because we can imagine a future point where going from $0.0001 to $0.000000001 per kilowatt-hour, for example, just isn’t relevant.
Total factor productivity and GDP per capita are even better, agreed.
While a cop-out, my best guess is that a mixture of qualitative historical assessments (for example, asking historians, entrepreneurs, and scientists to rank decades by degree of progress) and using a variety of direct and indirect objective metrics (ex. patent rates, total factor productivity, cost of energy, life expectancy) is the best option. Any single or small group of metrics seems bound to be biased in one way or another. Unfortunately, it’s hard to figure out how to weight and compare all of these things.
While a cop-out, my best guess is that a mixture of qualitative historical assessments (for example, asking historians, entrepreneurs, and scientists to rank decades by degree of progress) and using a variety of direct and indirect objective metrics (ex. patent rates, total factor productivity, cost of energy, life expectancy) is the best option.
Patent rates aren’t an objective measure of innovation. Cutting down the number of trival patents might very well mean increased and not decreased innovation.
I meant objective in the sense that the metric itself is objective, not that it is necessarily a good indicator of innovation. Yes, you’re right. I do like Cowen and Southewood’s method of only looking at patents registered all of the U.S., Japan, and E.U.
The subjects making the judgment seem here to be burocrats in the patent office. I don’t see how that’s substantially more objective then historians making judgments.
I do think that standards of what is a trivial invention change over time. There are court cases that invalidate certain patents and then patent officers change their patent giving to not give out the kind of patents that are likely to be declared invalid. Laws also change.
I understand your argument that there’s a systematic bias from tracking progress on relatively narrow metrics. If progress is uneven across different areas at different times, then the areas that saw progress in the recent past may not be the same areas in which we see progress today.
You don’t seem to make any suggestions on what would be a better metric to use. But to me it seems like the simplest solution is just to use broader metrics. For example, instead of tracking the cost of installing solar panels, we could measure the total cost of our electric grid (perhaps including environmental concerns such as carbon emissions as one part of that cost).
Along those lines, the broadest metrics we have are macroeconomic statistics such as GDP per capita. The arguments I’ve seen for stagnation (mostly from Jason Crawford or Tyler Cowen) already use the recent observed slowdown in GDP growth extensively.
If we see the same trend across most areas and most levels of metrics (both narrow, specific use cases and overall summary statistics) - isn’t that strong evidence in favor of the stagnation hypothesis?
Or do you think there are no reliable metrics for measuring progress as a whole?
Basically agree with this suggestion: broader metrics are more likely to be unbiased over time. Even the electric grid example, though, isn’t ideal because we can imagine a future point where going from $0.0001 to $0.000000001 per kilowatt-hour, for example, just isn’t relevant.
Total factor productivity and GDP per capita are even better, agreed.
While a cop-out, my best guess is that a mixture of qualitative historical assessments (for example, asking historians, entrepreneurs, and scientists to rank decades by degree of progress) and using a variety of direct and indirect objective metrics (ex. patent rates, total factor productivity, cost of energy, life expectancy) is the best option. Any single or small group of metrics seems bound to be biased in one way or another. Unfortunately, it’s hard to figure out how to weight and compare all of these things.
Patent rates aren’t an objective measure of innovation. Cutting down the number of trival patents might very well mean increased and not decreased innovation.
I meant objective in the sense that the metric itself is objective, not that it is necessarily a good indicator of innovation. Yes, you’re right. I do like Cowen and Southewood’s method of only looking at patents registered all of the U.S., Japan, and E.U.
The subjects making the judgment seem here to be burocrats in the patent office. I don’t see how that’s substantially more objective then historians making judgments.
Fair point, but you’d have to think that the tendencies of the patent officers changed over time in order to foreclose that as a good metric.
I do think that standards of what is a trivial invention change over time. There are court cases that invalidate certain patents and then patent officers change their patent giving to not give out the kind of patents that are likely to be declared invalid. Laws also change.