Mohammed Bavarian, a research scientist at OpenAI, tweeted this thread arguing that he could see “the overall US GDP growth rising from recent avg 2-3% to 20+% in 10 years.” Feel free to check out those arguments, though they’ll probably be familiar to you: GPT-3, GitHub Copilot, and image synthesis will drive unprecedented improvements.
Cameron Fen, an economics PhD student at the University of Michigan, responded with this thread disagreeing with Bavarian’s argument. I wanted to share some of the arguments that I found novel and interesting.
Argument #1: The impacts of previous transformative technologies
There have been 3 industrial revolutions in history, mechanization, electricity and mass production, and IT and the internet. China went through all 3 at the same time and was barely able to go above 7% growth annually.
The newest industrialization will be big, but will it be bigger than moving 95% of the population working in agriculture to working factories, not to mention all 3 combined?
In particular, the growth of the first industrial revolution accelerated growth from 1.5% to 3% a year in the UK (Source). Growth from electrification was 1.5% a year on average (source) and growth during the information industrialization was 3.5% a year (source).
Given these growth rates, it seems unlikely that a single industrial revolution can move the needle to such an extent that US GDP growth accelerates from 2% to 7%.
Argument #2: The size of the tech industry
Perhaps you can argue that the industrial revolution on the horizon is going to be 3x bigger than any industrial revolution in the past. Let’s see what that would imply:
7% growth implies 5% growth over our current trend rate of about 2%. This comes out to be about 1.15 trillion GDP additional growth this year and increasing as the base gets larger.
According to this article, Facebook contributes 100 billion to US economic activity. While much of this is uncounted because GDP methodology is imperfect, I’ll go with that number.
To get to the 1.15 trillion additional growth you need 11.5 Facebook created every year. This comes out to a market cap of 5.5 trillion dollars created every year in tech to get something like a 5% growth. The market cap of the entire tech sector is 13.5 trillion dollars.
Call adjacent markets another 6.5 trillion dollars. Thus a 20 trillion dollar market cap sector needs to add 6 trillion (5.5 + .02*20) dollars a year. Has the tech sector ever grown so fast that it is creating the equivalent of 12.5 Facebooks from nothing every year? No.
Let’s think about this another way, assume the US is growing at 2% a year. The tech sector is 10% of the US economy. If the US were to grow 7% a year on all tech sector growth, the tech sector would have to grow at 52% a year.
Even if you assume adjacent sectors (20% share) growing at 12% a year, you need 32% growth from the tech sector to just get 7% total GDP growth. This seems infeasible to me.
Argument #3: Estimating the market impact of LLMs, image synthesis, and AlphaFold
Just to get a sense of how massive innovation has to be to move the needle, I’m going to discuss three central innovations. Let me know if I missed something—more will be invented—but I plan on showing that these game changing techs will not drastically improve growth.
The triumvirate of GDP impactful techs are 1) GPT-3 and other LLMs, 2) Self-driving cars and other robotic control, 3) and Alpha Fold. I don’t include text-to-image models like DALLE and other diffusion models because idk any commercial applications that move the needle.
Market impact of natural language generation: 0.5% of GDP per year because there are free alternatives. (My response: Wouldn’t the free alternatives boost production for users?)
LLM: GPT-3 is good at a lot of NLP tasks, but versus specialists with fewer parameters it often performed worse. For example, BERT outperformed GPT-3 for word embeddings. Something like PaLM maybe outperforms BERT, but the improvement is incremental.
Is there a space for people who are afraid to go to Hugging Face to download BERT and press run on their computer? Yes. If you want Open AI to hold your hand for a fee, there is a place for that, but outside of NLG GPT-3 isn’t really a category killer
NLG: Maybe other applications add additional impact from GPT-3, using the same logic maybe another 5% growth a year from other adjacent sectors (10% of the economy). You have robot therapists, robot financial planners, and robot CRMs. Again, not everyone will choose these options as some will prefer the human touch, you also have the same cost disease problem, and people losing their jobs.
Most interesting argument: Code generation only raises tech sector growth by 3.5% / year because of the Baumol effect.
Codex: First I will start out with Codex, which to me seems like the category killer app of the category killer that is NLG. Assume that coders spend 2⁄3 of their time coding and 1⁄3 of their time doing other stuff. This is optimistic.
Let’s say Codex makes all coding instantaneous. Coders would only accomplish 3x more work. Why? This is an example of Baumol’s Cost disease. Certain things don’t get more productive and as the productive things get more productive, the unproductive things take up more of your time. Coders would now spend 100% of their time on not coding, but that takes 33% of their time before coding.
Let’s say the tech sector is 33% coders, again Baumol’s cost disease hurts us as that would imply that 66% more work is being done in the tech sector due to Codex. This is widely optimistic.
The total number of developers inside and outside tech in the US is 4.1 million people. This includes anyone who codes, not just engineers. Tech employs 12 million people.
If you assume it takes 15 years for Codex to diffuse (the lightbulb was invented in 1879 and it took until 1925 for half the homes in the US to have electricity (source), that comes out to about 3.5% growth a year for the tech sector, from Codex, and by fiat assumption 3.5% growth for coders in the 10% of the economy outside tech that has software engineers.
At the same time, as Codex is improving efficiency, coders are losing their jobs. So there is a net increase in GDP there is also a drag on growth. I don’t know how much this is but someone losing their 100K-200K job will spend a lot less on the economy. I ignore this.
Self-driving cars isn’t a huge industry
Self-Driving Cars: The problem with self-driving cars is that the industry is too small. Automobiles is a 100-billion-dollar industry (source). Since the US economy is 23 trillion dollars in size even if you have a 5x growth in the industry that’s still only a one-time growth of 1.7%. Businesses like Covariant and Boston Dynamics are niche businesses in a niche industry and do not move the needle.
Pharmaceuticals aren’t a huge industry either
Alpha Fold: The story for Alpha Fold is the same as for self-driving cars. The pharmaceutical industry is 625 billion and is 2.7% of US GDP. (source) Even increasing the pharmaceutical industry by 5x would lead to an additional 11% one-time bump in GDP. Smoothed over fifteen years comes out to be less than a 1% increase in growth.
Conclusion: These hypothetical boosts only add up to 2% additional GDP growth from AI
Adding all these things together, 3.5% increase in tech sector (10% of economy) = .35%, 5% increase in adjacent sector = .5%, 3.5% increase in engineers outside of software (10%) = .35%, Self-driving cars 1.7%/ 15 years = .1%, pharma growth is .66%.
Putting all this together you have 2% additional GDP growth from deep tech and related advances, 2% from current growth which leads to 4% GDP growth in this widely optimistic scenario. You still need 3% growth from another tech that I haven’t named or hasn’t been invented yet, but I hope my exercise has illustrated how much inertia must be overcome GDP-wise to get even 3% more GDP growth.
This will be even harder by 2030 when the economy will have increased from 23 trillion to around 30 trillion in size. The amount needed to grow 7% at 30 trillion would be more than 25% higher than what is needed at 23 trillion.
Awhile back johnswentworth wrote What Do GDP Growth Curves Really Mean? noting how real GDP (as we actually calculate it) is a wildly misleading measure of growth because it effectively ignores major technological breakthroughs – quoting the post, real GDP growth mostly tracks production of goods which aren’t revolutionized; goods whose prices drop dramatically are downweighted to near-zero, so that when we see slow, mostly-steady real GDP growth curves, that mostly tells us about the slow and steady increase in production of things which haven’t been revolutionized, and approximately-nothing about the huge revolutions in e.g. electronics. Some takeaways by the author that I think pertain to the “AI and GDP growth acceleration” discussion:
“even after a hypothetical massive jump in AI, real GDP would still look smooth, because it would be calculated based on post-jump prices, and it seems pretty likely that there will be something which isn’t revolutionized by AI… Whatever things don’t get much cheaper are the things which would dominate real GDP curves after a big AI jump”
“More generally, the smoothness of real GDP curves does not actually mean that technology progresses smoothly. It just means that we’re constantly updating the calculations, in hindsight, to focus on whatever goods were not revolutionized”
“using price as a proxy for value is just generally not great for purposes of thinking about long-term growth and technology shifts” (quote)
Ever since I read that post I’ve generally discounted claims that AI progress would accelerate GDP growth to double digits unless the claims address the methodological point about how GDP is calculated. This isn’t an argument against the potential for AI progress to revolutionize human civilization or whatever; it’s (as I interpret it) an argument against using anything remotely resembling GDP in trying to quantify how much AI progress is revolutionizing human civilization, because it just won’t capture it.
Is there any GDP-like measure that does do a better job of capturing growth from major tech breakthroughs?
Total Factor Productivity would fit the bill
Nope, see here
I’m skeptical. Guzey seems to be conflating two separate points in the section you’ve linked:
TFP is not a reliable indicator for measuring growth from the utilization of technological advancement
Bloom et al’s “Are Ideas Getting Harder to Find?” is wrong to use TFP as a measure of research output
The second point is probably true, but not the question we’re seeking to answer. Research output does not automatically translate to growth from technological advancement.
Is it absurd? I’m not so sure. Between ’73 and ’82 the oil shock led to skyrocketing energy prices. Guzey acknowledges this economic crisis but goes on to claim that the indicator must be bad since semiconductors got better, crop yields improved, and life expectancy improved. And he’s right, for Bloom’s paper, this is a major discrepancy. TFP is not a good measure of research output. However, TFP roughly measures an economy’s technological capacity given current restraints.
America in ’73 was more productive than America in ‘82 because a key technological input (energy) was significantly cheaper in ’73 than it would be for most of the following decade while the technological advancements made during the same period were not enough to offset the balance.
Let’s look at the other examples provided:
According to the data provided, France’s TFP peaked prior to the Great Recession and has largely stagnated since. This doesn’t seem surprising given France’s sluggish economic growth since then. French GDP peaked in 2008. Its labor productivity has also barely grown. If one examines the data without holding the bias that tech advancements since 2001 MUST have vastly improved productivity, the results are hardly surprising.
This is harder to explain. According to the data, Italy’s TFP effectively peaked in 1979, remained near this peak until just before the Great Recession, and declined since. Italy’s GDP peaked around the time of the Great Recession and declined since. Nonetheless, its TFP being higher in 1970 than 2019 is shocking. CEPR argues that Italian manufacturing misallocates resources on a massive scale but I’d hesitate to give any firm opinion. Rising energy costs may also play a role? This is worthy of further research, but as Guzey points out, Italy is not on the technological frontier and is a bit of a basket case.
Unsurprising. 2009 was an unusually weak year for TFP in Japan given the Great Recession’s effects. Moreover, since the 1990s, Japan has been in its lost decades. Japan’s TFP growth looks more healthy and similar to America’s compared to France and Italy.
(See Italy)
Skipping Sweden and Switzerland as they are small countries.
The United Kingdom’s TFP peaked in 2007, one year before its GDP peaked. Like France, Italy, and Spain, it has yet to recover from the Great Recession.
TFP is NOT a measure of the pure technological frontier. It cannot tell you how much cutting-edge lab research has progressed over time. What it can tell you is how much technological advancement has soaked into the economy. Recessions, market shocks, structural barriers, and other forms of inertia can slow or even regress TFP.
Guzey goes on to give other takes I find puzzling like the following:
He already has his conclusion and dismisses arguments that reject it. “Of course the internet has provided massive economic value, any metric which fails to observe this must be wrong.” What is the evidence that Google Search provides consumers with $500/month of value? The midcentury appliances revolution alone saved families 20 hours or more of weekly labor. No one argues that the digital revolution hasn’t improved technological productivity, economists cite it as the cause of the brief TFP growth efflorescence from the mid-90s to the early 2000s. But Guzey seems to think its impact is far larger and imagines scenarios to support this claim.
this
I’ve heard economists comment that the IT revolution didn’t have any measurable impact on TFP, and we’re not working many more hours than we did 50 years ago, so how did IT have so much impact on growth?
Regarding the topic in general, it seems like for AI to have a big impact, it will have to break through the cost disease. Taking the coding example, if the AI can take on the roles of not just coding but project management, design, business etc, to create a complete software-solutions-in-a-box platform, then it can result in massive sector growth, cost savings, my unemployment, etc. But if AI only makes things cheaper that were already made massively cheaper by the first three industrial revolutions, then it will just make the cost disease worse.
As this argument implies, getting very high growth rates probably requires getting humans (largely) out of the loop in at least some major fields. Unclear how AGI-complete doing that would be.
I would expect that 20% is basically a slow takeoff AGI scenario.
For what it’s worth, I’m fairly confident self-driving cars will cause a bigger splash than $500 billion. Car accidents in the US alone cost $836 billion. In a world with ubiquitous self-driving cars, not only could this cost be slashed by 80% or more, but reduced parking spots will also allow much more economic activity. Parking spots currently comprise about a third of city land in the US. The total impact could easily be over a trillion for the US alone.
Still not enough to get even close to 20% growth though.
Saving money through having less car accidents doesn’t produce GDP. It might even reduce GDP because people don’t have to buy a new car to replace the crashed car.
Wouldn’t the capital saved on fewer car accidents be free to boost consumption and production? Moreover, most of the $800 billion figure does not entail savings from car repairs/replacements but working hours lost to injuries, traffic jams, medical bills, and QALY lost.
Not a comment on the argumentation or anything, I know we want to be rationalists and worry about the arguments (so thank you for posting about the disagreement that actually offers some analysis), but just registering my initial reaction to the 20%/year in 10 years claim:
Anyone with a cursory understanding of the history of economic growth (I don’t even mean professors who have spent their careers studying growth economics) will know that number is facially ridiculous. My first thought was, great, now I know this person has no idea what they are talking about and can be safely ignored. As a communication device, that prediction failed miserably for me because it did not make me want to assess the underpinnings or energize me to research further about economic growth, which is the least such a tweet should have done if not prompt an update. Though good luck with the latter because, again, if you have an iota of knowledge about the area, your prior distribution for the realm of US economic growth possibilities is correctly more narrow than “let me throw out a shocking round number to myself, see if I can live with it, and then see if others take it seriously.” Am I being too harsh? Well, no, he literally “didn’t put much effort into calibrating the numbers.”
Now back to the regularly scheduled programming of arguing about the merits.
A thousand and one days in the life of a turkey—i mean, human civilization
Arguments that rely on gdp historically being tightly clustered around a small number miss the obvious jump-diffusion nature of the claims being made here. It’s not a draw from a normal distribution. It’s a value you’ve ever seen before because the distribution isn’t what you think.
perhaps it should be rephrased as an argument for a total phase change in economic paradigm, then?
Right, when you go to argue the merits, you ask “well, if there were to be a phase change, what would the phase change look like?” And the original estimate was derived from not much effort in calibrating the numbers, and the reply was that even if we saw an utterly shocking phase change, we’d get nowhere close to 20%. You can do varying degrees of in-depth analyses to get to that point (good on you), or you can do like I did and rely on a semi-informed prior.
Here’s US growth from 1947. Imagine all the things that happened since then that could have induced mild phase changes to the growth trajectory. ASSUMING that there will be a substantial phase change (again, see Cameron Fen’s thread), 20% is still ludicrous.
https://fred.stlouisfed.org/graph/?g=TE7B
You have the profits from the AI tech (+ compute supporting it) vendors and you have the improvements to everyone’s work from the AI. Presumably the improvements are more than the take by the AI sellers (esp. if open source tools are used). So it’s not appropriate to say that a small “sells AI” industry equates to a small impact on GDP.
But yes, obviously GDP growth climbing to 20% annually and staying there even for 5 years is ridiculous unless you’re a takeoff-believer.
If Fen is using the most appropriate basic approach to forecasting growth, then his conclusions are correct.
Fen does not seem to be addressing the kind of models used in this OpenPhil report.
In particular, that report considers substituting capital for labor a potential driver of explosive growth. Fen’s argument from the Baumol effect relies on the premise that there are baseline levels of labor that cannot be automated, and that productivity growth is therefore limited by those bottlenecks.
A very sobering article. The software I use certainly doesn’t get better, and money doesn’t get less elusive. Maybe some unimagined new software could change people’s lives like a mind extension or something.