The key insight is that ordinary human leaking and spying and tech diffusion happens at ordinary human pace. If e.g. China is doing an intelligence explosion such that it’s making tech and economic progress at 100x normal speed, the process of ideas gradually diffusing to the US and causing catch-up growth won’t be 100x faster to compensate. Instead e.g. the lag between “the chinese are selling an exciting new product” and “our engineers have dissected it and deduced some interesting ideas about how it was made and replicated these ideas in our own manufacturing processes” will be… approximately the same? And so for practical purposes it might as well not be happening.
I don’t think it will be the same? If the economy is growing 100X as fast then that means:
physical artefacts transported from place to place 100X faster
new tech and products adopted throughout the economy 100X faster
Both these things help with diffusion between countries as well.
And if designing new tech is 100X faster, i’m not seeing why we would expect copying tech to be <100X faster. With copying, you start with additional clues—e.g. a physical artefact or using a digital product or public materials (like OAI’s o3 demo) -- and design the tech from there. Why would designing new tech from scratch be sped up by more than designing from clues? I.e. we’ve got super-smart AI doing both.
the rate of tech diffusion via leaks and spying will go way down
Yeah, agree spying will go way down once humans are replaced, and cyber hacks will go down too. Though i would have thought a small minority of tech diffusion comes from spies + hacks? E.g. i’d have thought selling and publishing about o1 was more important than those things, though i could be wrong and spies might be unually important for AI diffusion in particular.
...
I don’t think you’ve responded to my argument that companies in the leading country could make way more short-term revenues by producing and selling their technologies in other countries? (See the argument in the box.) Bc ASI cognitive capabilities are massively complementary with human labour and physical capital inputs.
Re: producing and selling tech in other countries: Yes I expect them to do that. I agree with that bit.
I don’t think the economy growing 100x as fast due to an ASI-led industrial explosion means that human firms who get to dissect the products produced will get to produce their own replication competitor products 100x as fast as it usually takes. Like, they are still human! Operating at human speed! I feel like you haven’t really engaged with my arguments.
Sure, some physical objects will be transported 100x faster (though not all?) and some new tech will be diffusing 100x faster e.g. some new products will be selling like hotcakes. But that doesn’t undermine the basic picture I sketched.
Why would designing new tech from scratch be sped up by more than designing from clues? I.e. we’ve got super-smart AI doing both.
OH I get it, you are imagining that you have similarly smart ASIs on both sides of this exchange whereas I was imagining one side is ASI and the other side is humans. I agree that if e.g. Germany, France, Brazil, etc. all had their own ASI labor force, then the process of tech diffusion might well be (almost) 100x sped up when the pace of progress is 100x’d. But that’s not the relevant situation to be imagining; the situation to imagine is one where e.g. China or the US has ASI but no one else does for the next few months (years?) at least. Or, more nuanced, one country is deeper into the intelligence explosion than everyone else, even if everyone else also has quite powerful AIs.
Chinese AI used in Europe. US AI also used in Europe. Deep integration with the physical economies due to the massive complementarities.
So then the Q is why Europe can’t quickly use its AI to copy the new physical tech?
And similarly for China, which can use its worse AI (and the US AI as well if the status quo continues).
Also, even absent copying the new tech, the amazing new physical robots will be made on European soil. (But maybe US companies still capture the profits?)
Chinese AI used in Europe. US AI also used in Europe. Deep integration with the physical economies due to the massive complementarities.
If it’s 6mo behind during the intelligence explosion, Chinese AI will be very uncompetitive. Perhaps its main use case will be trying to reverse engineer US products lol.
So then the Q is why Europe can’t quickly use its AI to copy the new physical tech?
Let’s suppose that the current overall speedup of the US-run intelligence & industrial explosion is “100x normal human speed.” Then (I’ve argued above) unassisted German humans would still be at roughly 1x human speed at the task of reverse-engineering the US-designed products. Which means it takes months if not years, which is too late to matter. If the Chinese AIs are 6months behind, but still superhuman, then the reverse-engineering process goes faster 1x but slower than 100x. Depends on takeoff speeds.
How to estimate this?
Well, in AI 2027, when the algo progress speedup is 100x, a six-month-behind AI project would still not yet be at top human level at research (though they’d be almost at the Superhuman Coder Milestone). Now, speed-at-reverse-engineering stuff is different from this, but maybe this is an OK proxy. Eyeballing it, later on there seems to remain something like a 100x gap between the R&D multiplier at time t and the multiplier at time t-6mo.
So yeah. And remember, after reverse-engineering stuff, you then have to contend with the fact that your stuff is probably already obsolete. Even if it takes you only a month or two to reverse-engineer (consistent with having AIs that are only 3months behind the frontier, at least on AI 2027 takeoff speeds)
Also, even absent copying the new tech, the amazing new physical robots will be made on European soil. (But maybe US companies still capture the profits?)
Yes. Similarly, many amazing new factories and mines and plantations and windmills and railroads were built on African, American, Indian soil by European colonizers. Many of which was peaceful & free trade-y, e.g. “factories.” Doesn’t change the bottom line conclusion. (And also, decolonialism happened, but I don’t think we should expect an analogous thing to happen in a hypothetical outgrow-the-world industrial explosion scenario.)
If there’s a AI-2027 style SIE, then 6 months is a gap of ~6 OOMs of effective compute, which is maybe equivalent to a having 1000X more researchers thinking 4X faster and much smarter. (Maybe 2X the gap as from top expert to median expert.)
That’s a big advantage. Much bigger than US had over China over the last 70 years. So yeah it’s pretty plausible that the reverse engineering would be too slow for Germany to catch up using Chinese AI…
Though once the SIE fizzles out Germany will have access to ~equally good AI. And if the industrial explosion has still got many years to go that could give them time to catch up.
There’s also the question of how Germany is able to use the US AI. For example, suppose two US projects are neck and neck. Or suppose there’s one US project, but multiple downstream ‘product’ companies that can sell fine-tune and sell access to the model. In that case, competition could lead for German companies to be able to access AI for a wide range of tasks. Including ‘R&D’ that really involves copying/adapting US tech developed elsewhere. In this case, Germany doesn’t have to use worse AI.
OTOH, maybe there’s just one US project and it’s single-handedly making tech progress in the US and selling its systems abroad. In this case, it could have strong incentive to prevent German companies using its AI to copy US tech. Bc it developed that tech itself! This is a case where the US is strongly coordinated bc all US economic activity is just one company.
Another scenario. There’s one US AGI project. It sells API access to existing companies in the US, at a very healthy profit, and those companies all make amazing new tech and own the IP. In this case, teh US project still has incentive to sell its AI to Germany companies that will use it to copy the tech developed in US companies. (In fact, there’s no reason the US companies would be ahead of the German companies in developing the tech in this scenario.) Here is seems like USG needs to step in to make sure US companies keep the profits.
Like, here’s one way of putting it. There’s def a chance that one project outgrows the world. Especially if there’s a monopoly on AI within the US with the support of the govt. That project would be internally coordinated and avoid trades that disadvantage it relative to RoW. But suppose one project doesn’t outgrow the world. There’s then a question of why the larger entity that does outgrow the world is the US. Like, why are other US companies in on the action, but no non-US companies. ASML and TSMC will have a lot to offer! My answer to this question is that USG would have to make specific efforts, significantly restricting trade, to make this happen.
Then the scenario where amazing tech is built in Germany by US companies. Let’s put aside manipulation and military exploitation for a second. The GDP produced by these companies is German GDP and the German government can tax it. So, if most physical manufacturing is not on US soil and most GDP is from physical stuff being produced, then >25% GDP will be produced on non-US soil and could be taxed by non-US governments. But yeah, i agree that the analogy of colonization is chilling here.
(sorry, this isnt’ very well organised. But overall my tentative position is that it is plausible US outgrows the world here, but it seems less certain and more complicated than what appears to be your position)
To be clear, I think it’s pretty obvious the US could outgrow the world in the scenario you describe, and am arguing for a stronger claim—that China or even maybe Israel could outgrow the world if takeoff speeds are like they are in AI 2027 and they start with a 6-month lead in the intelligence explosion and there’s no war or trade embargo.
Though once the SIE fizzles out Germany will have access to ~equally good AI. And if the industrial explosion has still got many years to go that could give them time to catch up.
OK, I agree that IF the natural shape of things is for a SIE to fizzle out and then there be a period of many years in which an industrial explosion gradually occurs, then yes, perhaps trailing nations could catch up or almost catch up at least, being substantially worse off in relative terms than they were when they started but not completely obsolete. However I don’t think that’s the natural shape of things. I think the intelligence explosion and industrial explosion will overlap / blur into each other somewhat, and I think the industrial explosion will be fast enough that by the time you’ve overcome your adversary’s six-month lead and managed to get AIs that are almost as superintelligent where it counts, they’ll be deep enough into the industrial explosion that they’ll have more than 50% of the GWP already (in the relevant sense, i.e. weighted towards usefulness-for-future-growth) and can just cut you off and outgrow you. Like, concretely, suppose Israel is 6 months ahead in the intelligence explosion and has 5% of the world’s compute, the US having more than an OOM more. (OK actually that case doensn’t work because I think the US would actually catch up pretty fast with that compute advantage...) So suppose Israel has ASI and the US needs 6 more months to get ASI. Unlikely setup yes but suppose it happens. Then I think Israel will probably have some pretty nifty converted factories producing some pretty awesome cheap effective robot designs PLUS effectively unlimited income from various software and IP services and products (drugs, video games, movies, SAAS, consulting, …) which it will be spending to suck rare minerals and other valuable materials in like a black hole. They’ll be months, not years, away from beginning to tile their desert with fusion reactors and strip mines. Or nanobots, or whatever the new economic engine is. Sure, at some point they need to convert this massive economic advantage into actually conquering the territory of the globe, otherwise the rest of the globe will eventually catch up. (Or they can do the space route as you discuss). But when the US is just trying to convert its factories to build the first robots, Israel will already have super-advanced factories producing high quantities of super-advanced robots, weapons, etc. Not a good time to be the US.
Again referencing my earlier post—a 20-year lead in military tech over the last century or so has arguably corresponded to a pretty decisive military advantage, decisive enough to let a small nation like Israel beat a large nation like the USA. If 20 is too much of a stretch, think 40. A six month lead in the industrial & intelligence explosions, assuming things are generally going 100x faster, should correspond to the military equivalent of a 50 year lead. Just think about what the military of Israel in year 19XX+50 could do to the military of the USA in year 19XX.
Then the scenario where amazing tech is built in Germany by US companies. Let’s put aside manipulation and military exploitation for a second. The GDP produced by these companies is German GDP and the German government can tax it. So, if most physical manufacturing is not on US soil and most GDP is from physical stuff being produced, then >25% GDP will be produced on non-US soil and could be taxed by non-US governments. But yeah, i agree that the analogy of colonization is chilling here.
Yeah, I agree that if the country with the superintelligences allows other countries to tax, they’ll be getting a substantial fraction of the profits. And of course there’ll also be money rolling in from the payments for raw materials and labor too.
The question is what % of the lightcone they can buy with that money. Property rights over distant galaxies will by default be up to the whims of the superintelligent coalition that builds the spacecraft, not the German government with all their $.SoeithertheGermansneedtospendthat$ to get von neumann probes quick (but how? they don’t have the tech yet...) or they need to rely on the morality of the superintelligences to give them some, perhaps in return for $$$ or labor or raw materials.
Great—I agree that if you can get to >50% physical capital (as valued by how useful it is for the new supercharged economy) within 6 months of the SIE fizzling out, then you can outgrow the world in the scenario you describe. Sounds like you’re more bullish on a very fast industrial explosion than I am—I think it’s hard to know whether we’ll get physical capital doubling times of <3 months within a few months of SIE. On longer timelines to an SIE, this seems more plausible as there’s more time for robotics to improve in the meantime.
Another blocker—cooperation from pre-existing companies
I want to mention another blocker to someone outgrowing the world via a very fast industrial explosion. You’ll be able to go faster if you draw heavily on the knowledge, skills and machines of existing companies. So you’ll need their cooperation. But they might not cooperate unless you give them a big fraction of the economic value. Which would prevent you from outgrowing the world.
Of course, you could “go it alone” and leapfrog the companies that won’t do business with you. But then it’s even harder to do a very fast industrial explosion. E.g. take ASML and TSMC. I predict it will be much more efficient for the leader in AI to work with those companies (e.g. selling them API access to ASI, or acquiring them) than to leapfrog. If they try to leapfrog, the laggard can team up with them and recover some of the lost lead.
Concretely, let’s say OpenBrain gets to AGI first. Then its AIs are combined with ASML’s existing knowledge to create EUV++ machines. Are the profits from these machines going to ASML or to OpenBrain? Seems unclear to me. ASML’s bargaining position might be strong—they have a strong monopoly and can wait 6 months and deal with the laggard instead.
So maybe other companies have a lot of economic leverage post-AGI, and they convert that to economic value. And, as discussed, maybe other governments tax the activites happening in their domains.
Will $ be helpful?
But you suggest these “mere $” might not be very useful, if ASI has colonised space and doesn’t care. But $ can be used to buy AI and buy crazy new physical tech. So if other actors have lots of $, they can convert that to ability to colonise the stars themselves.
Q about your Israel scenario?
Are you basically assuming a merge between the lab and govt here? And then an nation-wide effort to utilise Israel’s existing human and physical capital to kick-start the industrial explosion within it own borders?
Asking bc absent a merge i’m still not seeing a reason for all actors outside Israel to be left in the dust. The AI company would trade with some Israeli companies and (many more) non-Israeli ones. In which case Israel per se would only outgrow the world if the AI company outgrows the world by itself.
I agree there’s lots of uncertainty about what the industrial explosion will look like; AI 2027 was my extremely unconfident best guess. I think this cuts both ways though; I think it’s entirely plausible (>5% likely, though less than 50%) that the army of superintelligences in the datacenter could follow the following strategy in less than three months:
(1) Keep doing the intelligence explosion to create vastly superhuman AIs that are basically qualitatively different from human scientists at the task of figuring out how to rapidly design and build nanobots etc. (2) Build nanobots etc. by doing tons of parallel experiments in various biolabs and physics labs and whatnot around the world (maybe thousands of such facilities could be acquired simply by spending OpenBrain’s money?) The theory will have been worked out beforehand by the godlike AI scientists from step 1, and the experiments won’t be fucking around, they’ll be doing the minimum necessary experiments to efficiently cut down the search space and estimate the relevant parameters to fix the most viable designs, and then building those designs.
Sure sounds wild, sounds sci-fi even. But historically things that sound at least this wild and sci-fi sure do seem to happen sometimes e.g. nukes would have sounded this way to people beforehand, and the industrial revolution (“machines that move themselves and make other machines? Enabling peasants to live like kings after only a half-dozen generations of growth?”) probably would have sounded this way to people in 1500 too. e.g. the modern cell phone would have sounded like this to someone in 1925 or even maybe 1975. And importantly, all these past examples of wild things happening happened with mere humans doing the R&D. I think we should firmly reserve >5% credence for “ASIs will basically just be gods as far as we can tell, just like how humans are basically gods from the perspective of dolphins or monkeys.”
Re: Cooperation from pre-existing companies: I don’t think this is going to really change the basic picture. Taking your example with OpenBrain and ASML. OpenBrain has ASI, no one else does. ASML’s internal data is useful for the ASIs, maybe, and ASML’s machinery is useful to the ASIs, probably. But the ASIs can probably find a workaround if ASML is obstinate. For example they can hack ASML and steal the data. Or they can do a hostile takeover maybe, and buy ASML. As for the machines, I bet they could offer giant gobs of money to ASML’s suppliers to supply OpenBrain instead and then OpenBrain could build their own machines in less than six months. ASML will also be a fallible human institution, vulnerable to OpenBrain’s ASI-powered trickery, persuasion, lobbying, and shenanigans (e.g. bribing some of the employees to quit and join OpenBrain, possibly even bribing the BoD)
And ASML is probably one of the best examples you’ve find, of the closest thing to a monopoly against the power of the AI company. Other stuff (robots, physical manufacturing, raw materials) is already less monopolistic and thus really hard to coordinate against OpenBrain.
Re: money:
But you suggest these “mere $” might not be very useful, if ASI has colonised space and doesn’t care. But $ can be used to buy AI and buy crazy new physical tech. So if other actors have lots of $, they can convert that to ability to colonise the stars themselves.
No, the best AIs and physical tech will be kept by OpenBrain in house, and the second-best will be sold by OpenBrain, and the third-best will be way worse (since by hypothesis OpenBrain has ASI and no one else does). You can’t use your $ to buy AI and crazy new physical tech unless OpenBrain lets you, and they won’t let you if they suspect you’ll be able to use it to colonize the stars yourselves, since OpenBrain wants those stars.
Re: Q about Israel: I was imagining close cooperation between the govt and the company, yes. but also, I think it’s not necessary, I think the company itself could outgrow the rest of the world combined potentially. (Through trading with it. The company produces (a) cash cow products and (b) new sci-fi industrial tech that bootstraps towards a self-sustaining robot economy with a short doubling time, and it sells (a) to get cash to buy raw materials and physical labor with which to produce (b). After it’s produced enough (b), it outgrows the rest of the world combined & all the cash becomes basically worthless.)
Agree ASML is one of the trickiest cases for OpenBrain. Though I imagine there are many parts of the semiconductor supply chain with TSMC-level monopolies (which is less than ASML). And i don’t think hacking will work. These companies already protect themselves from this, knowing that it’s a threat today. Data stored on local physical machines that aren’t internet connected, and in ppl’s heads.
And i think you’ll take many months of delay if you go to ASML’s suppliers rather than ASML. ASLM built their factories using many years worth of output from suppliers. Even with massive efficiency gains over ASML (despite not knowing their trade secrets) it will take you months to replicate.
I agree that the more OpenBrain have super-strategy and super-persuasion stuff, the more likely they can capture all the gains from trade. (And military pressure can help here too, like with colonialism.)
Also, if OpenBrain can prevent anyone else developing ASI for years, e.g. by sabotage, i think they have a massive advantage. Then ASML loses their option of just waiting a few months and trading with someone else. I think this is your strongest argument tbh.
Biggest cruxes imo:
Size of SIE
Speed of industrial explosion when the SIE is finishing
Depends on size of SIE and on how hard it is to build fast-replicating robots
How long OpenBrain has a monopoly on ASI (maybe indefinitely via sabotage)
Depends on size of SIE, gap with laggard, whether they dare to sabotage
Whether OpenBrain can take ~all gains from trade with other complementary companies
Depends on AI persuasion/strategy after the SIE, whether OpenBrain can quickly rediscover all their insights with way fewer experiments, and whether OpenBrain has a sustained monopoly on ASI
I’m curious how confident you are a company with a 6 month lead could outgrow the rest of the world by themselves?
Thanks!
I don’t think it will be the same? If the economy is growing 100X as fast then that means:
physical artefacts transported from place to place 100X faster
new tech and products adopted throughout the economy 100X faster
Both these things help with diffusion between countries as well.
And if designing new tech is 100X faster, i’m not seeing why we would expect copying tech to be <100X faster. With copying, you start with additional clues—e.g. a physical artefact or using a digital product or public materials (like OAI’s o3 demo) -- and design the tech from there. Why would designing new tech from scratch be sped up by more than designing from clues? I.e. we’ve got super-smart AI doing both.
Yeah, agree spying will go way down once humans are replaced, and cyber hacks will go down too. Though i would have thought a small minority of tech diffusion comes from spies + hacks? E.g. i’d have thought selling and publishing about o1 was more important than those things, though i could be wrong and spies might be unually important for AI diffusion in particular.
...
I don’t think you’ve responded to my argument that companies in the leading country could make way more short-term revenues by producing and selling their technologies in other countries? (See the argument in the box.) Bc ASI cognitive capabilities are massively complementary with human labour and physical capital inputs.
Re: producing and selling tech in other countries: Yes I expect them to do that. I agree with that bit.
I don’t think the economy growing 100x as fast due to an ASI-led industrial explosion means that human firms who get to dissect the products produced will get to produce their own replication competitor products 100x as fast as it usually takes. Like, they are still human! Operating at human speed! I feel like you haven’t really engaged with my arguments.
Sure, some physical objects will be transported 100x faster (though not all?) and some new tech will be diffusing 100x faster e.g. some new products will be selling like hotcakes. But that doesn’t undermine the basic picture I sketched.
OH I get it, you are imagining that you have similarly smart ASIs on both sides of this exchange whereas I was imagining one side is ASI and the other side is humans. I agree that if e.g. Germany, France, Brazil, etc. all had their own ASI labor force, then the process of tech diffusion might well be (almost) 100x sped up when the pace of progress is 100x’d. But that’s not the relevant situation to be imagining; the situation to imagine is one where e.g. China or the US has ASI but no one else does for the next few months (years?) at least. Or, more nuanced, one country is deeper into the intelligence explosion than everyone else, even if everyone else also has quite powerful AIs.
Ah, cool.
I was roughly imagining:
US gets to ASI first
China 6 months behind
Chinese AI used in Europe. US AI also used in Europe. Deep integration with the physical economies due to the massive complementarities.
So then the Q is why Europe can’t quickly use its AI to copy the new physical tech?
And similarly for China, which can use its worse AI (and the US AI as well if the status quo continues).
Also, even absent copying the new tech, the amazing new physical robots will be made on European soil. (But maybe US companies still capture the profits?)
If it’s 6mo behind during the intelligence explosion, Chinese AI will be very uncompetitive. Perhaps its main use case will be trying to reverse engineer US products lol.
Let’s suppose that the current overall speedup of the US-run intelligence & industrial explosion is “100x normal human speed.” Then (I’ve argued above) unassisted German humans would still be at roughly 1x human speed at the task of reverse-engineering the US-designed products. Which means it takes months if not years, which is too late to matter. If the Chinese AIs are 6months behind, but still superhuman, then the reverse-engineering process goes faster 1x but slower than 100x. Depends on takeoff speeds.
How to estimate this?
Well, in AI 2027, when the algo progress speedup is 100x, a six-month-behind AI project would still not yet be at top human level at research (though they’d be almost at the Superhuman Coder Milestone). Now, speed-at-reverse-engineering stuff is different from this, but maybe this is an OK proxy. Eyeballing it, later on there seems to remain something like a 100x gap between the R&D multiplier at time t and the multiplier at time t-6mo.
So yeah. And remember, after reverse-engineering stuff, you then have to contend with the fact that your stuff is probably already obsolete. Even if it takes you only a month or two to reverse-engineer (consistent with having AIs that are only 3months behind the frontier, at least on AI 2027 takeoff speeds)
Yes. Similarly, many amazing new factories and mines and plantations and windmills and railroads were built on African, American, Indian soil by European colonizers. Many of which was peaceful & free trade-y, e.g. “factories.” Doesn’t change the bottom line conclusion. (And also, decolonialism happened, but I don’t think we should expect an analogous thing to happen in a hypothetical outgrow-the-world industrial explosion scenario.)
Yeah let’s think.
If there’s a AI-2027 style SIE, then 6 months is a gap of ~6 OOMs of effective compute, which is maybe equivalent to a having 1000X more researchers thinking 4X faster and much smarter. (Maybe 2X the gap as from top expert to median expert.)
That’s a big advantage. Much bigger than US had over China over the last 70 years. So yeah it’s pretty plausible that the reverse engineering would be too slow for Germany to catch up using Chinese AI…
Though once the SIE fizzles out Germany will have access to ~equally good AI. And if the industrial explosion has still got many years to go that could give them time to catch up.
There’s also the question of how Germany is able to use the US AI. For example, suppose two US projects are neck and neck. Or suppose there’s one US project, but multiple downstream ‘product’ companies that can sell fine-tune and sell access to the model. In that case, competition could lead for German companies to be able to access AI for a wide range of tasks. Including ‘R&D’ that really involves copying/adapting US tech developed elsewhere. In this case, Germany doesn’t have to use worse AI.
OTOH, maybe there’s just one US project and it’s single-handedly making tech progress in the US and selling its systems abroad. In this case, it could have strong incentive to prevent German companies using its AI to copy US tech. Bc it developed that tech itself! This is a case where the US is strongly coordinated bc all US economic activity is just one company.
Another scenario. There’s one US AGI project. It sells API access to existing companies in the US, at a very healthy profit, and those companies all make amazing new tech and own the IP. In this case, teh US project still has incentive to sell its AI to Germany companies that will use it to copy the tech developed in US companies. (In fact, there’s no reason the US companies would be ahead of the German companies in developing the tech in this scenario.) Here is seems like USG needs to step in to make sure US companies keep the profits.
Like, here’s one way of putting it. There’s def a chance that one project outgrows the world. Especially if there’s a monopoly on AI within the US with the support of the govt. That project would be internally coordinated and avoid trades that disadvantage it relative to RoW. But suppose one project doesn’t outgrow the world. There’s then a question of why the larger entity that does outgrow the world is the US. Like, why are other US companies in on the action, but no non-US companies. ASML and TSMC will have a lot to offer! My answer to this question is that USG would have to make specific efforts, significantly restricting trade, to make this happen.
Then the scenario where amazing tech is built in Germany by US companies. Let’s put aside manipulation and military exploitation for a second. The GDP produced by these companies is German GDP and the German government can tax it. So, if most physical manufacturing is not on US soil and most GDP is from physical stuff being produced, then >25% GDP will be produced on non-US soil and could be taxed by non-US governments. But yeah, i agree that the analogy of colonization is chilling here.
(sorry, this isnt’ very well organised. But overall my tentative position is that it is plausible US outgrows the world here, but it seems less certain and more complicated than what appears to be your position)
<3
To be clear, I think it’s pretty obvious the US could outgrow the world in the scenario you describe, and am arguing for a stronger claim—that China or even maybe Israel could outgrow the world if takeoff speeds are like they are in AI 2027 and they start with a 6-month lead in the intelligence explosion and there’s no war or trade embargo.
OK, I agree that IF the natural shape of things is for a SIE to fizzle out and then there be a period of many years in which an industrial explosion gradually occurs, then yes, perhaps trailing nations could catch up or almost catch up at least, being substantially worse off in relative terms than they were when they started but not completely obsolete. However I don’t think that’s the natural shape of things. I think the intelligence explosion and industrial explosion will overlap / blur into each other somewhat, and I think the industrial explosion will be fast enough that by the time you’ve overcome your adversary’s six-month lead and managed to get AIs that are almost as superintelligent where it counts, they’ll be deep enough into the industrial explosion that they’ll have more than 50% of the GWP already (in the relevant sense, i.e. weighted towards usefulness-for-future-growth) and can just cut you off and outgrow you. Like, concretely, suppose Israel is 6 months ahead in the intelligence explosion and has 5% of the world’s compute, the US having more than an OOM more. (OK actually that case doensn’t work because I think the US would actually catch up pretty fast with that compute advantage...) So suppose Israel has ASI and the US needs 6 more months to get ASI. Unlikely setup yes but suppose it happens. Then I think Israel will probably have some pretty nifty converted factories producing some pretty awesome cheap effective robot designs PLUS effectively unlimited income from various software and IP services and products (drugs, video games, movies, SAAS, consulting, …) which it will be spending to suck rare minerals and other valuable materials in like a black hole. They’ll be months, not years, away from beginning to tile their desert with fusion reactors and strip mines. Or nanobots, or whatever the new economic engine is. Sure, at some point they need to convert this massive economic advantage into actually conquering the territory of the globe, otherwise the rest of the globe will eventually catch up. (Or they can do the space route as you discuss). But when the US is just trying to convert its factories to build the first robots, Israel will already have super-advanced factories producing high quantities of super-advanced robots, weapons, etc. Not a good time to be the US.
Again referencing my earlier post—a 20-year lead in military tech over the last century or so has arguably corresponded to a pretty decisive military advantage, decisive enough to let a small nation like Israel beat a large nation like the USA. If 20 is too much of a stretch, think 40. A six month lead in the industrial & intelligence explosions, assuming things are generally going 100x faster, should correspond to the military equivalent of a 50 year lead. Just think about what the military of Israel in year 19XX+50 could do to the military of the USA in year 19XX.
Yeah, I agree that if the country with the superintelligences allows other countries to tax, they’ll be getting a substantial fraction of the profits. And of course there’ll also be money rolling in from the payments for raw materials and labor too.
The question is what % of the lightcone they can buy with that money. Property rights over distant galaxies will by default be up to the whims of the superintelligent coalition that builds the spacecraft, not the German government with all their $.SoeithertheGermansneedtospendthat$ to get von neumann probes quick (but how? they don’t have the tech yet...) or they need to rely on the morality of the superintelligences to give them some, perhaps in return for $$$ or labor or raw materials.
Great—I agree that if you can get to >50% physical capital (as valued by how useful it is for the new supercharged economy) within 6 months of the SIE fizzling out, then you can outgrow the world in the scenario you describe. Sounds like you’re more bullish on a very fast industrial explosion than I am—I think it’s hard to know whether we’ll get physical capital doubling times of <3 months within a few months of SIE. On longer timelines to an SIE, this seems more plausible as there’s more time for robotics to improve in the meantime.
Another blocker—cooperation from pre-existing companies
I want to mention another blocker to someone outgrowing the world via a very fast industrial explosion. You’ll be able to go faster if you draw heavily on the knowledge, skills and machines of existing companies. So you’ll need their cooperation. But they might not cooperate unless you give them a big fraction of the economic value. Which would prevent you from outgrowing the world.
Of course, you could “go it alone” and leapfrog the companies that won’t do business with you. But then it’s even harder to do a very fast industrial explosion. E.g. take ASML and TSMC. I predict it will be much more efficient for the leader in AI to work with those companies (e.g. selling them API access to ASI, or acquiring them) than to leapfrog. If they try to leapfrog, the laggard can team up with them and recover some of the lost lead.
Concretely, let’s say OpenBrain gets to AGI first. Then its AIs are combined with ASML’s existing knowledge to create EUV++ machines. Are the profits from these machines going to ASML or to OpenBrain? Seems unclear to me. ASML’s bargaining position might be strong—they have a strong monopoly and can wait 6 months and deal with the laggard instead.
So maybe other companies have a lot of economic leverage post-AGI, and they convert that to economic value. And, as discussed, maybe other governments tax the activites happening in their domains.
Will $ be helpful?
But you suggest these “mere $” might not be very useful, if ASI has colonised space and doesn’t care. But $ can be used to buy AI and buy crazy new physical tech. So if other actors have lots of $, they can convert that to ability to colonise the stars themselves.
Q about your Israel scenario?
Are you basically assuming a merge between the lab and govt here? And then an nation-wide effort to utilise Israel’s existing human and physical capital to kick-start the industrial explosion within it own borders?
Asking bc absent a merge i’m still not seeing a reason for all actors outside Israel to be left in the dust. The AI company would trade with some Israeli companies and (many more) non-Israeli ones. In which case Israel per se would only outgrow the world if the AI company outgrows the world by itself.
I agree there’s lots of uncertainty about what the industrial explosion will look like; AI 2027 was my extremely unconfident best guess. I think this cuts both ways though; I think it’s entirely plausible (>5% likely, though less than 50%) that the army of superintelligences in the datacenter could follow the following strategy in less than three months:
(1) Keep doing the intelligence explosion to create vastly superhuman AIs that are basically qualitatively different from human scientists at the task of figuring out how to rapidly design and build nanobots etc.
(2) Build nanobots etc. by doing tons of parallel experiments in various biolabs and physics labs and whatnot around the world (maybe thousands of such facilities could be acquired simply by spending OpenBrain’s money?) The theory will have been worked out beforehand by the godlike AI scientists from step 1, and the experiments won’t be fucking around, they’ll be doing the minimum necessary experiments to efficiently cut down the search space and estimate the relevant parameters to fix the most viable designs, and then building those designs.
Sure sounds wild, sounds sci-fi even. But historically things that sound at least this wild and sci-fi sure do seem to happen sometimes e.g. nukes would have sounded this way to people beforehand, and the industrial revolution (“machines that move themselves and make other machines? Enabling peasants to live like kings after only a half-dozen generations of growth?”) probably would have sounded this way to people in 1500 too. e.g. the modern cell phone would have sounded like this to someone in 1925 or even maybe 1975. And importantly, all these past examples of wild things happening happened with mere humans doing the R&D. I think we should firmly reserve >5% credence for “ASIs will basically just be gods as far as we can tell, just like how humans are basically gods from the perspective of dolphins or monkeys.”
Re: Cooperation from pre-existing companies: I don’t think this is going to really change the basic picture. Taking your example with OpenBrain and ASML. OpenBrain has ASI, no one else does. ASML’s internal data is useful for the ASIs, maybe, and ASML’s machinery is useful to the ASIs, probably. But the ASIs can probably find a workaround if ASML is obstinate. For example they can hack ASML and steal the data. Or they can do a hostile takeover maybe, and buy ASML. As for the machines, I bet they could offer giant gobs of money to ASML’s suppliers to supply OpenBrain instead and then OpenBrain could build their own machines in less than six months. ASML will also be a fallible human institution, vulnerable to OpenBrain’s ASI-powered trickery, persuasion, lobbying, and shenanigans (e.g. bribing some of the employees to quit and join OpenBrain, possibly even bribing the BoD)
And ASML is probably one of the best examples you’ve find, of the closest thing to a monopoly against the power of the AI company. Other stuff (robots, physical manufacturing, raw materials) is already less monopolistic and thus really hard to coordinate against OpenBrain.
Re: money:
No, the best AIs and physical tech will be kept by OpenBrain in house, and the second-best will be sold by OpenBrain, and the third-best will be way worse (since by hypothesis OpenBrain has ASI and no one else does). You can’t use your $ to buy AI and crazy new physical tech unless OpenBrain lets you, and they won’t let you if they suspect you’ll be able to use it to colonize the stars yourselves, since OpenBrain wants those stars.
Re: Q about Israel: I was imagining close cooperation between the govt and the company, yes. but also, I think it’s not necessary, I think the company itself could outgrow the rest of the world combined potentially. (Through trading with it. The company produces (a) cash cow products and (b) new sci-fi industrial tech that bootstraps towards a self-sustaining robot economy with a short doubling time, and it sells (a) to get cash to buy raw materials and physical labor with which to produce (b). After it’s produced enough (b), it outgrows the rest of the world combined & all the cash becomes basically worthless.)
Thanks for this!
Yep i agree that scenario gets >5%!
Agree ASML is one of the trickiest cases for OpenBrain. Though I imagine there are many parts of the semiconductor supply chain with TSMC-level monopolies (which is less than ASML). And i don’t think hacking will work. These companies already protect themselves from this, knowing that it’s a threat today. Data stored on local physical machines that aren’t internet connected, and in ppl’s heads.
And i think you’ll take many months of delay if you go to ASML’s suppliers rather than ASML. ASLM built their factories using many years worth of output from suppliers. Even with massive efficiency gains over ASML (despite not knowing their trade secrets) it will take you months to replicate.
I agree that the more OpenBrain have super-strategy and super-persuasion stuff, the more likely they can capture all the gains from trade. (And military pressure can help here too, like with colonialism.)
Also, if OpenBrain can prevent anyone else developing ASI for years, e.g. by sabotage, i think they have a massive advantage. Then ASML loses their option of just waiting a few months and trading with someone else. I think this is your strongest argument tbh.
Biggest cruxes imo:
Size of SIE
Speed of industrial explosion when the SIE is finishing
Depends on size of SIE and on how hard it is to build fast-replicating robots
How long OpenBrain has a monopoly on ASI (maybe indefinitely via sabotage)
Depends on size of SIE, gap with laggard, whether they dare to sabotage
Whether OpenBrain can take ~all gains from trade with other complementary companies
Depends on AI persuasion/strategy after the SIE, whether OpenBrain can quickly rediscover all their insights with way fewer experiments, and whether OpenBrain has a sustained monopoly on ASI
I’m curious how confident you are a company with a 6 month lead could outgrow the rest of the world by themselves?