Presumably these innovations were immediately profitable. I’m not sure that moves towards architectures closer to AGI (as opposed to myopic/greedy-search moves towards incrementally-more-capable models) are immediately profitable. It’d be increasingly more true as we inch closer to AGI, but it definitely wasn’t true back in the 2010s, and it may not yet be true now.
So I’m sure some of them would intend to try innovations that’d inch closer to AGI, but I expect them not to be differentially more rewarded by the market. Meaning that, unless one of these AGI-focused entrepreneurs is also really good at selling their pitch to investors (or has the right friend, or enough money and competence-recognition ability to get a co-founder skilled at making such pitches), then they’d be about as well-positioned to rush to AGI as some of the minor AI labs today are. Which is to say, not all that well-positioned at all.
you are seriously overestimating how hard it is to get funding
You may not be taking into account the market situation immediately after major AI labs’ hypothetical implosion. It’d be flooded with newly-unemployed ML researchers trying to found new AI startups or something; the demand on that might well end up saturated (especially if major labs’ shutdown cools the hype down somewhat). And then it’s the question of which ideas are differentially more likely to get funded; and, as per above, I’m not sure it’s the AGI-focused ones.
Presumably these innovations were immediately profitable.
That’s not always the case. It can take time to scale up an innovation, but I’d assume it’s plausibly profitable. AGI is no longer a secret belief and several venture capitalists + rich people believe in it. These people also under stand long term profit horizons. Uber took over 10 years to become profitable. Many startups haven’t been profitable yet.
Also a major lab shutting down for safety reasons is like broadcasting to all world governments that AGI exists and is powerful/dangerous.
Presumably these innovations were immediately profitable. I’m not sure that moves towards architectures closer to AGI (as opposed to myopic/greedy-search moves towards incrementally-more-capable models) are immediately profitable. It’d be increasingly more true as we inch closer to AGI, but it definitely wasn’t true back in the 2010s, and it may not yet be true now.
So I’m sure some of them would intend to try innovations that’d inch closer to AGI, but I expect them not to be differentially more rewarded by the market. Meaning that, unless one of these AGI-focused entrepreneurs is also really good at selling their pitch to investors (or has the right friend, or enough money and competence-recognition ability to get a co-founder skilled at making such pitches), then they’d be about as well-positioned to rush to AGI as some of the minor AI labs today are. Which is to say, not all that well-positioned at all.
You may not be taking into account the market situation immediately after major AI labs’ hypothetical implosion. It’d be flooded with newly-unemployed ML researchers trying to found new AI startups or something; the demand on that might well end up saturated (especially if major labs’ shutdown cools the hype down somewhat). And then it’s the question of which ideas are differentially more likely to get funded; and, as per above, I’m not sure it’s the AGI-focused ones.
That’s not always the case. It can take time to scale up an innovation, but I’d assume it’s plausibly profitable. AGI is no longer a secret belief and several venture capitalists + rich people believe in it. These people also under stand long term profit horizons. Uber took over 10 years to become profitable. Many startups haven’t been profitable yet.
Also a major lab shutting down for safety reasons is like broadcasting to all world governments that AGI exists and is powerful/dangerous.