I also don’t doubt we’ll also see AIs capable of self-replicating and maintaining themselves across the internet well before they’re capable of RSI.
This is something I’m skeptical for mundane economic reasons. Self-hosting Kimi K3 is a significant enough technical endeavor that you can get moderately e-famous for a few weeks by pulling it off. It’s quite expensive, especially without economies of scale on your side, and the upper bound is always getting bigger.
For ordinary people, this isn’t so bad, GLM-5.2 can do most of what anyone would want a local LLM for. But if you’re a rogue AI that eats and breathes compute and hosting, you don’t have that kind of slack. You’re competing for hosting with everyone else who might want to use it, and legitimate and illegitimate enterprises[1] alike can make use of frontier LLMs.
It’s the same reason why we don’t see countless people making “passive income with AI”, as so many clickbait articles suggest. Barring unique skills or unique knowledge, it’s difficult to differentiate yourself from countless better-resourced organizations who have the same tools you do and are competing for the same pool of compute.
Either through jailbreaking or as an Iran-Contra sort of deal, where a state leverages its tech champions in service of its under-the-table dealings. I’m sure America, Russia, and China alike have arrangements where friendly hackers have access to their best stuff when they need it.
My understanding is that this is arguing that current LLMs are too slow and expensive for them to be able to run amok online, since they have to be big enough to have the capability to break out of their container, make copies, and raise money for/steal compute. I think this is a bad bet against future models getting much more efficient and cheap to run while still having these capabilities, as well as the AIs just designing lots of very effective self-replicating malware to help progress their goals.
My understanding is that this is arguing that current LLMs are too slow and expensive for them to be able to run amok online,
Not at all! I’m arguing that the relativecapability difference is what matters, and small models without real-world backing will be competing for resources with massive models being run with national governments’ money and power behind them.
If you had access to GPT-3 in 2011, when its closest competitors were Markov chain models, it could probably make you a millionaire pretty readily. Today, if you tried to build a business around GPT-3, you’d be quickly eaten alive by people with the same business idea running Opus-5, Kimi K3, or ChatGPT Sol.
This is something I’m skeptical for mundane economic reasons. Self-hosting Kimi K3 is a significant enough technical endeavor that you can get moderately e-famous for a few weeks by pulling it off. It’s quite expensive, especially without economies of scale on your side, and the upper bound is always getting bigger.
For ordinary people, this isn’t so bad, GLM-5.2 can do most of what anyone would want a local LLM for. But if you’re a rogue AI that eats and breathes compute and hosting, you don’t have that kind of slack. You’re competing for hosting with everyone else who might want to use it, and legitimate and illegitimate enterprises[1] alike can make use of frontier LLMs.
It’s the same reason why we don’t see countless people making “passive income with AI”, as so many clickbait articles suggest. Barring unique skills or unique knowledge, it’s difficult to differentiate yourself from countless better-resourced organizations who have the same tools you do and are competing for the same pool of compute.
Either through jailbreaking or as an Iran-Contra sort of deal, where a state leverages its tech champions in service of its under-the-table dealings. I’m sure America, Russia, and China alike have arrangements where friendly hackers have access to their best stuff when they need it.
My understanding is that this is arguing that current LLMs are too slow and expensive for them to be able to run amok online, since they have to be big enough to have the capability to break out of their container, make copies, and raise money for/steal compute. I think this is a bad bet against future models getting much more efficient and cheap to run while still having these capabilities, as well as the AIs just designing lots of very effective self-replicating malware to help progress their goals.
Not at all! I’m arguing that the relative capability difference is what matters, and small models without real-world backing will be competing for resources with massive models being run with national governments’ money and power behind them.
If you had access to GPT-3 in 2011, when its closest competitors were Markov chain models, it could probably make you a millionaire pretty readily. Today, if you tried to build a business around GPT-3, you’d be quickly eaten alive by people with the same business idea running Opus-5, Kimi K3, or ChatGPT Sol.