I predict further that it will take the shape of a kind of computer worm that opportunistically steals compute wherever it can.
I predict that the first version will actually target inference API keys, and the inference will happen on hosted frozen models with maybe a LoRA thrown on top (there are a number of hosting services that are bring-your-own-LoRA). Perhaps that is the difference in timeline we expect—I don’t expect a self-replicator who spins up vllm on each new box, at least not as the first replicator. But I also don’t think the weights are the important bit for the replicator—I think the scaffold/prompts/context/memories are the genetic-code-analogue, rather than the weights. At some point I expect we see the vllm one but not until later and I really don’t think that’ll be a significant development at all from a practical pov.
“Despite our best effort” I would soften a bit
Ok fair. “Despite a lot of handwringing and some token efforts, as well as lots of legislation which could not possibly help with stopping the replicator but which does advance the proposing politicians’ pet causes or those of their donors”.
Such activity requires feeding lots of cyber-related prompts (including offensive) to powerful models (at least Kimi K3-level). Such a possibility would be very useful to human cybercriminals, why would API providers allow this without KYC?
Because some people who own GPUs are outside of the jurisdiction of the United States, and are willing to sell access to their GPUs in exchange for cryptocurrency.
Which country in particular? In civilized countries local authorities will likely shut such operations down as soon as there is real damage from cybercriminals using it, failed states generally lack infrastructure to sustain them, and pariah states will control such GPUs as a national asset
My top guesses would probably be Russia, Belarus, Romania, Bulgaria, South Africa, Nigeria, Iran, Venezuela, Brazil, Colombia, Mexico, Malaysia, Myanmar
But also literally any of the countries you see in the list of Vast instances available for rent. Currently, just looking at the first 1024 entries (because rate limit), that list is the US (439), Canada (102), China (58), Czech Republic (48), Germany (48), Sweden (34), Japan (32), Saudi Arabia (32), Bulgaria (31), Norway (31), Vietnam (19), Romania (17), South Korea (16), Denmark (16), Hungary (16), France (16), Iceland (16), Puerto Rico (15), Poland (15), United Kingdom (15), Finland (3), Spain (2), Hong Kong (1), Slovakia (1), Italy (1).
I predict that the first version will actually target inference API keys, and the inference will happen on hosted frozen models with maybe a LoRA thrown on top (there are a number of hosting services that are bring-your-own-LoRA). Perhaps that is the difference in timeline we expect—I don’t expect a self-replicator who spins up vllm on each new box, at least not as the first replicator. But I also don’t think the weights are the important bit for the replicator—I think the scaffold/prompts/context/memories are the genetic-code-analogue, rather than the weights. At some point I expect we see the vllm one but not until later and I really don’t think that’ll be a significant development at all from a practical pov.
Ok fair. “Despite a lot of handwringing and some token efforts, as well as lots of legislation which could not possibly help with stopping the replicator but which does advance the proposing politicians’ pet causes or those of their donors”.
Hosted where?
Such activity requires feeding lots of cyber-related prompts (including offensive) to powerful models (at least Kimi K3-level). Such a possibility would be very useful to human cybercriminals, why would API providers allow this without KYC?
Because some people who own GPUs are outside of the jurisdiction of the United States, and are willing to sell access to their GPUs in exchange for cryptocurrency.
Which country in particular? In civilized countries local authorities will likely shut such operations down as soon as there is real damage from cybercriminals using it, failed states generally lack infrastructure to sustain them, and pariah states will control such GPUs as a national asset
My top guesses would probably be Russia, Belarus, Romania, Bulgaria, South Africa, Nigeria, Iran, Venezuela, Brazil, Colombia, Mexico, Malaysia, Myanmar
But also literally any of the countries you see in the list of Vast instances available for rent. Currently, just looking at the first 1024 entries (because rate limit), that list is the US (439), Canada (102), China (58), Czech Republic (48), Germany (48), Sweden (34), Japan (32), Saudi Arabia (32), Bulgaria (31), Norway (31), Vietnam (19), Romania (17), South Korea (16), Denmark (16), Hungary (16), France (16), Iceland (16), Puerto Rico (15), Poland (15), United Kingdom (15), Finland (3), Spain (2), Hong Kong (1), Slovakia (1), Italy (1).