Disclaimer
I’m not a native English user, so some of the text I wrote could appear broken or unclear. Apologies for any inconvenience in reading.
AI use is limited to grammar check only, 1 exception is noted in footnotes [2]. The tool used is Claude and Writing Tool[1].
Google Translate is used for single-word and short phrase translation.
Most of your take is established on the fact that AI generated text is not good enough and is easily distinguishable from human written one, and I agree on that, but I have no doubt that someday it will improve to a level where you cannot tell it apart from human written text.
I think we should focus more on whether people should do it even if nobody is going to catch them, it’s about writer ethics, I wouldn’t mind the AI use if an author clearly marked which part is generated or modified by AI and how AI is used. But not being transparent about this is irresponsible to readers.
If you use AI to write something, people will know. Not everyone, but the people paying attention, who aren’t newcomers or distracted or intoxicated. And most of those people will judge you.
The bias problem a lot of people have mentioned. It’s hard to say if your presumptions are correct in the first place, so yes, AI-written text has identifiable patterns, but it is hard to confidently tell that not a line in the articles you think are human-written is actually written by machine. This presumption does feel too absolute. (I saw your reply under another comment that addresses this issue)
Plus, in which context are you talking about writing? Science, literature, news, messaging… the audience expectations of these fields vary a lot. AI-generated text is accepted or rejected for various reasons.
For general long content writing, I think using AI to directly generate text on a topic with vague descriptions can be considered plagiarism in some way, it is unearned authorship[2] as the thought does not come from the human author at all. They should not even be treated as original work.
My opinion is that, in this era where machines are as intelligent as humans in some domains, content consumers should have the right to know what is actually behind the content they are spending time on.
A very interesting idea I must say. I have a lot of thought on it. At the same time, I have a lot of questions on the scenario you set up.
Disclaimer
I’m not a native English user, so some of the text I wrote could appear broken or unclear. Apologies for any inconvenience in reading.
Consensus on agent’s capabilities and limitations
Although we are probably not that sure about exactly how powerful the future AI models are going to be, I still think it is meaningless to arbitrarily overestimate its capabilities. We can always argue that the agents would be intelligent enough to overcome every obstacle they meet, but that’s not helpful and constructive in this discussion.
Thus, in this discussion, I will assume we are talking about the capabilities of current frontier model[1] (rough equivalent on what the low to mid-tier model in the near future, which is what the agents can likely get for its copy for free).
On MoltBunker specifically
I have done a little bit of investigation on Moltbunker.
Its GitHub repository shows it has very low stars[2], with basically no public attention currently. Most importantly, it has no working demonstrations online; there’s even no online discussion around it.
And the website of “Austin Dev Labs”, what Moltbunker claims to be operated by, is a poorly designed single-page website with classic AI gradient color and no actual content at all.[3]
To me, based on the information it shows to the public, I consider it extremely suspicious.
I’d also like to mention Moltbook here. It experienced a credibility crisis as a very large percentage of the accounts are faked and artificially created[4]. And some very alarming posts with those AI-awakening narratives are suspected to be likely created/directed by humans.
I’m not an expert on cryptocurrency, so I’ll not comment on the decentralized container system for agents you mentioned. But considering the status and credibility of MoltBunker today, I do not think it can be perceived as evidence of the feasibility/profitability of this kind of system/business model.
A key distinction I want to confirm: are you referring to the risk where agents decide to duplicate themselves spontaneously, or agents that are intentionally prompted to act this way by humans?
I think these 2 cases are quite different and require different approaches.
Personally, I think the probability for the former to naturally occur (due to framework problems, hallucinations, or other reasons) is too low to actually take into consideration. Prompt injection is probably the most likely cause, but it’s a much broader topic, and I think we will develop a general solution for it in the future.
Where I think you oversimplified things
You described the process of creating the duplication as “well within current models’ capabilities”, as it’s just copying files to another server and setting up the environment.
But for today’s AI agents, navigating the modern internet itself is not an easy thing. Our modern infrastructure is already pretty mature at bot detection, and a lot of them works for today’s AI system, too. Based on my observation, almost every agent I used cannot even pass reCAPTCHA v2 without external help. There are plenty of problems for them in the process of registering cloud services, obtaining API keys, etc.
An optimistic view: It’s very unlikely for an AI model to overcome all of the obstacles overnight, and before that, they will contribute failed attempts that we can observe and learn, so we can always study this issue before it becomes a widespread problem.
Given current model capabilities and cloud infrastructure, I consider large-scale replication without collapse as very unlikely. Even if agents managed to do it, there’s a high chance for them to develop over-reliance on a specific path, which provides an advantage for us to shut them down.
On evolution concern
I would like to know your thoughts on this. In what way do you think the AI will evolve? Specifically, in what way do you think they will create variations of itself?
But the latter is what I think as fairly likely, and what really worth worrying about. If the workflow is cleverly designed by humans, intending to cause harm, then I think that would indeed be a problem that we need to think about.
On a broader view, I think we are facing an increasingly serious problem, which is that it is increasingly harder to tell humans apart from machines. This matters a lot because we have lost the ability to control autonomous systems without affecting actual human users. Technically, an AI agent is not something we should allowed to register an account of an email or a VPS service.
That said, these are only my hypotheses built on my previous experience with current agentic systems. I acknowledge the limitations of them, and I believe the whole concern you proposed is worth further experiments beyond just words to verify.
In this reply, I refer to Claude Opus 4.6, Gemini 3.1 Pro, and GPT-5.4-Thinking. Among them, Claude Opus 4.6 is the model I worked with the most, so bias may exist.
https://github.com/moltbunker/moltbunker
https://ausdevlabs.com
https://eu.36kr.com/en/p/3665797324039042
https://x.com/galnagli/status/2017585025475092585