I recently learned from Simon Willison’s Feb 7th blog about StrongDM, who are implementing a Dark Factory level of AI adoption where nobody even looks at the agent-written code, and for security software to boot (I’d be keen to get @lc’s take on what they’re doing). StrongDM’s public description of what they’re doing is here. Quote:
In previous regimes, a team might rely on integration tests, regression tests, UI automation to answer “is it working?”
We noticed two limitations of previously reliable techniques:
Tests are too rigid—we were coding with agents, but we’re also building with LLMs and agent loops as design primitives; evaluating success often required LLM-as-judge
Tests can be reward hacked—we needed validation that was less vulnerable to the model cheating
The Digital Twin Universe is our answer: behavioral clones of the third-party services our software depends on. We built twins of Okta, Jira, Slack, Google Docs, Google Drive, and Google Sheets, replicating their APIs, edge cases, and observable behaviors.
With the DTU, we can validate at volumes and rates far exceeding production limits. We can test failure modes that would be dangerous or impossible against live services. We can run thousands of scenarios per hour without hitting rate limits, triggering abuse detection, or accumulating API costs.
Thought to signal-boost and get takes on what StrongDM are doing after seeing Gordon’s comment that
There are still a wide variety of software tasks I can’t trust Opus 4.6 to do autonomously, and in fact, it’s unreliably enough that I still need to manually review every line of code. To me the obvious breakpoint is when it’s reliable enough that there’s no need for human code review to reliably achieve desired outputs, or at least minimal code review with automated code review agents filling the gaps and consistently identify a more limited amount of code that does require human review.
because it made me go “but StrongDM is already there?”.
This also brings to mind a few other examples, although none of them are at StrongDM’s level (dark factory + group not solo + security software):
Liu Xiaopai, the infamous Beijing vibe coder, although of course code quality is well down his list of priorities (that would be maximising revenue growth for his Claude Code-driven budding conglomerate)
The real unlock into building like a factory was GPT 5. It took me a few weeks after the release to see it—and for codex to catch up on features that claude code had, and a bit to learn and understand the differences, but then I started trusting the model more and more. These days I don’t read much code anymore. I watch the stream and sometimes look at key parts, but I gotta be honest—most code I don’t read. I do know where which components are and how things are structured and how the overall system is designed, and that’s usually all that’s needed.
From this interview at 16:17 onwards with Boris Cherny (creator and head of Claude Code at Anthropic) I’d guess he’s somewhere ahead of Gordon and behind Peter, in that he “ships something like 10-30 pull requests a day” and “hasn’t edited a single LoC by hand since Nov ’25″, although he still “looks at the code” because he doesn’t think they’re at the totally hands-off point “especially when there’s a lot of people running the program”
I mostly pay attention to this because I keep being reminded of Rudolf L’s 2025-27: Codegen, Big Tech, and the internet section of his “history of the future” whenever I see recent developments, it’s become quite a useful and underrated intuition pump since it’s so granular.
I recently learned from Simon Willison’s Feb 7th blog about StrongDM, who are implementing a Dark Factory level of AI adoption where nobody even looks at the agent-written code, and for security software to boot (I’d be keen to get @lc’s take on what they’re doing). StrongDM’s public description of what they’re doing is here. Quote:
Thought to signal-boost and get takes on what StrongDM are doing after seeing Gordon’s comment that
because it made me go “but StrongDM is already there?”.
This also brings to mind a few other examples, although none of them are at StrongDM’s level (dark factory + group not solo + security software):
Liu Xiaopai, the infamous Beijing vibe coder, although of course code quality is well down his list of priorities (that would be maximising revenue growth for his Claude Code-driven budding conglomerate)
Peter Steinberger seems close behind but he’s building for himself and none of his projects are security software AFAICT
From this interview at 16:17 onwards with Boris Cherny (creator and head of Claude Code at Anthropic) I’d guess he’s somewhere ahead of Gordon and behind Peter, in that he “ships something like 10-30 pull requests a day” and “hasn’t edited a single LoC by hand since Nov ’25″, although he still “looks at the code” because he doesn’t think they’re at the totally hands-off point “especially when there’s a lot of people running the program”
I mostly pay attention to this because I keep being reminded of Rudolf L’s 2025-27: Codegen, Big Tech, and the internet section of his “history of the future” whenever I see recent developments, it’s become quite a useful and underrated intuition pump since it’s so granular.