Sources: posts I reposted on X between September and October 2026: Josh Rosen’s Software Factories for Skeptics (via Matt Stockton), Josh Rosen’s The Software Factory Debate: Build, Buy, Fork or Delegate, Lauren’s (@poteto) 2,500 PRs talk and Matt Pocock’s notes on it, and Tailscale on the Secure Software Factory.
“Software factory” kept showing up in my feed this month. My first reaction was the same as most people’s: that sounds like a lights-out shop where nobody reads the code. After reading the posts I reposted side by side, I think that framing is wrong, and the useful version is much smaller and much closer to what I already do.
This post pulls the threads together. In Background Agents: What 13 Companies Actually Run I looked at the runtime. This one is about the system around it.
Light factory, not dark factory
Josh Rosen’s Software Factories for Skeptics starts from Addy Osmani’s split:
| Dark factory | Light factory | |
|---|---|---|
| Humans read code | Rarely | Yes |
| Humans plan, review, approve | Mostly no | Yes |
| What the factory is | The agents | The system around the work |
| Where we are today | Aggressive bet | Practical now |
His point: light factories are still factories. Factories always had people in them. The factory is the system that organizes production, and you already have some version of it. The level of automation can change without changing that system.
Matt Stockton summarized the piece in one line that stuck with me: it is about org-wide agent transparency and a learning, compounding loop, not tools.
Coding agents already run tiny factories, then throw them away
Give Claude Code or Codex a big enough task and it spawns subagents: one explores the repo, one researches a dependency, one reviews. That is a small factory with workers.
The catch, per Rosen: the plan and the recipe are ephemeral. We keep the code and drop the structure that produced it. Multiply that by 100 engineers, each with several loops running, and the org sees a ticket at the start and a PR at the end. Everything in between is invisible.
A factory is just putting a system around that work so the breakdown is reusable and visible:
flowchart LR
T[Ticket / Slack / CI failure] --> F[Factory loop]
F --> W1[Worker: Claude Code]
F --> W2[Worker: Codex]
F --> W3[Human]
W1 --> V[Verification]
W2 --> V
W3 --> V
V -->|fail| F
V -->|pass| PR[PR / Deploy]
V --> D[(Run data)]
D --> S[Strategy loop]
S -->|propose changes| F
style F fill:#4A90E2,color:#fff
style V fill:#F39C12,color:#fff
style D fill:#27AE60,color:#fff
style S fill:#9B59B6,color:#fff
Iteration does not go away. An agent opens a PR, a reviewer comments, someone adds context to the ticket, the agent picks it up again. Same loop as local steering, but in a shared system instead of trapped in one session.
Software factories are data factories
This was the part I underrated. Agents generate a lot of process data: which approach they took, what tests ran, when they asked for help, whether they understood the docs. Git, CI, and Jira hold some of it. Most of it is lost.
Across thousands of tasks, that data tells you which work needs humans, where agents get stuck, and which prompt, model, or verification setup actually wins. You need a system first, then data about how it runs, and only then can you improve it.
That leads to factory as code: prompts, verification requirements, model routing, tools, context, policies, and approval points in a versioned repo. An AGENTS.md is the seed of this, not the end state.
Build, buy, fork, or delegate
The follow-up, The Software Factory Debate, compares four ways to get one:
| Option | What it means | Example | Improvement loop |
|---|---|---|---|
| Build | Compose queues, sandboxes, workers yourself | Temporal + Modal/E2B + Claude Code/Codex | You build traces, evals, replay |
| Buy | Factory platform | Warp, Factory, Port | Built in (benchmarks, scorers) |
| Fork | Adapt an open-source factory | Open SWE | Comes with tracing (LangSmith) |
| Delegate | Give an always-on agent enough access | Grok Bot, OpenAI Dots | Weak today: hard to measure |
The line I liked most: Build, Buy, and Fork put agents inside a factory. Delegate puts the factory inside an agent.
Delegate is the simplest and the least engineerable. Evidence is spread across the bot, the coding worker, GitHub, CI, and the ticket. There is no single place to see what was supposed to happen, what did happen, and whether a change made it better. Rosen’s proposed fix is checkpoints: declare what the work must satisfy and what evidence proves it, and let the agent choose the path.
His realistic answer is a mix: fork a factory, buy sandboxes, build your own verification, and delegate the coordination you do not want to hard-code.
Constraints and verification are the factory floor
Lauren’s talk on shipping 2,500 PRs in a month is the practitioner side. Matt Pocock’s notes pulled out three things:
- Lock down your agents. Humans like sharp-knife abstractions. Agents do better in locked-down environments where abstractions cannot be misused and lint rules enforce it. Her team built an internal framework to keep agents on track.
- Build verification infrastructure. Either watch the agent or make it produce evidence. Custom CLIs that let the agent drive the app and measure it, and an app that is “factory ready” from day one: deployable somewhere the agent can poke at it.
- Feature maps. Her outer loop turns vague bug reports into fixes. A maintained map of every main feature, synced by automation, is the navigation layer that makes that fast on a large codebase.
Her own follow-up is the best framing I have seen: before agent slop there was human slop. Big companies solved it with lint rules, better compilers, high-quality tests, and observability. Agents just make big-company problems everyone’s problems. None of it is novel. It is good engineering.
Security is part of the spec
The last piece is access. Tailscale put it bluntly: agents writing good code does not mean they need the keys to the entire network. They joined Agent Baseline with Docker, Keycard, Snyk, Socket, Tessl, and Tracebit on a “Secure Software Factory”. I have not dug into that spec yet, but the direction matches the “lock down your agents” lesson: scoped identity and network access per worker, not a shared god token.
What I am taking away
- Start light. I do not need a dark factory. I need visibility into the agent work I already run.
- Keep the recipe, not just the code. If a decomposition worked, it belongs in version control next to the prompts and checks that made it work.
- Capture run data before optimizing. Without traces and outcomes, every prompt or model change is a vibe.
- Constraints over cleverness. Lint rules, typed boundaries, verification CLIs, and feature maps pay off for agents and humans alike.
- Delegate with checkpoints. An always-on bot is the fastest path, but only engineerable if I define what done looks like and what evidence proves it.
For my own setup, the honest gap is the data loop. I run agents on a laptop and in the background, but I am not measuring which configurations work. That is the next thing to fix.
References
- Josh Rosen - Software Factories for Skeptics (my repost via Matt Stockton)
- Josh Rosen - The Software Factory Debate: Build, Buy, Fork or Delegate (my repost)
- Lauren (@poteto) - How I shipped 2,500 PRs last month
- Matt Pocock - Notes on Lauren’s talk
- Lauren (@poteto) - Constraints are freeing for humans and agents
- Tailscale - Joining Agent Baseline on the Secure Software Factory
- Dex Horthy / HumanLayer - Lights-out software factory write-up
- Addy Osmani - Software factories, light and dark
Comments