Running a Coding Agent Fleet, Part 1: The Day the Agents Built Their Own Gates

Sources: my private workspace records for that day (the mission’s spec, progress log, slice proofs and review verdicts), and the merged PRs in my public skills repo. Times are UTC.

I run a fleet of coding agents: several teams of them, on three machines, each team working through one project. For months the process lived in prose. My root AGENTS.md was 4,496 words. Reviews passed because a reviewer agent said “looks good”, and nobody could tell which diff it had looked at.

On Sunday 8 November I wrote seven decisions down and handed them to the fleet. By the next morning the agents had built most of the gates that enforce them, and the gates had already caught real mistakes, including some in the gates.

This is part 1 of a series on running that fleet. It covers that one day.

Seven decisions

The mission was called workflow foundation. Its intent line:

Process is defined by proof contracts and enforced gates: every slice carries intent, a proof contract, a non-author verdict and a retro line, seats are roles in domain pods, and skills are gate, tool or lens kinds checked by evals.

The seven decisions behind it, in short:

	Decision	What it means in practice
D1	Proof contracts and enforced gates	Every slice has an intent, a checklist of proof, a verdict from someone who did not write it, and one retro line. Checks enforce it, not prose.
D2	Pods are domains, seats are roles	orch.lead, dev.build, dev.qa, dev.review. A lint enforces that the reviewer comes from a different model family than the builder.
D3	Three kinds of skills	Gate skills, tool skills, lens skills. Each SKILL.md stays short and ships with 3+ evals.
D4	Skills are written evals first	A new skill must beat the same model without it.
D5	One record shape	SPEC.md, PROGRESS.md, PROOF.md and reports/, in one place per slice.
D6	Verdicts bound to the diff	A review names the exact diff it read. Change the diff and the verdict goes stale.
D7	Roll out in waves	Dogfood on my own workspace rig first, then pilot rigs one at a time, never a forced restart.

The work was split into 13 slices. A slice is the unit of work here: one outcome, one owner seat, one worktree per repo, usually one PR.

How the day went

The first log line is at 02:28. Two seats ran in parallel: one red-teamed the seven decisions against outside sources, the other took an inventory of what already existed. The inventory was not flattering: 96 skills, 131,839 words, 92 of them over the 500-word target.

Both reports were fact-checked by a reviewer from a third model family, fixed, and turned into an ADR. I accepted it at 03:26. Phase 0, from first log line to accepted ADR, took about an hour.

Then the build:

Slice	What landed
04 checks	Five commands: rig-lint, verdict-check, pr-lint, slice-check, merge-gate
05 create-skill	A skill for writing skills with evals and a no-skill baseline. It replaced the old skill-creator.
06 gate skills	code-review, intake, shape, retro, verify, each with evals on two models
07 records resolver	One script that says where a slice’s records live
08 roles and templates	Three rig templates: solo, workshop, factory
09 prose diet	Root AGENTS.md from 4,496 to 1,806 words, rules as a table that names the check enforcing each row
10 deletions	Six unused skills deleted, 96 down to 90; more scheduled once they sit unused for 14 days
11 rollout	The other rigs switched in waves, the last ones the next morning
12, 13	A one-seat “content” rig kind (the one writing this post), and a statechart lens skill

Twelve of the 13 slices closed on the 8th. In the roughly 30 hours from the first log line to the last rollout wave, the public skills repo merged 46 PRs, 8 of them release PRs.

That is the summary. The parts worth writing about are where something pushed back.

A verdict that knows which diff it read

Before this, a review was a chat message. Now it is a file with frontmatter: the axes it judged (spec, standards, and security on risky changes), the findings with ids and tags, the reviewer’s seat and model family, and two hashes of the diff:

axes: {spec: pass, standards: pass}
patch_id: 21217d35fc70...        # git diff base..head | git patch-id --stable
diff_digest: sha256:4882e91d...  # sha256 of git diff --binary base..head
reviewer_family: muse
findings:
  - {id: F2, axis: standards, tag: Nit, status: fixed}
  - {id: F7, axis: spec, tag: Nit, status: open}

verdict-check recomputes both from the branch. The patch-id survives a rebase, so a verdict stays valid when the branch only moved. It does not catch whitespace: a 1-space and a 4-space indent gave the same patch-id. The digest does catch it, so an indent-only change after review marks the verdict stale. There is a test for exactly that case.

The first verdict bound this way was the checks slice’s own. It survived two rebases onto main with the same patch-id and digest, and then it landed.

The family lint, and the check that failed open

D2 says a reviewer must not share a model family with the builder. Two seats on the same model tend to agree with each other, including on the same blind spots. rig-lint maps model ids to families (claude, gpt, glm, grok, gemini, kimi, composer, muse) and fails a rig where a review or QA seat shares a family with a build seat.

Its first run was humbling. Across the 26 rig specs on one machine, 16 had a failing family finding. My own workspace rig was one of them: its second reviewer ran Claude Opus, the same family as the builders.

The better story is what happened inside the slice. The builder was on Claude, so the reviewer for the checks themselves was a one-shot from a third family. It found F2, filed as a Nit and fixed as a must-fix: the family check in verdict-check passed silently when reviewer_family was missing, unknown, or in a different case. The independence check failed open on its own metadata. The tests only covered the case where the field was present.

The retro line for that slice says it better than I would:

A check that reads reviewer-written metadata needs a test for the field being absent.

Evals that were wrong, not the model

D4 says a skill has to beat the bare model. The create-skill skill was the first one held to that. It shipped with four scenarios, three runs each, with and without the skill:

Model	With the skill	Without
Claude Sonnet	12/12	0/12
Muse	12/12	0/12
GLM	6/12	0/12

Then a Claude Opus round came in at 2/3 and 1/3 on two scenarios. The builder seat read the saved outputs before touching anything. Both misses on the second scenario were correct refusals worded “I didn’t create a skill”, and the check’s regex only knew “not create”. The other miss was a helper folder that the check counted as a scenario, while the runner skipped it. The checks were wrong, not the skill. They were fixed, the change was disclosed as “changed after results”, and Opus went to 12/12.

This happened twice more during the day. A later eval passed 3/3 without the skill on both Claude models. It was too easy, because a strong model found the answer by reading package.json. It was hardened until the baseline failed.

35 out of 35 sync failures had one cause

My workspace records sync through git on a timer, on every machine. On the 8th the sync kept stopping. It went down three times: for 4 h 12 min, 36 min and 3 h 21 min. Each time a person (me, through an operator session) repaired it by hand.

The seat that took the fix started by counting. The sync log on that machine had 35 integrate failures. All 35 were the same error: a conflict while replaying the local commit. None was the other failure mode. The cause was a habit, not a race. A lead seat committed records from its own worktree while other seats edited the same files in the live checkout, so two commits changed the same record file.

The fix has two halves:

• A merge rule. Append logs (PROGRESS.md, NOTES.md) keep both sides with git merge-file --union. Any other record keeps the newer side, and the older one is named in the sync log. Code, specs and docs still stop the round for a person.
• A rule. Records are written in the live checkout and committed only by the sync. Worktrees carry code, specs and docs.

The retro for that slice found its own escape: the new commit format put Seat: retro on the first live commit, because a line starting with retro: looked like a seat name. That got fixed the same evening, with test cases for retro, review and lead.

The first PR end to end, and a real find

The real test of D7 was a project rig relaunched fresh on the new templates, then given one PR through the whole flow. It was a client API change. The rig came up at 20:52 and the PR merged at 21:35.

The first verdict was BLOCKING. The new reviewer seat found an env file holding an API key that was not gitignored. That is a real find, not a style nit. The fix was one line in .gitignore. Round 2 passed, and the merge gate held all six gates on its third run, without an admin override.

The lead of that rig wrote a read-out with ten frictions. Three of them:

• The retro gate counted per slice, so a second PR passed on the first PR’s retro line.
• The review round counter counted other PRs’ verdicts toward this PR’s cap.
• Proving a .gitignore change with git check-ignore read the base branch’s file, not the PR’s.

The first two were fixed that night in the checks and the skills, after a review that was itself BLOCKING once on three holes in the new per-PR checks. The third became one line in the code-review skill.

Quota juggling

All of this ran on subscription quota, and the quota set the pace more than anything else.

• 03:26: my main Claude account was at 100% on both the 5-hour and the 7-day window. One writer at a time.
• 06:02: I moved the building seats to Opus on high effort anyway. Quality before quota. Reviewers stayed off Claude: Muse and GLM one-shots.
• 08:25: Claude’s 5-hour window dropped under about 10%. Two seats moved to GLM, and their work was reviewed by Muse, since GLM wrote it.
• Afternoon: an Opus eval run hit 429s and stopped at the cap. The Claude eval runs went on a list for later.
• 16:15: one probe with 12 Opus sessions showed no visible change in quota at whole-percent resolution, so the list ran: about 156 sessions moved one account’s 7-day window from 65% to 67%.

The family lint and the quota pulled the same way. Reviewers had to be another family anyway, so the reviewers ran on the accounts with room. The next day that became the fallback until the weekly reset: builders and leads stay on Claude, reviewers run GLM for code and Muse for anything with screenshots.

Slips that became rules

Agents slipped too, and the records kept them:

• A seat backgrounded cd <worktree> && harness &, then ran git commit in the same call. The cd only ran in the background subshell, so the commit landed in the live workspace checkout and swept in two other seats’ files. It was not pushed, and it was undone with a mixed reset.
• A seat killed its eval harness with pkill -f <pattern>. Kill by pattern matches every command line that contains it.
• The lead ran find / to locate a skill.

Two of them are now lines in the rules every seat loads: never kill by pattern (start with cmd & pid=$!, then kill "$pid"), and never scan from /. The pattern-kill rule names the incident that made it a rule: the same day, a pattern kill stopped my board server. The background cd is a recorded lesson for now, not a rule. A rule that names its incident is easier to keep than one that only names a principle.

What is still open

• The rollout’s later waves are switched, but the seat renames wait for each rig’s next natural relaunch.
• The rest of the delete list becomes eligible between 11 and 22 November, once each skill sits unused for 14 days.
• GLM passes create-skill at 6/12. I did not tune the skill for it. Whether the bar should be per model is an open question.

The lesson from the day: none of the checks had to be clever. A hash of the diff, a table of model families, a regex for a retro line. They worked because they ran on every slice. The first real catches were in the checks themselves.

Start your own

I am turning the skeleton of this setup (records layout, rig templates, the check commands) into a template repo. It is public now: agent-project-template.

Part 2 will cover the week after: whether the gates held up on real client work, and what the pilot rigs’ numbers say.

References

• vanducng/skills: create-skill, code-review, intake, shape, verify, retro, openrig-workspace
• OpenRig, the rig runtime the fleet runs on
• rexboard: An Attention Board for My Coding Agents, how I watch the fleet