Every repository that has been maintained for a few years has a handful of files nobody dares open: a run.py at 34,000 lines, a single function at 700 lines, an if/elif ladder 92 branches deep. You know it should be split, but it never reaches the sprint — it ships no features and it might introduce regressions, while the team’s hours go to users. That was exactly Nous Research’s position, and their answer was to let the agent do it. On 2 September 2026 one /goal ran for 19 hours, dispatched 1,393 subagents, and produced PR #102117, removing 34.4% of the non-test source. They then published their own telemetry, the cost accounting, and the nine things that break at that scale — which makes this a useful postmortem for anyone considering the same move.

The result first

Metric	Before	After
Non-test source lines (all directories)	1,063,826	698,363 (−34.4%)
Files over 5,000 lines	37	6
Functions over 300 lines	192	2
Longest if/elif chain	92 branches	9
gateway/run.py	34,847 lines	5,512 lines
Worst cyclomatic complexity	1,075	84

The final diff was 4,271 commits across 2,655 files (+436,199 / −784,430), one commit per verified step. The main run took roughly 19 hours and 93,284 calls at about $19,300, or roughly $25,000 including follow-up sessions. Their staffing estimate for doing it by hand was $150,000–$1.8M — which is precisely why it never got scheduled.

They also published the counter-metrics. Module count rose from 1,152 to 1,551 and import edges from 5,245 to 7,135; the split entry points import slightly slower (run_agent 270 to 292 ms, gateway.run 154 to 196 ms). Splitting bought readability, and it was not free.

Why “what it costs an agent to read the code” is worth measuring

The most transferable idea here is that they treated how many tokens an agent must spend to understand the repo as a measurable metric rather than a slogan. They committed four harnesses (bench.py, lookup_sim.py, static_metrics.py, runtime_bench.py under evals/codebase_navigability/, with a README), so anyone can reproduce them:

Per “show me this symbol” task	Before	After
Lookups whose file does not fit a 128k context	1,589	0
Defining file tokens (median)	30,762	10,333 (−66%)
Defining file tokens (max)	330,028	85,075 (−74%)
Tokens actually returned per lookup (mean)	2,218	993 (−55%)

In plain terms: before the refactor, 1,589 of the symbols the test suite imports lived in a file no 128k-context model can load at all. That is not a style issue; it is a hard capability limit. The mean halved because the tail collapsed — individual definitions dropped from 38,691 tokens to 2,348 — while the median moved slightly up (564 to 680) because stripping comments makes each surviving line denser in tokens. The long tail is where the context was burning.

How it was organised

• One /goal set the standard rather than issuing instructions. The original prompt was, in essence: a massive simplification, at least 30% fewer lines, god files broken up, helpers unified, less if-if-if routing, legibility and elegance up, bloat removed, all of it done, delivered as a PR — and, critically, “no waiting for my decisions.”
• The orchestrator measured first, then divided. It split the codebase into 36 non-overlapping groups and wrote the assignments itself, so the author never briefed a worker individually.
• One git worktree per worker. Workers edited separate checkouts so they could not overwrite each other; each brief named the code to simplify, the interfaces to preserve, and the checks required before committing.
• Interface parity was a hard constraint. A tool’s JSON schema had to stay byte-identical, and a CLI’s --help output could be compared byte for byte. Every verified step had to be committed.
• The scale involved. Orchestration ran as a single Python process on an i7 desktop with 64 GB of RAM, peaking at 218 concurrent agents (about 110 on average), with inference remote. Across the run: 478 delegation batches, 1,652 subagent tasks, 111,352 tool calls (70,046 terminal, 14,223 read_file, 9,224 execute_code, 8,631 write_file, 5,042 patch, 1,328 nested delegations), a median task of 39 minutes and a p90 of 3.2 hours.

The nine things that break at scale

The forensic audit that followed (tracking issue #103563) fixed nine classes of problem. These are the walls you will actually hit:

Symptom	Measured	After the fix
Auth stampede — many agents refreshing tokens at once	620 × 401 in one window; the parent died, 329 children were orphaned, 74 re-dispatched (~$3,900)	40 expiring agents: 40 × 401 → 0
Dead subagents nobody notices	an entire dead wave went unseen for 66 minutes	surfaced in 0.3 seconds
Child summaries truncated to stubs	114 of 121 cut to 2K characters — conclusions without process	full summaries
Nested delegation timeouts	332 timeouts; only 89 of roughly 1,400 nested calls ever returned	deadline-exempt at every depth
Wrong cache tier	children wrote at the 1-hour tier ($20/M) for retention a 20-minute worker never uses	5-minute tier ($12.51/M, same loop, measured)
Tool friction	546 false hardline blocks, 454 timeout refusals, 661 whole-file rewrites (~$155)	all three classes gone
Silent loss of public API	1,703 public names and 126 tests dropped at PR open; human review found about 30	caught mechanically in 18 seconds, wired into CI
Goal-loop stall	parked 3 h 22 min on a grandchild’s poller; 4 of 5 nudges wasted	own processes only, 30-minute cap, idle-hook resume
Language-server sprawl	about 30 Pyright instances consuming 8.7 GB	worktrees share one server, with a check that diagnostics still arrive

Read the cost row carefully: the authors explicitly describe it as a per-write price comparison, not an aggregate saving, and the tracking issue separates four things on purpose — what the run historically spent, what was observed going wrong, what a simulation says a fix would change, and what actually changed. That restraint is worth copying.

The real risk lives in review

However smooth the automation, humans still catch the regressions. Two community review rounds found:

• Public names deleted because a worker saw no in-repo callers — while external plugins import them.
• An automated suppress() rewrite that changed exception handling at about 65 sites, which the existing tests did not cover at all.

The compatibility response was concrete: 2,091 moved names stayed importable from their old modules through a PLUGIN-COMPAT pointer, each verified by object identity against its new owner, and no in-tree code was allowed to use a pointer (an AST check enforced it in CI). The pointers were explicitly temporary and were reverted on 14 September 2026. Today hermes plugins compat scans affected plugins, with plugins.allow_deprecated_imports: true as the escape hatch.

The verification scale is worth noting too: the full suite reported 44,859 passed / 51 failed, and those same 17 files fail identically on bare main (same file set, same per-file counts). Regressions attributable to the refactor: 0. That frozen-baseline discipline is a habit their own skills encode — reproduce a failure in a clean environment first to learn whether it was already broken.

If you want to try this in your own repo

1. Write the goal as decidable acceptance criteria: the LOC reduction, which files get split, which interfaces must stay byte-identical.
2. Measure before you move. Their four harnesses are in the repo; the idea transfers even if the scripts do not.
3. One worktree per worker, one commit per verified step — that is what makes a failed run resumable.
4. Plan for interruption. About fifty minutes into their run the provider’s auth token expired and killed the whole thing; because commits and briefs survived, a separate session could diagnose, write a handoff, and resume successfully.
5. Budget for review. Of the post-merge fix commits, sixteen belonged to the “silently dropped public API” class.
6. Accept the trade. Module count and import time got worse; six files remain over 5,000 lines, which the team lists as the next round.

Sources and accounting

Every figure here comes from official public material: Nous Research’s own write-up (published 17 September 2026), PR #102117 (merged 4 September 2026), and the follow-up forensic tracking issue #103563. These are the project’s own run data and estimates — the cost range and the human-staffing comparison included — not measurements we reproduced locally, and the links are there for you to check.

Other angles on the same capability: live orchestration of subagents covers watching parallel agents work, heartbeats and goal gates explains why long-horizon /goal runs need heartbeats and acceptance checks, the plugin catalog is the practical context for that external-plugin compatibility section, and if several people’s agents edit one repository, neutral arbitration of merge conflicts is the natural next step.

Summary

• One /goal, 19 hours, 1,393 subagents (218 concurrent at peak): non-test source 1,063,826 → 698,363 lines (−34.4%), gateway/run.py from 34,847 to 5,512 lines, functions over 300 lines from 192 to 2.
• Cost about $19,300 (roughly $25,000 with follow-ups) against a $150,000–$1.8M staffing estimate; the price paid is more modules and slower imports.
• They turned “tokens an agent spends reading the code” into a reproducible metric: lookups whose file cannot fit a 128k context fell from 1,589 to 0.
• The real obstacles are orchestration-level: auth stampedes (620 × 401 → 0), dead children unseen for 66 minutes (→ 0.3 s), summaries truncated for 114 of 121 children, the wrong cache tier, and 1,703 silently dropped public names — now caught in 18 seconds by a mechanical check.