What a User Story Actually Costs in a Dark Code Factory – O’Reilly

Between March and July 2026, I built a production application of 861,601 lines of code. This included 696 user stories and 779 merged pull requests over 105 days, but I can’t tell you what it cost.

The first version of an autonomous SDLC framework driving Claude Code did the work. That generation didn’t record usage, and Claude Code’s default 30-day transcript retention erased the only other record. The bill isn’t approximately known because it’s gone. If measurement isn’t part of the pipeline, it doesn’t exist.

The instrumented factory

The second generation of the factory persists its own bill as it works. Every stage attempt writes its tokens (input, output, cache read, cache write), its cost, its model, and its failure category to a ledger. That first-generation application left no records because its framework kept none; this one cannot run without keeping them.

I should define the unit before pricing anything. In this pipeline, a story is the agile artifact we know. It’s a small requirement decomposed from an epic with acceptance criteria that give the agent its stopping condition. The Definition of Done is how the machine knows it’s finished. We humans batched stories into sprints to manage the delivery. In this pipeline, a story-build is just a story going through its own full delivery cycle. This includes writing tests first, the build, a coverage gate, review by a dedicated reviewer agent, and the merge. It also includes bugfixes and repeat requests when an agent’s reply is malformed. Up to five stories are in flight at once in isolated git worktrees. The dataset for this article covers every story the factory built in one repository between June 25 and July 18, 2026. This includes 17 runs, 193 story-builds, 374 stage attempts, and 336 session logs. The June runs used Claude Opus 4.8 and the July runs used Claude Fable 5, while Claude Haiku 4.5 handled smaller parts.

Usage lives in the ledger and the raw session logs, but they disagree. The logs are the ground truth for a reason I’ll explain later. The factory (claude-code-config) and the repository it built (local-code-bench) are both public. The methodology section includes the CSVs and extraction script so you can check every number. The production application from the opening remains private, so only its ghost appears here.

What a story costs

The factory consumed 595.7 million tokens to ship 77 stories, 7.7 million tokens per delivered story: at list prices for those models, $837.53, or $10.88 per story. The numerator includes every token thrown away, the five stories that ended FAILED, the 22 failed stage attempts, the bugfix and re-ask loops, and the retries; the denominator counts only stories that shipped.

In a previous essay I estimated my factory’s stories at “a few dollars to a few tens of dollars.” The meter’s answer is $3.02 to $43.24 with a median of $9.56, so my estimates were valid. Two findings in the table surprised me. Story points barely predict cost because the medium and large bands are only 8% apart at the median. The most expensive story was $43.24 for a 3-pointer that hit a review retry and a bugfix loop. The wall-clock mean is roughly two and a half times the median because the overnight run hit the subscription plan’s rate-limit window twice and stalled for hours. This was a billing artifact rather than an agent one.

Prices reflect Anthropic’s list rates as of the run dates.

696 stories of the ghost application at this rate is roughly 5.4 billion tokens. We’ll never know.

The factory is a reading machine

Here’s where my estimates were off. In that essay’s worked example, I priced a story as if cache writes were free. They aren’t, and they aren’t even small.

An agent resends the same instructions and repository context on every turn. The API caches that stable context, which makes a cached reread cost a tenth of fresh input, though it charges a premium to write new content into the cache.

95.4% of all tokens are cache reads. The factory rereads about 73 cached tokens for every new token it writes or receives. A dark code factory is mostly a reading machine that occasionally types.

The cost side of Table 2 shows where my estimate broke. Cache writes are only 3.3% of tokens but 31.2% of the bill. Cache traffic overall is 77% of the cost. Fresh input is merely a rounding error at 1.6% of the cost.

This shape is not a quirk of one pipeline. The cache-read share is 96.4% in my interactive framework-development sessions and 91.9% in the ghost’s surviving scraps: three independent samples, two framework generations, two working modes, same shape. It looks like a property of how agentic development consumes compute.

The practical consequence surprised me most: Cost optimization in an agentic pipeline is cache management, not prompt shortening. Context discipline, cache-tier awareness, and orchestrators that don’t stuff their own windows move the bill. Trimming your prompt wording does not.

The honest denominator

There are two ways to read the failure number. The narrow reading, attempts marked FAILED, is 5.0% of tokens. The honest reading, all rework, retries, bugfix and re-ask loops, plus the crashed sessions that streamed tokens and died, is about 13%. Public cost claims rarely say which reading they use.

Only 34 of 76 stories were clean first-pass, but rework stays cheap because retries are small relative to builds. I count the 13% as a quality bill because the gates catch problems.

I found a bug while dissecting the raw data that showed my meter lied. The ledger missed a sixth of real consumption, recording $694.65 against the logs’ $837.53. When a result envelope failed validation, the controller’s re-ask overwrote the original stage row’s usage. This erased the expensive failed session from the books, and crashed sessions never wrote back at all. 57 attempts were affected, which is why the session logs are the ground truth.

The measurement system needed auditing just like the code it measures. So I filed the bug against my own factory and let its fix pipeline handle it. It decomposed the report into three defects and repaired the overwrite and the model recording in one merged PR (issue #480, PR #482, 3,200 tests passing). The factory audited its own meter and fixed most of it, while the work to recover spend from crashed sessions is queued as open work.

Who actually pays

The marginal bill for all of this was zero. I run a $200-per-month Max 20x subscription, which is why every dollar in this piece is labeled API-equivalent.

The subscription’s real currency is quota rather than money. The overnight run stalled twice on the 5-hour rate-limit window, and ten dispatches waited 3.3 to 4.2 hours before auto-resuming. On a flat monthly fee, time is the fence.

One rolling month of measured work across all three codebases totals about $1,088 API-equivalent against the $200 fee, more than five to one, and that’s a floor, because older transcripts are purged. This proves a pricing asymmetry against list rates, not a subsidy: List price isn’t Anthropic’s cost; it includes their margin.

Can a professional, or a small firm, legitimately run on these flat fees? Nothing in the plan terms stops them. There’s no revenue test and no company-size cap. The line Anthropic draws is contractual, not financial. Individual seats run under consumer terms; a Team premium seat at $125 buys business terms and central administration, but roughly half the quota per dollar. Climbing the subscription ladder buys governance, not tokens.

This flat-fee window won’t stay open forever; quotas tighten and tiers reprice. A factory that meters itself will notice the day the trade turns. One that doesn’t will simply feel slower and poorer, without knowing why.

What the meter changes

I discovered while analyzing the data for this article that every number was produced with model routing switched off. Mechanical merges burned premium-model prices on Haiku-grade work, which accounted for 12.3% of all tokens. This means 7.7 million tokens per delivered story is the unoptimized rate. The article you’re reading found the bug, and the fix is already in the factory’s backlog.

A second find came from pointing the meter at myself. Writing the factory’s specifications—its epics and stories, in interactive sessions—consumed about 190 million tokens, which is roughly 25 stories’ worth of consumption (about $160 in converted terms). When implementation is this cheap, the code is no longer the expensive artifact. The difference between the $10.88 story and the unknowable 861,601 lines is that one pipeline wrote its bill down.