The Right Amount of Spec for Agentic Development – O’Reilly

0
1
The Right Amount of Spec for Agentic Development – O’Reilly


I keep seeing the same idea in conversations about agents: Detailed specs are old-world overhead now. Give the model a rough goal, let it explore, fix what comes back, move on. It sounds efficient but it also hides the cost.

A simple prompt looks cheap and tempting because it gets implementation started right away. Then the correction loops start. You review output, clarify intent, ask for changes, rerun tests, find the next gap, and do it again. Someone still has to decide whether the result matches the real goal. That person becomes the oracle.

At the other extreme, full formal specification is obviously expensive up front. Writing acceptance criteria, contract tests, or behavior-driven development (BDD) scenarios takes real effort. But the downstream cost is different because more of the oracle is executable. A test checks the same condition every time. It doesn’t get tired, rushed, or optimistic five minutes before lunch.

That is the actual trade-off. The question is not whether specification is good or bad. It’s where the minimum total cost sits. For most agentic work, it’s somewhere in the middle: enough structure to constrain the work, enough examples to make intent concrete, and enough executable checks that review does not turn into guessing.

Zero spec is not intelligent and lean; it’s just costly vibe-coding.

The bottleneck moved, not disappeared

Software engineering was never mainly about typing or even producing code. It was about deciding what should exist, what should never happen, which trade-offs matter, and what “done” means once the problem touches the real world.

For years, teams discovered missing specification through human friction. A reviewer noticed an edge case, QA found the path nobody described, a senior engineer carried half the real requirements in his head and translated them one meeting at a time. None of that was elegant, but it did force ambiguity into the open.

Agents change that fundamentally. They make implementation much cheaper and much faster. It also means an underspecified idea can turn into a plausible system before anyone has really agreed on what the system is supposed to mean.

In the old world, vague requirements ran into human slowness. In the agent world, vague requirements run into machine speed.

That’s why specification suddenly feels important again. It was always important. We just used implementation cost as a crude forcing function and called the result process.

As implementation gets cheaper, more of the difficulty moves into deciding what correct means and checking it reliably.
As implementation gets cheaper, more of the difficulty moves into deciding what correct means and checking it reliably.

Writing the spec is not enough

This is the part I see people skip most often. They talk as if the sequence is simple: write the spec, then let the agent implement it. The missing step is the expensive one.

The spec itself needs review.

Even a careful spec can fail in familiar ways. It can contradict itself or cover the happy path and say nothing useful about retries, rate limits, or partial failure. It can describe behavior that sounds precise but cannot actually be verified. And sometimes it is precise in exactly the wrong way: it says what you wrote, not what you meant.

When an agent executes a flawed spec faithfully, the failure gets harder to diagnose. The implementation may look coherent. It may even pass the checks you provided. But the real problem lives upstream, in the spec, so fixing it means unwinding code and reasoning together.

That’s why I think spec validation deserves its own line item. Before implementation starts, someone needs to ask a few plain questions. Is this internally consistent? Is it complete enough for this task? Which parts are testable? Where are we still depending on human judgment? Which failure modes are missing because everyone silently assumed them?

Agents can help here, but only if we use them for something more useful than “write requirements.” That prompt usually produces polished fog. A better prompt is much more specific:

Draft the smallest spec that would let another agent implement this safely. Include assumptions, nongoals, acceptance criteria, edge cases, observable outcomes, and open questions. Mark which claims can become automated tests and which still require human review.

After that, hand the draft to a different agent and tell it to attack the result:

Find contradictions, ambiguous terms, hidden dependencies, untestable claims, missing failure modes, and places where an implementation could pass the written criteria while still violating the intent.

Even that simple workflow lowers the cost of getting to a spec that is worth human judgment.

Agents do not remove the need for specs. They make it cheaper to reach a level of specificity that is actually useful.
Agents do not remove the need for specs. They make it cheaper to reach a level of specificity that is actually useful.

Why multi-agent systems need stronger contracts

A single agent working on a small, bounded task can often recover from loose instructions. The loop is tight, the blast radius is local, and a human can usually steer it back on course when it drifts. Humans can even easily spot the drift to begin with.

Multi-agent systems are a very different problem. Once one agent’s output becomes another agent’s input, interpretive drift starts to compound. Agent B does not know Agent A misunderstood a requirement by 10%. It just treats the output as ground truth and keeps going. By the time a human sees the result, the original mistake may be buried under several layers of competent-looking work.

At that point, the spec is no longer just guidance but more like a contract.

That contract needs more than a paragraph of intent. It needs schemas, invariants, allowed ambiguity, validation rules, and explicit failure behavior. In many cases, it also needs contract tests, typed interfaces, and machine-checkable handoff formats. The handoff is part of the product, which is less glamorous than people hoped, but much closer to reality.

This is also where BDD and executable acceptance tests belong. Their value is not just the methodology, it’s that they move part of the human oracle into something repeatable. When behavior is stable enough to specify precisely, an executable spec is often cheaper than another round of review.

Once agents start handing work to other agents, the handoff itself needs to be specified and validated like a real interface.
Once agents start handing work to other agents, the handoff itself needs to be specified and validated like a real interface.

A spec should have an expiration date

There is another failure that teams make here: It shows up when they keep pushing on the specification curve as if more text is always safer. It is not. At least for current models it’s not.

Chroma’s work on context rot makes the first part of the problem clear: Model performance gets less reliable as the input grows, even on simple tasks. In coding projects there is a second problem on top of that. The more design prose, examples, plans, comments, tickets, and old acceptance criteria you stuff into the context, the less obvious it becomes which parts are instructions and which parts are artifacts.

I wouldn’t call this prompt injection in the security sense. Nobody is trying to attack the model. It’s closer to self-inflicted instruction drift. The context contains old design intent, current implementation, half-valid examples, generated plans from three sessions ago, and maybe a stale software design document that still describes classes that no longer exist. At that point, the model is not reading one spec, it’s averaging across competing sources of truth.

That’s when overspecification stops helping and starts confusing the model. The agent can no longer tell whether a paragraph is an active requirement, a historical note, or something the code has already replaced.

A design document is useful early because the code doesn’t exist yet. Later, it needs to shrink. Once interfaces, tests, and invariants are real, the detailed build plan should start disappearing. “Keep the parts” code is bad at expressing on its own: business rationale, non-goals, safety constraints, external contracts, and the few invariants you do not want rediscovered by trial and error. Delete the prose that just restates what classes and methods already do.

Otherwise, you end up with two specs. Humans will complain about that in review. Agents will often try to obey both.

APIs can make code behave like spec

There is also a more optimistic version of this story. Some codebases reach the “code is the spec” point faster than others, and API design is a big reason why.

If an internal API hides behavior behind conventions, weakly typed parameters, setup magic, and generic errors, an agent cannot treat the code as the spec. It has to reconstruct the rules from scattered prose and trial and error. That’s slow for humans and worse for models.

The opposite is also true. An API with explicit names, task-level methods, strong types, readable validation, useful examples, and actionable errors gives the agent something concrete to stand on. If the agent can inspect the surface area, see what a method does, understand what input is legal, and recover from errors without guessing, then the code carries much more of the specification load by itself.

This is where the AI-friendly API design ideas matter in practice. Explicit discoverability beats convention. Methods should line up with real tasks instead of forcing the agent through a dozen fragile steps. Types and validation should show what legal input looks like. Error messages should point to the next fix, not just announce failure. Introspection and examples help the model learn the shape of the API from the codebase it already has. Performance transparency matters too, because an agent will happily write a correct and terrible loop around an expensive call if the API gives it no clue.

This isn’t only about public SDKs. It applies to internal service boundaries, library clients, repository abstractions, and even the helper classes in a large monorepo. The easier an API is to discover and inspect, the easier it is for an agent to treat the code as the authoritative spec instead of dragging more prose into the context. I’ve written about all this before in more depth if you’re interested.

Where to invest

What I strongly believe is that there is no single right amount of specification. The answer depends on the kind of work you’re doing. For a small, well-bounded task, the sweet spot is usually structured intent: the goal, a few examples, nongoals, and clear acceptance criteria. That is often enough to keep the agent productive without making setup heavier than the task.

For deterministic work such as CRUD flows, API integrations, and data transformations, the optimum moves to the right. These domains are easy to constrain and easy to test. More specification pays for itself quickly because it cuts repeated review and rework. This is where BDD, contract tests, and executable acceptance criteria help most.

For exploratory work such as architecture options, research synthesis, or novel product ideas, the optimum moves left again. Over-specification can kill the very flexibility that makes the agent useful. In that case, I would rather specify boundaries than outcomes: what must be true, what must not happen, what evidence is required, and which decisions still need a human.

For multi-agent pipelines, the optimum moves right once more. Every boundary between agents needs a contract. Without that, you aren’t coordinating a system. You’re stacking interpretations and hoping they cancel out.

There is no universal optimum. The right amount of spec depends on whether the work is exploratory, bounded, deterministic, or multi-agent.
There is no universal optimum. The right amount of spec depends on whether the work is exploratory, bounded, deterministic, or multi-agent.

The common rule across all four cases is simple: Validate the spec before you scale the implementation.

What survives from Agile and XP

I do not think agents make Agile or XP irrelevant. They make the useful parts easier to separate from the parts people were already tolerating.

The first casualty is the ceremony that existed mostly to coordinate human effort hour by hour. Daily status meetings, inflated backlog rituals, and estimates presented with more confidence than information do not get stronger because an agent wrote the code. If anything, they get weaker. Agents can change the shape of a task so quickly that old effort estimates become fiction even faster than before. That doesn’t mean planning disappears. It means planning has to stop pretending it can predict implementation cost with the same comfort it had when code was the slow part.

What survives from Agile is the feedback logic. Short cycles still matter. Thin vertical slices still matter. Customer or stakeholder review still matters. Working software is still better than progress theater because agents can generate a lot of convincing wrongness very quickly. In fact, I would argue that fast feedback matters more now, not less. If a team can go from vague idea to large implementation in a morning, it also needs a way to discover by lunchtime that the idea was wrong.

XP survives even better because it was always about keeping learning close to the code. Test-first thinking still matters because executable checks get more valuable as implementation gets cheaper. Continuous integration still matters because every agent change needs a gate. Refactoring still matters because agents can happily produce code that works, passes a few tests, and still leaves you with a structure nobody wants to maintain next month. The machine has no pride here. It will generate a mess with perfect confidence.

Pair programming changes shape, but the core idea survives. I still want design judgment close to code generation. Sometimes that looks like a human working directly with one coding agent. Sometimes it looks like one model generating code while another model reviews it with a narrower brief. Either way, the useful part of pairing was never two keyboards in harmony next to each other over a coffee with their humans. It was fast design feedback before the code settled into place.

Small releases also survive, maybe for a less romantic reason. When agents can make very large changes cheaply, the temptation is to accept very large diffs cheaply too. That is a bad idea. Review, rollback, and diagnosis are easier done in small batches. A short-lived feature branch is easier to reason about than a 4,000-line monster.

What fades is methodology as reassurance. What survives is methodology as error detection. Agile and XP were at their best when they made it cheaper to discover that the team understood the problem badly. That’s still the job. The agent era just removes a few excuses and adds new ways to be wrong at high speed.

The real leverage

The promise of agentic development is real. Agents can make implementation dramatically cheaper, but once code gets cheap, specification and verification become the place where projects succeed or fail.

The teams that get the most leverage will not be the teams that specify the least. They’ll be the teams that know when three bullets are enough, when they need a real contract, and when the contract has to become executable.

The agents are getting better. The decisions are still ours.