Building a Software Factory: Write Specs an Agent Can Build From

Part 2 of Building The React Native Software Factory, a series about tuning the setup around coding agents until you can leave it alone for a night. We cover how agents implement, test, and submit changes within that software factory workflow.

On the evening of October 9, an agent asked me what "clean" should mean for a phone.

I was planning a feature for Simlock, an open-source tool I maintain at Callstack. Simlock lets agents borrow a device, run and test an app, then return the device for the next agent. Until then, it handled iOS simulators and Android emulators.

The feature request, recorded in a GitHub issue,  was to let agents choose a real iPhone, iPad or Android phone and borrow it through the same system, with one agent using each device at a time.

We also needed to decide how to prepare a returned phone for its next borrower. A simulator instance can be deleted and recreated to provide a fresh environment; a physical phone needs a cleanup policy. Should Simlock uninstall apps, clear data or reboot it? The request left that decision open.

An implementation agent would eventually need an answer. I had already returned from an overnight run to find agents waiting for my decisions, and I wanted to settle this question while I was still at the keyboard.

That phone feature helped shape how I write specs for AI coding agents. We research the existing code, discuss the decisions, write a specification and have other agents check it before implementation. I'll use the feature to explain that process, including the parts we've since changed. What does the next agent need in the spec if it has never seen our planning conversation? Let's find out.

Write a spec the implementation agent can use

In my software factory workflow, I plan a feature with a planning agent, then hand the written plan to implementation agents. The planning agent's instructions cover investigating the code, asking me questions and writing the spec. I make the product and design decisions the existing code doesn't answer. Implementation agents follow separate instructions for changing the code and testing it; they get the plan without our conversation.

I want that separation. Planning conversations contain experiments, contradictions and ideas we abandon. Passing all of that along would make the next agent work out which parts still apply. A discarded idea could become a requirement again. Writing specs for AI agents means recording the agreed behavior with enough context for another agent to understand and build it.

We keep that spec in a GitHub issue. Requests also come from Simlock's users, and I want to preserve what they reported without rewriting their issues into instructions for my agents. We create a separate feature issue for the spec and link it back to the request. The user keeps their original report, and the agents have a place to read the agreed plan.

Before asking me to choose a solution, the planning agent reads the relevant code, earlier design decisions and related work, then draws how the system works today. I made this research an explicit instruction because agents don't always investigate enough on their own. For the phone feature, cleanup had to fit into the existing borrowing and return process. We needed to understand that process before changing it.

Work through the open decisions

The agent recommended uninstalling added apps and rebooting the phone. I asked what the reboot would add. It offered little benefit for the cleanup we wanted, and a passcode-protected iPhone would stay locked afterward until someone entered the code. I chose to uninstall added apps, with no wipe and no reboot. You see the problem? Rebooting sounded reasonable until we considered who would unlock the phone for the next agent.

I recommend asking an agent for an initial draft of the spec, with open choices marked. Reviewing a proposal gives you something concrete to discuss. A question such as "What are your reset requirements?" would have left me designing the policy from scratch.

With the recommendation in front of me, I could ask about its consequences and change the part that would leave a phone unusable without a person present.

The current interview instructions group questions into rounds. Each question covers one decision and includes a recommendation, the alternatives and their costs. Independent questions can share a round. If one answer could change another question, the agent waits for that answer before asking it. We start with what the user needs and work toward the technical choices that depend on it.

The agent keeps researching during the conversation. I decide whether returning a phone should reboot it; the agent verifies whether a command can do that.

For claims about another tool, the instructions require evidence from its source at the version we use. Commands and flags in the test plan also need to be checked, with source or command output cited. I've had enough trouble with plausible claims about other tools to want the evidence before we build a plan around them.

Define the feature’s behavior and completion criteria

For the phone feature, we wrote down what cleanup meant: remove the visible apps that weren't on the phone when we registered it with Simlock. Registration, which we call enrollment, records the starting app inventory. Simlock compares against that inventory when reclaiming the phone.

As defined in the feature spec, cleanup covers apps installed through developer tools on iOS and user-installed apps on Android. System apps and iOS apps installed through the App Store or TestFlight stay untouched.

The spec describes that policy in a few places, each answering a different question:

The implementation agent now has a rule for deciding which apps to remove, a limit on what cleanup may do and a result to check before the next agent borrows the phone.

Our feature template asks who needs the feature, how they use it and what they see when something fails. Writing this down helps me decide whether the proposed work is worth doing and how far it should go. For example, in a product with a command-line interface, an HTTP API and a UI, an agent might propose adding a capability to all three. If only UI users need it, we can limit the change to that experience and record the other interfaces as non-goals.

Accepted and rejected inputs make those boundaries concrete. Before-and-after command output shows what a user would experience more precisely than "add physical device support." With those details in the spec, agents can organize, implement and verify the agreed work without expanding it to every interface they could change.

Before we write detailed implementation tasks, a second agent reads the feature spec without the interview conversation. It checks for decisions we've left open. If it finds one, the planning agent brings it back to me and updates the spec. We can settle the choice before several implementation tasks start depending on it.

An approved spec can still leave a product decision unresolved. In Part 1, I described an agent that stopped overnight because a product decision was missing from an approved spec. A gap I could answer at my desk had become hours of waiting. I want the second reader to ask those questions while I'm still there.

Give each task a behavior to build and test

We split larger features into tasks for AI agents, each with its own GitHub issue linked to the feature. A task should give one agent a piece of behavior it can implement, test and submit as a pull request. Small features may need only one task. In our runs, long sessions made it easier for requirements to get lost among everything else in the agent's context. Smaller tasks keep the relevant instructions close to the work, and agents verify each pull request before more work builds on it.

An implementation agent starts from a task and follows its link to the feature when it needs the larger context. The request records the original problem, the feature describes the agreed solution, and the tasks explain how to build it.

We choose task boundaries using the feature's behavior table. Each row records an action, its expected result, the cases to cover and the task responsible for it. The planning agent turns those rows into testable requirements for coding agents, naming where each test enters the system and what it checks. For cleanup, we need to prove that the added apps are gone. A test that only confirms a cleanup function returned successfully could miss that requirement entirely.

Tests use simulated platform responses so they can cover different cases without needing a real phone for every one. The planning agent also asks which incorrect implementation each test would catch. If the requirement could be broken while the assertion still passes, we need to change the test plan.

Tasks need enough technical detail to work together. When several tasks touch the same value or state, the feature records who changes it, who reads it and which operation wins if they conflict. Each task refers back to that rule. Dependencies identify the tasks that need another task's code, allowing unrelated work to proceed in parallel.

Each task names the modules to change, how they interact with existing code, what happens on failure and how long an operation may wait. The evidence gathered during planning goes into a Facts table, with source locations at a recorded commit or command output.

We added this after seeing specs use a nonexistent flag or an identifier another tool would reject. The implementation agent needs to be able to check the claims its instructions depend on.

Simlock aims for about ten behavior-based test requirements per task. A much longer test plan prompts us to look for a behavior we can build and test separately. We also check that the tasks cover every completion condition in the feature. Finishing all the assigned tasks should mean we've delivered the whole agreed outcome.

Check the tasks individually and together

The feature has already had a reader check for missing decisions. Now we need to examine the technical instructions and tests in each task, along with the agreements between them. I've had implementation runs stop halfway through because an agent found two requirements that couldn't both be satisfied.

A checking agent reviews each task, works through the proposed implementation and reports where it would get stuck, have to assume something or produce the wrong behavior. A feature checker reads the tasks together to find disagreements and missing parts of the feature. The planning agent makes the edits and brings unresolved decisions to me.

Before that review, an automated spec lint checks for missing sections, placeholders, broken source locations and dependency cycles. We fix those first so the agents can spend their review on whether the plan will work.

A checker sketches the implementation and considers how it could fail after shipping. It checks for contradictory requirements, undefined terms, failure cases and conflicts with existing code, recording a status for every review category. Its investigation has a limit on file reads and searches. If it can't settle a question within that limit, it reports the gap for us to resolve.

For the phone feature, the review also covered restrictions on device commands. A checker investigated devicectl, Apple's command-line tool for physical devices, and found a route that could copy a host file onto a phone's clipboard and read it back. Our proposed restrictions would have allowed it. We changed the spec to allow only a reviewed list of commands before implementation.

Recheck the changes before approving implementation

A later check made me change a decision I'd already made. Simlock sends commands to Android phones through adb, which has a command-line client and a background server running on the computer.

Real phones use the computer's shared adb server, which other tools may also be using. If Simlock's client expects a different protocol version, adb can stop and restart that server. That could interrupt other work using the connected devices. We wanted Simlock to check compatibility and refuse a command on a mismatch, without restarting the shared server.

I initially chose to reuse a recent compatibility check. We would save the result saying that the client's and server's protocol versions matched, and accept it for a short period. Once that result was too old, Simlock would refuse the command and tell the agent to try again.

The checker followed this rule through a round involving several phones. By the time Simlock reached a later phone, the saved result could have expired. The phone and server could both be working, yet Simlock would reject the command because its earlier check was too old.

I changed the requirement to check the server immediately before each physical adb command. Each command would get its own compatibility check, so a stale result would no longer force an agent to retry.

In the current workflow, the same checking agent reviews the revised spec as a diff. It checks whether the changes address its findings and whether they cause problems elsewhere. It already knows what it investigated, so it can focus on the fixes and the requirements they affect. The planning agent also rereads other tasks that use any changed shared values.

Fixes can introduce new gaps, and another review can lead to another rewrite. We limit the number of rounds so the agents eventually bring an unresolved problem to me for a decision instead of continuing indefinitely.

I can ask for a change to the plan or explicitly accept a finding. Smaller gaps that an implementer can handle with a stated assumption remain recorded as notes; they don't trigger another round. The approval summary lists unresolved findings, so I can see what I'm accepting.

I approve the feature, its technical decisions and its tasks together. The planning session then marks the feature as planned and the tasks as ready for implementation. Agents can pick them up in dependency order, with the decisions and evidence available in the issues they read.

Try the handoff on one feature

Those issues are what the next agent receives when it starts implementation. For the phone feature, they need to explain where cleanup fits into borrowing and returning a device, which apps to remove and why cleanup must avoid wipes and reboots.

The research and interview give us those answers; the written spec preserves them without the abandoned ideas from our conversation. Each task then says what to build and how to test it.

The checking agents examine whether that plan holds together. In the phone session, they found the clipboard route and the problem with reusing an old adb check. We changed the command restrictions and required a fresh compatibility check before each physical adb command. Rechecking made those corrections part of the plan I could approve.

Implementation can still reveal behavior in the existing code that our research missed. That may require another decision and an update to the spec. But a question such as whether cleanup should reboot a phone is one we can discuss before coding starts. I want to use the planning session to settle those choices, so agents have fewer reasons to wait for me during an overnight run.

Choose one feature you’re about to hand to a coding agent and use our feature specification template as a starting point for the written plan. Have it research the existing behavior, propose a draft and ask you about the open choices. Write down the outcome, split the work where needed and describe how each task will be tested.

Then give the plan to a fresh agent without the conversation and ask where it would get stuck. Resolve its findings, recheck the changes and approve the plan. If questions still reach you during implementation, use them to improve the next spec.

Next in the series: how to review the code agents write, with reviewers that don't share the writer's blind spots.