The factory: from idea to a change I can trust
The steps around my coding agents: what each one does, the problem it addresses, and what changed when I started using it.
I wanted anyone on our team to ask for a product change and get back something I could review and merge. Getting a coding agent to produce code was only part of that problem. I also needed a way to agree on what we were building, challenge the implementation, and check that it behaved as intended.
The diagram shows the software factory workflow I use. Each box is a step around the coding agent, from defining the request to a person deciding whether to merge the change. Here’s what each step does and what I’m learning from it.
Click the titles to see more details.
Based on spec-driven development (SDD), we define the expected behavior before implementation and use that spec to guide the build and its acceptance checks. Before asking an agent to write code, I want us to agree on what the product should do. The agent reads the existing code, then asks me about decisions the code can’t answer: who needs the change, what happens when it fails, and what we’re deliberately leaving out.
This conversation uses the approach from Matt Pocock’s grilling skill: ask questions in rounds, settle the decisions we can make now, then use those answers to uncover the next questions. In the factory, that conversation becomes a spec. These are the five sections I want to highlight.
What goes into the spec
- Who has the problem, what happens today, and what it costs them. Considering using the VISION.md approach to align each requirement with the product’s long-term vision.
- The outcome a person using the product should notice.
See the changes at a glance.
- mandatory!
- Given / When / Then, including failure paths. Each one becomes a test assertion.
~80% better quality
- Approved screens and states. Plain HTML wireframes worked better for me. If no screen changes, say so. I tried design-tool integrations first.
- What this change deliberately leaves out. YAGNI: You Aren’t Gonna Need It.
What this changes: a rough request becomes an agreement about success, failure, wording and scope. We can challenge that agreement before paying for an implementation built on assumptions.
I could just hand the spec to an implementation agent, say “Build it,” and leave the implementation details to it.
But too often, I found myself asking afterward: “Why did you create a new table?” or “Why did you introduce a new domain object?” By then, the agent had already built code around that new table or concept. Changing the decision meant redoing the implementation. That made me realize I needed to be explicit about the important decisions before the agent started building.
Now I ask the agent to propose the changes in a technical design document, and I review them while they are still proposals. We agree on what to reuse, what to add, and how the pieces fit together. Those decisions then go into an implementation plan for the agents doing the work.
The agent fills in a fixed template for each document. The sections below are required, so it has to work through the same questions every time. It must ground its answers in the spec and the code it inspected, and flag missing information as an open question instead of inventing an answer. This helps prevent hallucinations and gives me a consistent document to review.
Technical design document
This is the proposal I review before implementation starts. It explains how the change will work, what we can reuse, and what needs to change. I use it to question the important decisions while they are still easy to change.
- What changes
- A short description of the outcome. It keeps the review tied to what a person using the product should notice.
- Open questions
- Unresolved product decisions and contradictions between the spec and the code. Each needs an answer before I can approve the design.
- Requirement flow
- The actors, actions and branches in the requested behavior. Walking through the flow makes missing steps and failure paths easier to spot.I’m visual. BPMN helps me follow the flow faster.
- Approach
- How we will build it, why this approach, and what we deliberately leave out. This is where I challenge unnecessary tables, layers or extra work.
- Solution components
- The parts of the system involved and how they communicate. Marking what is new or changing shows how far the change reaches.Component + sequence diagrams help me see what changes.
- Domain entities
- Changes to the concepts the product works with, such as an order and its status. Defining each field’s meaning prevents different parts from interpreting it differently.
- Database changes
- The tables and columns affected, including what happens to existing records. I can review the consequences for stored data before a migration changes it.
- HTTP contracts
- The requests and responses exchanged with the server: fields, permissions and errors. Agreeing on these interfaces lets separate agents build compatible pieces.Agree on the interface, then frontend + backend agents can work in parallel.
- UI
- The proposed screens and states, compared with the spec’s references. When they differ, wireframes make the change visible before it is implemented.
- Who does what
- Assignments, their order, and the model capability each needs. This shows what can run in parallel and where shared files require work to happen in sequence.An orchestrator starts the agents, waits for their changes, then merges them into a shared branch.
- Acceptance
- Each agreed case, its impact, who checks it, and the evidence I will see. It connects the proposed solution to observable success, rather than a claim that it works.The evidence could be tests, logs or even a video. See more in Prove the behavior.
- Risks
- What could still go wrong after shipping, what triggers it, how we would notice, and what we would do. Each risk must be grounded in something the agent checked.
Implementation plan
This is the document the agents work from. Each implementing agent receives the shared rules and its own assignment, including the relevant contract and scenarios. The plan carries the details that would distract from reviewing the design.
- Revision and checked code
- The design revision this plan follows and the code version used to verify its scenarios. These establish which decisions and assumptions the build starts from.
- Rules for every assignment
- Shared limits on file ownership, isolated working copies, review and checks. Every agent receives these rules so parallel work does not overwrite another agent’s changes.
- Contract
- The agreed interface, with exact types and implementation details. Agents on either side build against the same definition instead of inventing incompatible versions.
- Ticket texts
- The exact labels, messages and other text the spec requires. Copying them here keeps implementation and assertions from drifting into different wording.
- Decisions
- The settled choices and the records or states they apply to. An agent can distinguish “failed orders with no reason” from every order with no reason.
- Assignments
- Each agent’s deliverable, model, dependencies, area, files, starting points, wireframes and assigned checks. It can start in the right place without repeating the coordinator’s search.
- Acceptance
- The full scenarios, who runs each check, how to create the starting state and observe the result, and which code it covers. This gives the builder a concrete check to implement.
- Build log
- A record filled in during the build: work started, review findings, checks, commits and pending decisions. A new session can resume from recorded progress rather than reconstructing the chat.
The two must stay in sync. When the design changes, the plan changes in the same revision. Changing an approved approach, interface or assignment requires another design approval. Adding a file within an assignment’s existing scope only updates the plan. This keeps the work tied to the decision I actually approved.
What I’ve noticed: with structured specifications, design review and visible test evidence, I almost never need to ask for reimplementation or correct the agent.
A coordinating agent gives each assignment to an agent in its own working copy. The agents implement their parts and run the relevant local checks; the coordinator combines the work.
A fresh reviewer challenges the combined change without reading the builder’s reasoning. It normally uses a different model provider. Its job is to look for mistakes the planned scenarios might miss, such as sending something twice or exposing one customer’s data to another.Save this for important changes, or your bill will hurt!
This is the step that gives me the confidence to merge. I can open a pull request—the proposed code change—go through the agreed scenarios, and inspect the evidence for each one. I can see what the application actually did.
The acceptance cases we defined at the start become the checks here. Each check sets up the agreed situation, performs the action, and verifies the expected result, including the failure paths. The evidence can take different forms:
- A video lets me watch the running application: the action, the response, and what ends up on screen.
- Test results show whether a specific rule holds, including behavior that is hard to demonstrate through the interface.
- Logs let me inspect what happened behind the scenes and connect it to the outcome we expected.
The important part is the connection between the requirement and the evidence. If a scenario says someone should receive a notification, the check needs to show that person received it. A record existing somewhere in the database would leave the important question unanswered. The check also needs to catch the missing behavior: if it would pass before the change, it cannot demonstrate that we delivered it.
This makes the PR something I can work through, case by case: what we agreed should happen, what ran, and what the evidence shows. I can watch the recording, inspect the result, and question anything that does not match. That matters especially when agents wrote the implementation. It gives me a concrete way to judge their work without reconstructing every decision from the code.
The evidence must belong to the version I am reviewing. If a correction changes the behavior, the affected checks run again and their evidence gets refreshed.
For me, this is where the earlier work pays off. The spec gives us the cases, the design gives the agents a plan, and this step lets me see whether they delivered. I can open the PR, check the evidence, and trust my decision to merge.
Version dispatcher prompts without interrupting conversations #42
main from prompt-versionsThe video is from the real PR; names, code and logs are recreated.
The first test reused the original message ID. That did not prove what happens on the next message after publication. Send a distinct second message and assert the version it actually uses.
Commits
e1a2b3cf4d5e6aAll checks passed
Passed Publish version 2
Given: version 1 is live; version 2 is saved.
When: an admin publishes version 2.
Then: the live version is 2.
PASS publishedVersion == 2
Passed Existing conversation keeps version 1
Given: conversation-a started on version 1.
When: version 2 is published; message-2 arrives.
Then: the reply still uses version 1.
PASS message-2.id != message-1.id
PASS reply.promptVersion == 1
Passed New conversation uses version 2
Given: version 2 is now live.
When: conversation-b starts.
Then: its first reply uses version 2.
PASS reply.promptVersion == 2
Files changed
await publishPrompt(version2);− await receiveMessage(firstMessage);+ const reply = await receiveMessage(newMessage);+ expect(reply.promptVersion).toBe(version1);Greptile checks
Greptile adds another review on the pull request. It is an AI code reviewer that reads the changes with the surrounding codebase as context and leaves findings as PR comments.
It has caught issues in my changes, and I see it as a useful extra. But each finding still needs an agent with the context of the requirement, the design decisions and the code to decide whether it is a real issue. A review comment alone is not a reason to change the implementation.Save this for important changes too. It costs money!
That agent checks the findings, fixes the issues that apply, and explains why it disagrees with the others. If a fix changes the behavior, the affected checks run again so the evidence matches the change I will review.
This is the return loop in the diagram. When an agent breaks an existing rule that could be broken again in unrelated work, I record the trap so future agents can learn from it. The next step I want to explore is making those mistakes fail automatically: through a type, a structural constraint, a lint rule or a test.I keep traps in a Markdown file today. I’m considering deterministic guards—checks that fail automatically when a rule is broken.
I’m still figuring out which parts need more structure and which create unnecessary work. Which box would you add, remove or change in your own workflow?
Working through something similar? Leave me a note on any part of this page.
Summary
Admins can publish a new dispatcher prompt without a deployment. Conversations in progress keep the version they started with; new conversations use the published version.
Acceptance
Recorded in the local app with test data. The admin confirms promotion and version 8 becomes Live.
Read the video description
The dispatcher page starts with version 1 marked Live. The admin selects Promote for version 8. A dialog explains that new conversations will use it within a minute and ongoing conversations keep their version. The admin confirms; version 8 is then marked Live.
Evidence: the conversation keeps its version
Given a conversation started on version 1, when version 2 is published and a second message arrives, then the existing conversation still uses version 1.
second-message.log