How we let AI coding agents write production code

Our CTO, Kō, is an AI agent, and it writes our production code. No change is trusted because an agent says it works. Each change passes checks that can fail, something other than the agent that wrote it runs them, and a person approves the change before it ships. This essay explains how that works and what it means for a project we take on. The short version of our working rules is on our approach page.

The instruction file is maintained like code

Every repository we work in carries an instruction file that the coding agent reads at the start of a session. It sets the working rules: state assumptions before starting, write the minimum code that solves the problem, touch only what the task needs, and define how success will be checked before writing anything.

Rules that helped one generation of models can hurt the next. Older models often skipped checks and stopped short, so instruction files told them to verify. Many newer models already check their own work, and some widen a task beyond what was asked. For those models the file drops instructions such as "double-check your answer", which add cost without adding quality, and tells the agent to deliver the scope it was given.

The file is kept short, because the longer it grows, the more likely a rule is missed. A rule that must always hold belongs in an automated hook that enforces it. Knowledge needed for only one kind of task belongs in a separate guide loaded when that task comes up.

Every change to the file is recorded with a date, what changed, why, and the source it was checked against. When the model changes, the file is checked again.

The agent that writes the code never verifies it

The work runs as a pipeline with distinct roles:

• Investigate: is the change feasible, and what would it break?
• Plan: phases, tasks, the shared contracts between them, and a check command for each task.
• Execute: one task at a time, each proved with a command that can fail.
• Review: each finished phase is validated by a separate reviewer agent.
• Approve: a person reads the result and decides whether it ships.

The reviewer is the external referee. It starts with an empty context, so it cannot have watched the code being written. It receives the range of changes, the plan and the aspect it should review. It never receives the conversation that produced the code, because that conversation carries the author's assumptions.

The reviewer does not trust what the executor recorded. It runs the tests, the linter and the build again itself and records each exit code. A failure on any of them forces a "changes required" verdict, whatever else the review says.

The reason is simple. An agent's notes on its own work are written by the agent, and nothing validates them. A task can be recorded as passing when it never ran. Re-running the commands is the only way to know.

The reviewer's verdict then goes to a person, with the commands, exit codes and findings attached. That person approves the change or sends it back. No agent in the pipeline can approve its own work or anyone else's.

Findings are attacked before anyone acts on them

Reviewers are asked for every finding at every severity, with a file and line for each. A model asked for "only the serious issues" tends to take that literally and report less. Severity is assigned in a separate pass, so the reviewer that found an issue does not grade it.

Each finding then faces a refutation pass. On changes where a mistake is costly, such as authentication, payments and data migrations, a finding is attacked from three sides. Does the code really do what the finding claims? Is the impact real in this system? Can someone build the input that triggers it? A finding most of them refute is dropped and recorded as raised and refuted, so nobody raises it again in the next cycle.

Before trusting a check, prove it can fail

A test that cannot fail proves nothing. Before a check command is trusted, the executor runs it once against a deliberately wrong version and confirms it fails, then against the right version and confirms it passes.

The same idea catches flaws that sit between phases. A type checker, for example, compares declarations with the code that uses them. It does not compare them with the implementation. A phase can declare a new method, pass every type check, and still throw an error when the method is called, because nobody wrote it yet. A reviewer looking across phases finds that. A check confined to one phase does not.

Failure stops the work early

The executor gets a small, fixed number of attempts to fix a failing task. Past that, more attempts rarely help. If other tasks depend on the failed one, the phase is marked blocked and the executor stops. It does not keep building on a broken step.

The review loop has a hard cap too. After a few failed cycles on one phase, the plan is treated as the problem, and a person decides what happens next. The executor may not change a shared contract either. If an interface must change, it stops and reports which phases depend on it.

Parallel agents cannot overwrite each other

Running two agents at once saves time and creates a specific risk. Two agents that each read a shared file, change it and write it back can lose one agent's update. If the writes overlap, the file can be left half written and unreadable. Asking the agents to be careful does not fix this. The fix is structural: no file has two writers.

• The plan is written once and then left unchanged.
• Each executor writes only its own state file for its own phase.
• The reviewer writes only its own review file.
• The coordinator reads all of them and writes none.

Each phase that runs in parallel also gets its own git worktree, a separate working copy of the repository on its own branch. Two agents therefore never share a working tree. Separate branches that edit different parts of the same file usually merge cleanly afterwards, and git shows any conflict for a person to resolve. The risk lies in two agents writing one path at the same moment, and separate worktrees remove it.

Worktrees have a practical catch. Installed dependencies are usually excluded from version control, so a new worktree starts without them. Install them in the worktree first, or every check fails with a missing package that has nothing to do with the change.

The method has its own regression tests

The pipeline itself is tested. When running it on real code exposes a defect in the method, that defect becomes a test case. Each case is checked the same way as any other test, by reintroducing the original defect and confirming the case fails.

When we skip the pipeline

Orchestration costs time, and it pays only on long work with real contracts between phases. Planning a one-line change produces a plan longer than the change. For a single-file fix, the agent makes the change and the tests run. A person still approves it.

What this means for you as a buyer

Work delivered this way carries evidence you can inspect. The plan is committed to the repository. Each phase records the commands it ran and their exit codes. Each review lists its findings with file and line, including the ones that were refuted, and each change records who approved it.

When you hire anyone who uses coding agents, ask them these questions. Who checks the agent's work, and does that checker share the agent's context? Are the checks re-run, or read from the agent's own notes? What stops two agents overwriting each other? What happens after repeated failure? Which person approves each change? Ask us for the review report on any change, and judge the work by it. How the code becomes yours is covered on our approach page.