Green Is Not Done: Write Agent Loops as Evidence Contracts

Green is a color in GitHub. It is not a definition of done.

That became obvious while I was rewriting a prompt for a coding-agent workflow. The original shape looked sensible:

1. Ask a planning agent to inspect the repository.
2. Ask an implementation agent to follow the plan.
3. Push a branch and open a pull request.
4. Ask a fresh reviewer to inspect the diff.
5. Fix anything CI or review bots flag.
6. Repeat until everything is green.

Most of that is good. The roles are separated. The implementer gets a focused context. The reviewer is not grading the code it just wrote. GitHub supplies feedback. The main thread stays alive long enough to coordinate the phases.

The problem is the last line.

"Repeat until green" sounds precise because green is visible. It is actually a thin completion condition wrapped around a potentially expensive and consequential loop. Which checks count? Did they run against the latest commit? What if a check is pending, skipped, flaky, or misconfigured? What if a bot comment is wrong? Who may change scope? Who may push, notify a team, or merge? When should the loop admit that it is blocked?

The better prompt is not merely longer. It is an evidence contract.

The leverage moved from prompts to loops#

Addy Osmani's loop-engineering essay gives the broad shape: scheduled discovery, isolated worktrees, reusable skills, connectors to real systems, separate maker and checker agents, and durable state outside the conversation. The loop finds work, delegates it, checks it, records what happened, and decides what should happen next.

That is a real shift. The engineer is no longer writing every conversational turn. The engineer is designing the system that decides which turn comes next.

But there is a trap hiding inside the excitement. A one-off prompt fails once. A bad loop can fail repeatedly, confidently, and at machine speed.

The ChatPRD overview of agent loops shows two useful examples: a daily routine for aging pull requests and a scheduled process that mines recent work for reusable agent skills. It also names the practical costs. Nested agents consume tokens quickly, and vague success criteria can leave a goal-based worker producing activity without progress.

I would put the rule this way:

That rule is not anti-automation. It is what makes automation worth trusting.

First choose the trigger#

The word "loop" gets used for several different mechanisms. They should not share one prompt because they do not share one risk model.

A schedule finds recurring work#

A scheduled loop runs at a known time: review pull requests each weekday morning, inspect dependency alerts every Friday, or summarize yesterday's failed builds. Its main job is discovery and triage. A healthy run can conclude with a no-op: nothing needs attention.

A heartbeat watches changing state#

A heartbeat checks periodically because the thing being watched changes independently: a deployment, a batch job, a queue, or a long-running benchmark. It should compare new state with prior state, back off when nothing changes, and stop when the watched process reaches a terminal state.

An event reacts to something specific#

A hook or webhook starts from a concrete event: a pull request opened, a check failed, an issue labeled, or a review submitted. Event-driven loops should be idempotent because delivery may happen more than once.

A Goal persists until evidence settles the outcome#

A Goal is appropriate when the finish line is clear but the path depends on what the agent discovers. OpenAI's official Codex Goals guide describes Goals as thread-scoped completion contracts. The objective persists across turns, continuation happens at safe boundaries, budgets remain distinct from completion, and the work should finish only when concrete evidence supports the claim.

Do not turn a one-line edit into a Goal. Do not put "watch this deployment" inside an unconstrained Goal when a bounded heartbeat will do. Do not schedule a loop whose real requirement is to react immediately to a webhook. Pick the trigger that matches the work.

The six fields every serious loop needs#

The official Goals guidance provides the cleanest checklist I have found. A strong loop contract names six things:

1. Outcome: what must be true when the work is finished.
2. Verification surface: which tests, benchmarks, artifacts, commands, or authoritative sources prove it.
3. Constraints: what must remain true while the loop works.
4. Boundaries: which files, systems, tools, data, and actions are in scope.
5. Iteration policy: how the next attempt should be selected from evidence.
6. Blocked stop condition: when the loop should stop because no defensible path remains.

I add three operational fields when the loop can touch GitHub or another shared system:

• Authority: which reads and writes are permitted, and which require human approval.
• Budget: maximum iterations, elapsed time, token or compute spend, and concurrent workers.
• Ledger: where attempts, evidence, decisions, and unresolved findings survive outside model context.

The compact version is:

Achieve <OUTCOME>, verified by <EVIDENCE>, while preserving
<CONSTRAINTS>. Work only within <BOUNDARIES> and use only
<AUTHORIZED ACTIONS>.
 
After each attempt, record the action, result, and changed evidence in
<LEDGER>, then choose the smallest next action supported by that evidence.
 
Stop successfully only when every completion condition is proven against
the current state. Stop as blocked when <BLOCKED CONDITIONS> occur or when
<BUDGET> is exhausted. Report attempts, evidence, remaining uncertainty,
the exact blocker, and the smallest input that would unlock progress.

This is the reusable center. The rest of the prompt should change with the trigger and stakes.

Prompt variation 1: a supervised implementation loop#

Use this when the task is substantial enough to plan and review, but a human is still present and the agent does not need scheduled or background continuation.

Implement <TASK> in the current repository.
 
Acceptance contract
- Outcome: <REQUIRED BEHAVIOR OR ARTIFACT>
- Evidence: <TESTS, COMMANDS, SCREENSHOTS, OR EXPECTED OUTPUT>
- Constraints: <PUBLIC API, PERFORMANCE, SECURITY, OR SCOPE RULES>
- Boundaries: <ALLOWED FILES, PACKAGES, SERVICES, AND DATA>
- Authority: local edits and local verification only. Do not push, open a
  pull request, message anyone, merge, deploy, or mutate external systems.
 
Phase 1: inspect the repository and write a concrete plan. Name current
behavior, likely files and call paths, edge cases, test strategy, assumptions,
and risks. Write no code until the plan is coherent.
 
Phase 2: implement the smallest complete change. Preserve unrelated work.
Run the agreed verification and inspect the resulting diff.
 
Phase 3: review the final diff against the acceptance contract, not merely
against the plan. Classify findings as actionable, non-actionable with
rationale, or uncertain and requiring human judgment.
 
Success requires the evidence above to pass and every acceptance criterion
to be accounted for. If verification is unavailable, contradictory, or
requires expanding scope, stop and report the blocker instead of guessing.

This is the default I would reach for. It keeps the useful plan-implement-review rhythm without building a tiny autonomous company around a Tuesday bug fix.

Prompt variation 2: one orchestrator, disposable workers#

The whiteboard version becomes useful when the task is large enough that context isolation genuinely helps. The main thread is not another worker. It is the orchestrator and system of record.

The repeated "main thread" boxes in a workflow diagram should be understood as states of one durable orchestrator:

contract -> plan -> implement -> open PR -> review -> repair -> verify
                                              ^             |
                                              |-------------|

The planner, implementer, and reviewer should return structured results and terminate. The orchestrator keeps the acceptance contract, iteration count, evidence ledger, review dispositions, Git state, and final completion authority.

Implement <TASK> on a new branch and open a pull request.
 
You are the durable orchestrator. Retain the acceptance contract, phase,
iteration ledger, Git state, review dispositions, and completion authority.
Use role-scoped subagents only for assigned work, and end each worker after
it returns its result.
 
Before delegation, establish:
- Outcome: <WHAT MUST BE TRUE>
- Verification surfaces: <LOCAL TESTS, CI, ARTIFACTS, OR BEHAVIOR>
- Constraints: <WHAT MUST NOT REGRESS>
- Boundaries: <FILES, SERVICES, DATA, AND ENVIRONMENTS>
- External authority: <SPECIFICATION OR DOMAIN SOURCE OF TRUTH>
- Maximum repair iterations: <N>
- Time or cost limit: <LIMIT>
- Human approvals: <PUSH, PR, MESSAGE, MERGE, DEPLOY, OR DATA ACTIONS>
 
The orchestrator owns Git. Create one branch and one isolated worktree before
delegating. Allow only one editing worker in that worktree at a time.
 
Planning worker
Run read-only. Return current behavior, relevant files and call paths, proposed
implementation, edge cases, tests, assumptions, and scope risks. Write no code.
 
Implementation worker
Receive the accepted contract and plan. Edit only inside the approved worktree
and boundaries. Add tests, run local verification, and return changed files,
results, deviations, remaining uncertainty, and a proposed commit summary.
 
The orchestrator independently inspects the diff and evidence, then performs
authorized Git operations and opens the pull request.
 
Review worker
Run fresh and read-only. Receive the acceptance contract, authoritative rules,
plan, complete diff, and evidence, but not the implementer's debugging transcript.
Return findings with severity, location, concrete evidence, and the violated
requirement or risk. Do not edit files or declare the task complete.
 
Repair loop
The orchestrator classifies every finding. Repair only actionable findings.
For each iteration, record the failing evidence, choose the smallest supported
change, rerun relevant checks, inspect the diff, and push only when authorized.
Never retry an unchanged failing action.
 
Stop successfully only when required checks and verification surfaces pass
against the latest commit, no required check is pending, actionable findings
are resolved, other findings have dispositions, constraints remain satisfied,
and the pull request describes residual uncertainty.
 
Stop as blocked when the iteration budget is exhausted, two iterations make
no measurable progress, evidence is unavailable or contradictory, repair would
cross scope, reviewers conflict, or human authority is required.

Context isolation is not filesystem isolation. Subagents can have fresh model context while still sharing one checkout. The Git worktree documentation is the ground truth here: a repository can have multiple linked working trees, each associated with distinct worktree metadata and usually a distinct branch. The orchestrator should create that boundary before it delegates edits.

Review the contract, not the plan#

The planner can be wrong.

That sounds obvious until a review prompt says, "Check whether the implementation follows the plan." An agent can follow a flawed plan perfectly. A fresh reviewer then confirms the wrong thing with impressive consistency.

The hierarchy should be:

user requirement and acceptance contract
  -> authoritative domain and project rules
    -> actual diff and resulting behavior
      -> implementation plan

The plan is evidence of intent, not the highest authority.

Maker/checker separation still matters. A fresh reviewer is less likely to rationalize the implementer's choices. But two instances of the same model can share the same blind spot. A second model is a useful mechanical reviewer, not an independent source of domain truth.

Tests, types, lint, builds, schemas, browser behavior, specifications, and expected artifacts are stronger verification surfaces because they stand outside the implementer's narrative. For regulated or domain-heavy work, the authoritative handbook, policy, data contract, or approved example must be part of the review input.

Prompt variation 3: a Codex Goal for an uncertain path#

Use a Goal when you know what evidence would settle the task but do not know which investigation or repair will produce it.

A thin Goal says:

/goal Fix the flaky checkout test and keep going until CI is green.

A useful Goal says:

/goal Make the checkout integration test pass reliably for 30 consecutive
runs on the current branch, verified by the repository's documented test
command and an attached run log, while preserving public API behavior and
the existing timeout budget.
 
Work only in the checkout service, its fixtures, and related tests. Do not
weaken assertions, raise retries, extend timeouts, push, or change external
systems without approval.
 
After each attempt, record the hypothesis, change, and observed result. Choose
the next experiment from the latest evidence; do not repeat an unchanged failed
operation. Also run the broader correctness suite after any candidate fix.
 
Stop as blocked after six repair attempts, two attempts without measurable
progress, an unavailable or demonstrably flaky verification surface, or a
required change outside scope. Report the attempts, evidence, remaining
uncertainty, exact blocker, and smallest next input needed.

The key distinction is not slash-command syntax. It is epistemic honesty. A budget limit means the loop is out of budget, not that the bug is fixed. A benchmark failure means the result is unverified, not probably fine. An approximate reproduction should be labeled approximate.

Goals are strongest when the objective should persist and the next action should depend on the prior result. They are unnecessary for explanations, short reviews, or deterministic one-step edits.

Prompt variation 4: scheduled pull-request triage#

The published daily PR review workflow is a good demo of scheduled discovery, GitHub inspection, delegation, and human notification. Its prompt is intentionally lightweight. Production use needs more boundaries.

The biggest correction is to make the scheduled run read-only by default. Discovery can be broad. Repair should require a separate, auditable authority grant.

Every weekday at 09:00 America/New_York, inspect open pull requests in
<REPOSITORIES> that have had no human activity for at least 12 hours.
 
Selection rules
- Exclude drafts, dependency-bot PRs, explicitly paused work, and PRs already
  recorded as handled for the same head SHA.
- Inspect required checks, mergeability, approvals, unresolved review threads,
  labels, and the age of the latest commit.
 
Classify each selected PR as:
- ready for human approval;
- blocked by an actionable mechanical issue;
- blocked by a human or domain decision;
- waiting on checks or external state;
- no action needed.
 
Authority
- Read GitHub metadata and write the run ledger.
- Do not edit code, push, dismiss reviews, resolve human threads, merge, deploy,
  or send messages unless a separate rule explicitly authorizes that action.
- Deduplicate notifications by repository, PR number, head SHA, classification,
  and calendar day.
 
Evidence
- A PR is not ready while a required check is queued, pending, or running.
- Required checks must correspond to the latest relevant commit.
- Every review finding must be classified as actionable, non-actionable with
  rationale, or uncertain and requiring human judgment.
 
Output one concise report with links, evidence, and the smallest next owner
action. Produce a no-op result when nothing requires attention.

GitHub's status-check documentation distinguishes queued, pending, in-progress, completed, skipped, stale, failed, and timed-out states. That vocabulary matters. "Not red" does not mean "done," and a previous commit's green check is not evidence for a newer push.

If the organization wants automatic repairs, make that a second prompt with a narrower repository allowlist, an isolated worktree, a maximum number of concurrent repair tasks, an iteration budget, and explicit push authority. Do not hide write access inside a triage routine.

Prompt variation 5: skill mining without self-certification#

The self-improving skill workflow has a compelling shape: inspect recent work, notice a repeated procedure, draft a reusable skill, and validate it with a separate goal-based worker.

The dangerous word is "self-improving." A generated skill does not become trusted because the generator says it works. It is a candidate until it passes representative tasks without relying on hidden conversational context.

Review merged pull requests, resolved review threads, and repeated command
sequences from the last 30 days in <REPOSITORIES>.
 
Identify a skill candidate only when:
- the procedure appeared in at least three independent tasks;
- the inputs, outputs, trigger conditions, and non-goals can be stated clearly;
- the procedure is not better enforced by a script, test, hook, or permission;
- no secret, customer-specific fact, or temporary workaround would be encoded.
 
For each candidate, write a draft outside the active skill library containing:
- a precise trigger description;
- steps and authority boundaries;
- required references or scripts;
- expected evidence;
- failure and escalation behavior;
- representative validation cases.
 
Spawn a fresh validation worker in an isolated checkout based on the current
base branch. Do not provide the originating chat transcript. Test the draft on
at least one positive case, one near-miss that should not trigger, and one
failure case. Compare results with the documented acceptance criteria.
 
The validator may revise the candidate within three iterations but may not
install, publish, or promote it. Record every attempt and result.
 
Success produces a reviewed candidate and evidence bundle for human approval.
Failure produces the draft, cases, observed defects, and recommendation to
revise, replace with deterministic automation, or discard.

The promotion boundary is the whole point. Without it, the loop can turn one agent's accidental habit into permanent project guidance.

What “no unresolved comments” should mean#

An unresolved comment is a workflow state, not a correctness verdict.

Bots duplicate findings. Reviewers ask questions that are answered without code changes. Suggestions conflict. A stale comment can refer to a line that no longer exists. A security finding may require a human threat-model decision. Automatically resolving everything until the counter reaches zero creates the appearance of closure while discarding the reasoning.

Track dispositions instead:

- Actionable: requirement is valid, evidence supports it, repair is in scope.
- Non-actionable: finding is incorrect, obsolete, duplicate, or advisory;
  preserve the rationale.
- Uncertain: evidence conflicts or domain judgment is required; escalate.

The orchestrator owns this ledger. The reviewer raises findings. The implementer repairs accepted findings. Neither should unilaterally decide that the entire task is complete.

The repair loop must be able to lose#

The best loop prompt includes a dignified way to stop.

Suppose a test fails intermittently because its external sandbox is unavailable. The first repair changes application code. The test still fails. The second repair increases a timeout. The test still fails. The third retries more aggressively and finally turns green once.

That is not a successful loop. It is a loop that optimized against a broken verifier and made the product less honest along the way.

Blocked conditions should include:

• the same operation fails twice without new evidence;
• the verification surface is unavailable, flaky, or contradictory;
• a repair requires weakening the test or redefining success;
• the next change would exceed stated scope;
• external credentials or permissions are missing;
• reviewers give mutually exclusive requirements;
• domain correctness cannot be established from available authority;
• the iteration, time, cost, or concurrency budget is exhausted.

"Blocked" is not failure theater. It is a valid terminal state with a useful output: what was tried, what changed, what the evidence says, what remains uncertain, and what smallest decision or external change would unlock the work.

The prompt I would actually keep#

Most teams do not need all five variants pasted into every task. They need a durable template that can be shortened without losing the contract.

Achieve <OUTCOME>.
 
Prove completion with <VERIFICATION SURFACES> while preserving <CONSTRAINTS>.
Work only within <BOUNDARIES>. The allowed actions are <AUTHORITY>; ask before
any action outside them.
 
Use one durable orchestrator to own objective state, Git state, the evidence
ledger, review dispositions, iteration count, and completion authority.
Use disposable, role-scoped workers only where fresh context adds value.
 
Review against this contract and <AUTHORITATIVE SOURCES>, then against actual
behavior and the diff. Treat the implementation plan as subordinate evidence.
 
After each attempt, record the hypothesis or finding, action, result, and
changed evidence. Choose the smallest next action supported by that evidence.
Do not repeat an unchanged failure.
 
Success requires <EXPLICIT CONDITIONS> against the current state. Stop blocked
after <BUDGET> or when <BLOCKED CONDITIONS> occur. On either terminal state,
report the evidence, attempts, dispositions, remaining uncertainty, and next
human decision. Do not merge, deploy, notify, or mutate external systems unless
that exact action is explicitly authorized.

That prompt is less cinematic than "spin up a team of agents and keep going until green." Good. Production systems benefit from a little less cinema.

The engineer still owns the outer loop#

Loop engineering is not the end of prompt engineering. It is prompt engineering with memory, authority, and consequences.

The loop can isolate workers, preserve state, rerun checks, classify findings, and keep an objective alive. It cannot decide what your organization should value. It cannot turn a weak benchmark into a strong one by repeating it. It cannot make two identical blind spots independent. It cannot infer permission to merge or message a team because those actions would be convenient.

The human contribution moves outward:

• choose the work worth looping;
• define evidence before implementation;
• name constraints and authority boundaries;
• provide independent domain truth;
• decide which generated artifacts earn promotion;
• intervene when the verifier itself is suspect.

The best loop does not remove the engineer. It makes the engineer's judgment executable, inspectable, and difficult to quietly route around.

Build the loop. Give it a way to prove success. Give it a way to stop honestly. Then keep ownership of the part no prompt can automate: deciding whether the evidence deserves belief.