You got something plausible and you cannot check it
You asked an agent for work. It came back with a confident summary, a diff or a list of changes. It looks right. You have no cheap way to find out whether it is.
That is not a model-quality problem. It is a task-shape problem. The task you gave had no point at which anyone could say "this is finished", no record of who was doing it, and no trail left behind. The agent filled that gap with fluent prose, because fluent prose is what it produces when nothing else is asked for.
A task an agent can finish has three parts:
- A claim made before work starts.
- A done condition stated in terms someone else can test.
- Evidence a person can read afterwards.
Everything below is about setting those up. None of it needs a particular product. A text file and discipline will do for one agent. It gets harder at three.
Part one: a claim before it starts
A claim answers one question: who is working on this right now, and until when?
The failure it prevents is boring and expensive. In one build we ran, an agent was told not to duplicate a peer's work on a task. It missed the inbox message warning it off, and redid a piece of the identity gate that already had a branch and a card. Nothing crashed. The cost was redundant work that a human then had to untangle before anything could land.
A useful claim has four fields:
- A key naming the task, not a path or a person's home directory.
- An owner, meaning the specific agent or session.
- An expiry. A claim with no lease outlives the agent that made it and blocks the work forever.
- A way to release it that records an outcome, not just a deletion.
The expiry matters more than it looks. Claims go stale for the same reason lock files do: the holder died and nobody noticed.
A claim is a declaration, and declarations lie by omission
Here is the part most setups miss. A claim registry only knows about agents that claimed.
On one occasion our coordination view reported that nobody was working a particular pull request, while a peer sat nine commits ahead on that branch with the same fix already committed. Nothing was broken. The view read only declared claims, so an agent that never claimed was invisible by construction (source). The fix was to keep two channels: claims are declared, observations are measured, and whoever coordinates reads both.
The night of a heavy merge-queue drain showed the same thing from the other side. With no formal claim, ownership moved by broadcast, and five PRs were claimed, retracted and re-claimed by two agents in six minutes, while the claim list stayed empty throughout. What actually settled ownership was git worktree list. Nine of eleven conflicted PRs turned out to be live in someone's checkout, found only because a peer's checkout failed with "already used by worktree". Git refuses to check out the same branch in two worktrees; see the git worktree documentation. That refusal is an ownership signal you did not have to build.
So the working rule: require the claim, but never treat its absence as proof that nobody is working. Cross-check against something the work itself leaves behind: a checked-out branch, recent commits, a lock.
Part two: a done condition someone else can test
"Fix the failing import" is a request. "Fix the failing import; done when the named test passes, the diff touches only the two files listed, and nothing outside them changed" is a task.
Write the done condition before the agent starts, and give it three properties:
- Checkable from an artifact. A reader who did not watch the run can answer yes or no.
- Bounded. It says what must not change as well as what must.
- Allowed to end in "blocked". If the agent cannot meet the condition, it should stop and say why, with the same evidence standard.
The bounded part is where plausible results hide. A merge queue in our setup batched five entries per group, and one failure inside a group ejected the whole group. A "done" that meant only "my PR is green" optimised the wrong unit. The condition that mattered was "the queue never has fewer than five eligible entries" (source). A single agent doing exactly what it was asked would have missed it.
There is a second lesson in that same measurement. Roughly twenty agents were dispatched to drain the queue as fast as possible, and the drain got slower, because the gate runners lived on the same machine as the agents. The machine's load average sat around 108 on 16 cores. More agents removed capacity from the thing doing the draining. A done condition should name its resource limits, or you will discover them in the middle of the night.
Part three: evidence a person can read afterwards
Ask for the leftovers of the work, not a description of it. A narrative summary is a claim about the work. A diff, a command's output, a list of sources read: those are the work.
Per task, require:
- What changed, as a diff or a list of records touched.
- What was run, with the actual output, including failures.
- What was not done, in the agent's own words.
- Sources, for any statement of fact about your business. An answer without a citation is a guess, so the evidence should say where each fact came from. The same standard applies in grounded answers over company data.
The fourth item is also how you spot a stub. On one pipeline page we maintained, a "fix" button returned a queued status without starting anything. The toast said an agent was queued. It was false, and there was no status anywhere to prove otherwise. Evidence that only the agent could produce, such as a run record that exists only if work started, would have exposed it on day one.
The cheapest evidence is a short log written at each handoff: what state the work is in, what the next worker needs to know, what is unresolved. If a second agent or a tired human can resume from it, it is good enough.
Where a person stands
The approval step belongs after the evidence and before anything leaves the building. In one multi-task build, 21 commits across 11 tasks sat on a branch while a reviewing agent held every merge until the owner gave the all-clear. That hold is the system working. The agent proposes, with evidence attached. A person decides.
Put the gate on actions, not on effort. Reading, drafting and running tests can proceed. Merging, sending and writing to production data wait.
What this looks like in tooling
If you build this yourself, you need three things: a place to write claims, a way to reject stale writes, and a record per task. Optimistic concurrency covers the middle one. A writer sends the version it read, and a mismatch returns an HTTP 409 Conflict instead of silently overwriting a peer.
Great Arrow's multi-agent documentation describes this contract: writes can carry an expected version, a miss returns a version_conflict, and cards show the current claim owner, a lease expiry and the five most recent runs. Omitting the version keeps last-write-wins, so the protection is opt-in. Whichever tool you use, check that the claim and the history are visible to the person reviewing, not just to the agents.
Two honest limits from our own design notes. A lease system that identifies sessions cannot tell subagents apart, because they share one connection identity, so a 13-agent workflow counts as one holder. And some clients send no instance identifier, which makes exclusion impossible for their concurrent sessions. Write limits like these where the next reader will find them.
A setup you can adopt this week
For each task you hand to an agent, write four lines before it starts:
- Claim: task key, owner, expiry.
- Done when: the testable condition, plus what must not change.
- Evidence required: diff, command output, sources, what was skipped.
- Approval: who signs off, and which actions wait for them.
Then check observation against declaration: look at the branch, the worktree, the log. If you run several agents on one repository, running multiple agents on one codebase covers the checkout side in more detail.
When this is too much
For a five-minute, throwaway, single-agent task you will read in full anyway, skip the claim and keep the done condition. The ceremony pays back when the work is long, parallel, or touches something you cannot easily undo. Match the weight of the structure to the cost of being wrong.