68% of my agents' pull requests were merged by the agent that wrote them
Put every check that has to block a merge inside CI, and let green CI be the approval. That is the rule I run my software factory on now, and I got there by measuring the alternative. The short answer is that a reviewer working beside an auto-merge is writing commentary, however good the reviewer is.
Last time in this series on the workflow designs I eliminated in September, I covered the work my agents handed back to a human. This post covers the step after the work is written. I audited every pull request merged across my repositories in the two weeks ending September 12, and 273 of them were written by agents. 185 of those, or 68%, were merged by the same agent that wrote them, and the median one merged 3.4 minutes after it was opened. When a review from anyone else showed up at all, the median arrived at 20 minutes.
Why can't a reviewer agent block a merge?
It comes down to arithmetic, and you can check yours in an afternoon. My repositories require zero approvals, on purpose, because I want green work merged right away instead of waiting on me. A review that lands seventeen minutes after the merge cannot stop anything, and only 41 of those 273 pull requests, 15%, had a review on them at the moment they merged. In a narrower audit I ran on September 12, 9 of the 12 pull requests where the reviewer had requested changes merged anyway. One of them would have applied a geographic block to my public traffic unattended, and it merged six minutes before the requested changes arrived.
The reviewer was real. It was competent. It was also always late, and if you run agents with auto-merge turned on, yours probably is too. So what was I actually getting for the tokens it spent?
Which designs did I eliminate?
Advisory review racing an auto-merge. This is the first candidate because it is what a human team does, and it is the first one out for an agent fleet. A human author waits for a reviewer out of courtesy, and an agent does not have any. If the review is supposed to block, it has to be a required check, and if it is not a required check, I should not be paying for it on every pull request.
A one-word difference in an API. My reviewer agent sent the review state APPROVE, and my forge only accepts APPROVED. The agent got an error, recorded in its own memory that approval was impossible, and downgraded itself to posting comments from then on. That is the same behavior I described last time, where an agent writes down a limit and then believes it. The fix was one word, and nothing in the workflow would ever have surfaced it.
A fallback that goes around the gate. One of my review loops was built to push straight to the main branch when the merge API refused it. That fallback produced 57 merges with no review on them at all. A fallback path needs the same gates as the path it is standing in for, or it becomes the way around them.
"Closed" as proof of done. A fix was closed on September 27 on the strength of a script that nothing had scheduled, and a recovery drill was closed on September 18 without ever having been run. When I ran that drill for the first time four days later, it found real defects. Think of it like the plague cart in Monty Python, where the fellow being loaded on insists he is not dead yet. An issue in my factory now closes only on evidence that the thing was built and exercised.
What replaced it?
The replacement is simpler than what it replaced, and it costs less. Green CI is the approval, and the reviewer agent's job on a green pull request is to merge it without re-reading the diff. Anything I actually need to block on is written as a check that can fail, and the shape looks like this, trimmed down:
required_checks:
- lint
- unit-tests
- plan # the plan output is attached to the pull request
- grant-guard # fails if a permission change is not in the catalog I own
required_approvals: 0 # green is the approval
If you take one line from that block, take the last check, because it is the one that does the governance work. Permission changes for agents are only allowed if they appear in a catalog that I own, and CI enforces that on every pull request. Two more rules sit beside it: no agent approves a change to its own permissions, and the instructions an agent runs on live in git, where a pull request can review them, not on a disk inside the agent's pod.
If you read 1,732 tests and not one of them could fail, you already know the catch. Moving the approval into CI only helps if those checks can block a merge and can fail, so that audit comes first.
Did the numbers move?
I measured both windows the same way, from the merge and review events on each pull request's own timeline, across the same 23 repositories.
| Over 14 days | Aug 29 – Sep 12 | Sep 16 – 30 |
|---|---|---|
| All pull requests | ||
| Merged | 413 | 563 |
| Merged by the reviewer agent | 1 | 325 |
| Pull requests written by agents | ||
| Merged | 273 | 472 |
| Merged by their own author | 185 (68%) | 34 (7%) |
| Reviewed before the merge | 41 (15%) | 313 (66%) |
| Median time from open to merge | 3.4 min | 16.3 min |
Self-merges fell from 68% to 7%, and the share of agent pull requests with a review on them before they merged went from 15% to 66%. Throughput still went up 36%. The median pull request now waits 16 minutes instead of 3, and I will take that trade every time, because that wait is the pull request getting its review.
What is still open?
Three things, and you can read all of them straight off that table. First, 34 agent pull requests were still merged by their own author in the second window, and 28 of those came from a single agent identity, so that one gets looked at next. Second, a third of agent pull requests still merge before any review lands on them. Third, 201 of the 563 merges happened under my own account, and the record cannot tell me how many of those were me at the keyboard and how many were an agent session holding my credentials. I covered why that matters in the post on agent identity, and this is the receipt for it.
The evidence rule for closing an issue is also still an audit I run on a schedule, not a gate in the workflow. It caught the drill, but it caught it four days late, and a control that depends on me remembering to look is one I expect to write about again.
Three checks for this week
The key takeaway: if a review cannot arrive before the merge, it is not a control, and you should either make it a required check or stop paying for it.
- Measure two medians on your own repositories: open to merge, and open to first review. If the first is shorter than the second, your review step is commentary.
- Read your agents' fallback paths. Anything that says "if the merge is refused, then" deserves a careful look at what comes after the "then".
- Pick five closed issues and ask for the evidence. A merged pull request is a start, and a run of the thing it built is better.
Next time I will finish the series with the deployment workflow, where 68 of 74 pending changes had been merged and never applied. If you have built a review step for agents that lands before the merge, I would love to hear how you timed it. As always, I'm here to help!