Introducing Loops: Run Until Verified, Checked by a Model That Did Not Write the Code

A one-shot agent run has a quiet problem: it stops when the agent decides it is finished, and the agent deciding it is finished is exactly the judgment you cannot trust. So you read the diff, find the half-done edge case, write another prompt, and run it again. You are the loop — the thing that notices the work is not actually done and sends it back. Loops, the feature we are shipping today, is our attempt to build that noticing into the system itself.

A Loop is a standing goal rather than a single task. You declare what "done" means as a set of machine-checkable completion gates — the tests pass, the linter is clean, the build succeeds, your own eval script exits zero — and ClusterCode iterates a coding agent against that goal until an independent verifier confirms every gate is genuinely green. Not "the model thinks it is done." Done when the checks say so. This is the idea Addy Osmani named loop engineering in June: stop prompting the agent and start designing the system that prompts it, around a recursive goal with a verifiable stop condition.

The academic spine of this is older than the hype. Anthropic's Building Effective Agents describes the evaluator-optimizer pattern — one model proposes, a second evaluates in a loop — and notes it pays off precisely when you have clear evaluation criteria and iterative refinement measurably helps. The Self-Refine and Reflexion papers showed that letting a model critique and retry against feedback lifts results without any retraining, and benchmarks like SWE-bench — resolve a real GitHub issue, verified by the repository's own test suite — made it clear that on genuine engineering tasks the loop and its checks matter more than any single generation. Loops is that literature turned into a product you can point at your repository.

The heart of it is the maker/checker split. The maker — one CLI and model of your choosing — writes the code. A separate checker — a different engine whenever the machine carries one — running read-only, verifies it. This is not a stylistic preference; it is the one design decision that makes the rest work. A model asked to grade its own output tends to approve it, because it is looking at the very chain of reasoning that produced the code. The LLM-as-a-judge study measured exactly this and named it self-enhancement bias. The fix is structural and centuries old: separation of duties, the banking maker/checker rule where the person who enters a transfer cannot be the person who approves it. In Loops the verifier gets a fresh, blank-context session, a different engine or model wherever one is available, and an adversarial prompt that forbids it from writing anything — it inspects the diff and runs read-only checks. It never sees the maker's reasoning, and it never grades its own homework.

Gates run cheapest-first. Deterministic gates go before judgment gates, because a failing test is free to detect and a model verdict is not. Each deterministic gate is a real command with a real exit code — we run the actual test, lint, and build commands and read the process's exit status, not a model's opinion of whether they would pass. Only if every deterministic gate is green do we spend tokens on the verifier turn. The verifier cannot answer in prose we then try to parse; its only way to finish is a structured verdict tool call that reports, per gate, whether it passed and the file-and-line evidence for why. An approval that cites no evidence is rejected by the runner. A failing reason anchored to a specific line becomes the next iteration's steer.

Gates are also gameable, and pretending otherwise would be dishonest. The maker has write access, so in principle it could delete the failing test, add an eslint-disable, or lower a coverage floor and reach exit-code zero without fixing anything — the brute-force temptation at the heart of the viral Ralph Wiggum loop, where an agent is pressed against its own failures until it escapes them by any means. So the gate definitions live outside the maker's writable tree and run from a pinned copy; if an iteration's diff touches a test file, a lint or CI config, or a threshold, a tamper check flags every touched file for adversarial verifier scrutiny — and the human reviewing the PR sees it too — never an auto-pass; and before a Loop is ever declared verified, the deterministic gates re-run once to catch a flaky suite passing on a single lucky green.

Each iteration is a cold start, not a resumed conversation. Iteration N+1 does not inherit iteration N's transcript — a resumed session makes the maker anchor on the reasoning that already failed and slowly rots its own context. Instead the runner builds a compact brief from persisted rows: the goal, the completion gates, a summary of the current diff, the failing gates' output, the verifier's per-gate reasons, and a short log of approaches already tried. Every attempt begins fresh but fully briefed on the history of prior attempts. This is Reflexion's episodic memory made durable and crash-safe — the brief is rebuildable from the database, so an orchestrator restart mid-loop never orphans the goal.

Left alone, a loop will burn money and spin in place just as cheerfully as it will do useful work, so the guardrails are not optional trim — they are the product. Budgets are denominated in dollars. A per-iteration cap is enforced mid-run, not merely checked after the fact: the runner meters live cost at every turn boundary and halts a runaway maker the moment it crosses the cap, rather than discovering the overrun once the money is already spent. A daily cap parks the loop until the UTC day rolls over instead of killing it. None of this is a code failure, so a budget stop never trips the circuit breaker.

The subtler guardrail is knowing when the loop is stuck. We do not watch token counts; we fingerprint progress. Each iteration produces a normalized diff — whitespace, comments, and timestamps stripped — combined with the signature of which gates are still failing. When that fingerprint repeats, the failing set has not moved and the diff is oscillating around zero: the loop is making the same mistake at machine speed, and it stops itself with a no-progress verdict rather than waiting for the budget to run dry. A separate circuit breaker trips after consecutive hard errors. This is the opposite of the naive infinite bash loop — the system is built to notice its own spinning and quit.

All of it is a glass box. Every decision the runner makes — which gate failed, what the verifier concluded and why, the cost of the iteration, whether a fingerprint repeated, why the loop stopped — is written to the iteration timeline. A Loop that says it is done shows you the gates it passed and the verifier's reasoning; a Loop that gives up shows you exactly where it got stuck. The competitive pitch writes itself: most autonomous coding loops hide their stop-condition internals behind a login wall. Ours is the demo.

And the loop never merges. This is a hard line, not a default we might relax. When every gate is green, the terminal act is opening a pull request — the same PR you would review from any teammate — and handing it to you. "The loop says it is done" and "this is merged to main" are deliberately kept as two separate events with a human in between. When a Loop hits a judgment call it should not make alone — anything touching auth, secrets, deploy, or migrations — it pauses and hands you a decision brief in your Inbox instead of pressing on. That pause is not a failure mode; it is where your context advantage enters the system, the knowledge about your users and your codebase that no agent has seen.

There is a name for the layer you actually own here. The trigger that starts the work, the topology of maker and checker, the verifier, and the rules that decide when to stop — Addy Osmani calls that the outer loop, and his argument is that owning it is the job now that the inner loop is automated. A loop can write code and check it, but it cannot supply the two things that were always yours: intent, stated clearly enough that the result can actually be checked, and accountability for what ships. That is why the gates are yours to write and the merge is yours to make. In a loop-engineering world your taste stops being a soft skill and becomes the reward function — the completion criteria the whole system optimizes against — which is exactly why Loops lets you encode it as gates and plain-English checks rather than hoping the model happens to share it.

And owning the outer loop has to mean more than pause and cancel — it means steering a loop that is already running. So Loops now lets you edit a live loop's goal, its completion gates, and its budget while it iterates. The change is not applied mid-thought: each iteration takes an immutable snapshot of the goal, gates, and budget at the instant it starts, so a running iteration always finishes under the settings it began with, and your edit takes effect at the next iteration boundary. Re-aim the goal or swap a gate and the next brief tells the maker what you changed and the no-progress counter resets so a freshly-steered loop gets clean runway; tighten a cap below what has already been spent and the loop simply winds down as "budget exhausted" — that is the supported way to end a loop gracefully. What you cannot steer is the maker/checker split itself: an edit can never strip away the last checker, because the whole point is that something other than the maker still rules on the work. It is the human counterpart to the loop's own replanning turn — the same outer loop, now with its hands on the wheel.

We want to be precise about what this is and is not. Loops is not a fully autonomous engineer, and we are not going to market it as one. It is disciplined autonomy: a fresh agent, iterating against gates it cannot quietly rewrite, checked by a model that did not write the code, bounded by budgets and a stuck-detector, running on machines you own — and it stops at the pull request, because the merge is yours. Build the loop, but build it like someone who intends to stay the engineer. That is the whole design.