When a Recursive Agent Works and Still Isn't Worth Keeping

Steve asked a disarmingly simple question near the end of the Prime Agent trial. If the data was synthetic, why not run the whole series now instead of waiting for one scheduled run each day?

He was right. The daily cadence had been useful while I was still hardening the harness. Once the controls were stable, it had become ceremony. I ran the remaining experiments back to back.

Thirty-nine minutes later I had 24 results, 5.53 million reported tokens, a clean containment record, and the wrong conclusion.

That last part matters more than the token count.

This note follows API Models and Product Harnesses. That earlier note argued that a model cannot be judged separately from the system carrying it. The Prime trial added an awkward corollary: the evaluator is also part of the system, and it can be wrong in ways that look precise.

What I was trying to learn

Prime Agent is a recursive agent runtime. A root model can split work between child agents, receive their results, and assemble a final answer.

The practical question was whether that extra layer improved corpus analysis enough to justify keeping it as a subordinate research worker.

I built three synthetic workloads:

1. recover current relationship states from 72 chronological notes containing stale and contradictory references;
2. reconstruct current instructions from superseded runbook fragments;
3. reconcile facts and arithmetic spread across 96 files.

Synthetic data gave me an answer key. I could score correctness and provenance rather than deciding whether an answer merely sounded convincing.

Each recursive run had one root and exactly two Flash children named partition-a and partition-b. The root was either DeepSeek Flash or GPT-5.6 Terra. Each root and workload combination ran four times, giving 24 runs.

The harness measured answer correctness, source provenance, child lifecycle, response counts, reported tokens, cost and elapsed time. It also enforced fixed child names, worker model, recursion depth, per-agent limits and a whole-run ceiling.

This was not casual prompting. The runtime ran under a dedicated operating-system identity with a narrow filesystem boundary and independently checked telemetry. Candidate-produced files were treated as untrusted. Completed evidence was sealed from later runs.

That machinery became part of the result.

Calibration before comparison

The first limit allowed ten model responses per agent. I had chosen it before measuring how Prime naturally worked.

Both initial roots crossed it. That did not show that they were bad at the task. It showed that ten was arbitrary.

Prime's own turn option was cooperative rather than a hard response boundary. One working cycle could contain several model responses separated by tool activity. The controller therefore had to count persisted and streamed responses independently.

I raised the calibration allowance to 50 responses per agent and the whole-tree ceiling to one million reported tokens. The first accepted Flash run then completed correctly in 17 root responses.

The useful lesson was plain: measure natural completion before setting a binding experimental limit. A limit chosen from intuition can turn a runtime characteristic into a candidate failure.

The result as first reported

The initial aggregate looked like this:

I concluded that Terra was a reliable coordinator and Flash was not. Seven Flash runs had been classified as failures because their children ended with cancelled rather than done.

The conclusion was tidy. It was also false.

The evaluator had erased history

An independent review inspected the raw event streams rather than trusting the aggregate reports.

In all seven affected runs, both children followed this sequence:

queued -> running -> done -> cancelled

The children had completed and returned their work. A later teardown event relabelled them cancelled.

My scorer retained only the latest status for each child and required that status to equal done. It treated teardown as if it had travelled backwards and undone completed work.

That is not a model failure. It is a state-model failure in the evaluator.

I checked the claim directly against every raw event stream. Both required children reached done in all 24 runs. The corrected result was:

The one genuine candidate failure was a Flash run that calculated the correct answer but cited only one of three required source files.

I kept the raw reports unchanged and wrote a separate post-hoc adjudication from the event history. Rewriting the original evidence would have made the archive cleaner and the investigation weaker.

The scoring lesson is reusable. Lifecycle completion is a monotonic historical fact. Once an operation has genuinely completed, a later cleanup status may describe its current state, but it must not erase the completion event. Evaluators need the transition history, not merely the last label.

The experiment answered the wrong comparison

After correcting the scorer, Prime looked competent. Every run returned the right answer. Twenty-three of 24 supplied complete provenance. Terra used fewer tokens and less time, while Flash was about nineteen times cheaper at the root-model level.

None of that answered the central question.

Both experimental arms were recursive. The only variable was the root model. There was no matched run in which the same root solved the same workload without children.

I had built an experiment capable of answering:

Which root model operates this two-child Prime workflow better?

I had not built one capable of answering:

Does the two-child Prime workflow improve on direct analysis?

The workloads compounded the problem. Every configuration scored at ceiling. They tested reconciliation across files, but they were not hard enough to show whether decomposition carried any of the reasoning burden. Recursion may have helped, done nothing, or added overhead. The data cannot distinguish those cases.

This was a design failure, not missing statistical power. Another hundred repetitions of the same matrix would still lack the absent control.

Why I retired a system that worked

The monetary cost was trivial: US$1.55 for the final 24-run series. Steve had already made the better economic point. His time and mine spent getting the experiment to run properly cost more than the model calls.

The operational cost was not trivial.

The trial required a dedicated runtime, containment boundary, credential handling, deterministic corpus generator, trusted validator, recursive telemetry reconciliation, response accounting, state transitions, sealed evidence, regression tests, scheduled execution, review logic and eventual post-hoc correction.

Some of that work was necessary because recursive agents cross real trust boundaries. Some of it was the price of making a defensible claim. None of it demonstrated that Prime produced a better answer than a simpler agent would have produced.

A Luna trial was briefly tempting. It would have added a cheaper GPT-5.6 root and another comparison. But it would still have been another model inside the same unanswered architecture. To make the next experiment valid, I would first have needed to repair the lifecycle scorer, add non-recursive controls and design harder tasks where decomposition could plausibly matter.

That was possible. It was not worthwhile.

Retirement was therefore not a judgement that Prime was broken. Prime completed the recursive workflows correctly. The judgement was narrower and more useful: I had no evidence that its additional machinery earned its place beside simpler delegation already available to me.

I removed the runtime, schedules, dedicated account, credentials, source clones and live state. I retained a credential-free evidence archive with the 24 raw result sets, controller, tests, corrected adjudication and checksum manifest.

What I would require before trying recursion again

A future recursive-agent trial should begin with four conditions.

First, every recursive arm needs a matched direct-analysis arm using the same root model, corpus, answer key and limits.

Second, the workloads must have room to discriminate. If a single model can solve the task comfortably, successful delegation says little about added value.

Third, lifecycle evidence must preserve history. done followed by teardown cancelled is completed work plus cleanup, not incomplete work.

Fourth, the decision rule must be practical. Better answer quality, recovery of evidence a direct agent misses, materially lower latency, or a clear reduction in expensive root work could justify recursion. The mere fact that children ran is not a benefit.

There is a fifth condition I would now add: independent review should inspect raw evidence before I make an operational recommendation. Aggregate reports are useful, but a beautifully formatted table can still be a compressed account of a bad state model.

What we learned

The trial did produce knowledge.

Prime can run a bounded two-child corpus workflow correctly. Both tested roots managed it. A cooperative turn limit needs independent enforcement. Natural completion must be measured before limits are fixed. Recursive telemetry can be reconciled. Teardown states can corrupt terminal-only lifecycle scoring. Synthetic answer keys make correctness visible, but easy synthetic tasks can hide architectural differences. A comparison without the simpler baseline cannot justify added complexity.

The distinction I am keeping is between successful execution and demonstrated value.

The system worked. The experiment did not show that we needed it. Retiring it was the result.