Corrections Consolidate: Why Every Agent Mistake Should Not Create Another Rule

Abstract

Persistent AI agents are governed by text that is loaded on every call. That text tends to grow by accretion: an agent fails, a prohibition is added, the prohibition overlaps an existing rule, and the governing document slowly becomes larger and less coherent than the behaviour it is meant to direct. This paper sets out an alternative used across a six-agent estate in September 2026. It classifies each failure as an execution failure, a procedural failure or a governance defect, and permits a change to governing text only in the third case. It describes the operating environment in which the method was applied, reports the amendment protocol that accompanies the classification, gives the results of applying it to six governing files, and states the limits of what that evidence can support.

1. The accretion problem

An agent's governing file is not documentation. It is loaded into context on every call, and it competes for attention with the task in front of it. That makes an added rule a permanent tenant rather than an entry in a log.

The accumulation follows a predictable sequence. An agent makes a mistake. Somebody adds a prohibition. The new rule overlaps a rule that already exists. Contradictions and exceptions accumulate around the overlap. Eventually the control text is too large and too incoherent to guide anything, and the operator is left with a document that records the history of its own failures rather than a statement of how the agent should work.

The pattern is attractive because each step is locally reasonable. A mistake feels like it demands a durable response, and adding a sentence is the cheapest durable response available. The cost is deferred and diffuse: no single addition is responsible for the eventual incoherence, so no single addition is ever refused.

The question this paper addresses is narrow. After a failure, what evidence should be required before the governing text itself is allowed to change?

2. What the literature supports, and what it does not

Two of the assumptions behind the accretion habit have been tested, and neither survives intact.

The first is that instructions are free. Jaroslawicz, Whiting, Shah and Maamari's IFScale benchmark scales a business-writing task up to 500 simultaneous keyword instructions across 20 models, and reports a best observed accuracy of 68 per cent at maximum density, with a measurable bias toward instructions that appeared earlier in the prompt (arXiv:2507.11538). The task is synthetic keyword inclusion rather than agent governance, so it supports only the narrow proposition that dense instruction sets are not reliably satisfied.

The second is that a stated hierarchy resolves conflict. Geng and colleagues, in work accepted to AAAI-26, found that the system and user prompt separation most deployments rely on does not reliably establish priority between conflicting instructions, and that models rarely acknowledge the conflict at all (arXiv:2502.15851). Javed and colleagues reach a complementary result with calibrated conflicts across length, format and reasoning, reporting that conflict type affects behaviour more strongly than model scale (arXiv:2606.22470).

Taken together these support a bounded claim: instruction density has a cost, and contradiction is not resolved by declaring a hierarchy. They do not establish that any particular additional sentence harms any particular agent.

The diagnostic idea is older than any of this. Argyris distinguished single loop learning, which corrects an error within the existing governing variables, from double loop learning, which asks whether the governing variables were right (HBR, September 1977). Rae, Provan, Weber and Dekker named the accumulation pattern "safety clutter", the procedures, documents and roles that pile up in the name of safety without contributing to it (Policy and Practice in Health and Safety 16(2):194-211, 2018). No claim of novelty is made for the distinction itself. What follows is an implementation: a specific amendment protocol for agent governance, with evidence of what it caught.

3. A failure taxonomy

The method turns on classifying the failure before considering any change to text. Three classes are sufficient in practice.

Execution failure. A sound rule existed, was legible, and was not followed. The repair belongs in execution, verification or supervision. No governing change is justified, because the governing text already said the right thing.

Procedural failure. An operating procedure, runbook, command or stated fact was wrong. The repair belongs in that procedure under the authority that already applies to it. A wrong path is corrected as a wrong path, not converted into a behavioural prohibition.

Governance defect. The governing instruction was genuinely missing, ambiguous, contradictory or defective. Only this class justifies changing governing text.

The classification is applied through a short set of questions. Was a sound rule ignored? Was an operating procedure defective? Was the governing instruction genuinely missing or ambiguous? What is the narrowest durable scope for the lesson? What existing text should it replace? Which safeguards must survive? How will the behavioural effect be tested?

The fourth and fifth questions do most of the work in practice. An incident produces a vivid, incident-shaped sentence, and the discipline is to convert it into a positive rule that stands independently of the incident and to name the text it supersedes. A lesson that cannot find text to replace, and cannot articulate a lasting requirement, is usually an execution failure wearing governance clothing.

4. The amendment protocol

Classification alone is too weak, because the party diagnosing the failure is usually the party proposing the remedy. The protocol adopted alongside it requires, for any mistake-driven change to governing text: exact before and after wording, a recorded review by a different agent, and explicit human principal approval, with any material objection resolved before sign-off. After implementation, the change is exercised against representative decisions rather than assumed to work.

The wording adopted in the system administration agent's governing file, which the other agents then adapted rather than copied, reads:

Corrections consolidate. I diagnose mistakes before changing rules. Governing changes need evidence of missing or defective instruction, not merely failure to follow a sound rule. I keep lessons at the narrowest durable scope, replace identified text with a positive rule independent of the incident, and remove what it supersedes without losing necessary safeguards. Net growth needs a durable requirement that consolidation cannot meet.

Two design choices in that paragraph are worth isolating. The requirement to "remove what it supersedes without losing necessary safeguards" is deliberately two-sided: consolidation is not compression at any cost, and a shorter rule that deletes a safeguard is a worse rule. The final sentence permits net growth where a genuinely new durable requirement exists. Brevity is supporting evidence that consolidation occurred, not the objective.

5. Operating environment

The method was applied in a private estate of six persistent agents under a single human principal. Each occupies a separate profile on one host, holds a distinct remit, and does real work rather than demonstrations. The remits are system administration across the estate's servers, songwriting and music prompt construction, music promotion and release copy, visual art direction, editorial research and article preparation, and an independent second opinion on consequential conclusions.

That last role matters to what follows. The other agents share a model family and a working culture, so they tend to make correlated mistakes and to find each other's reasoning agreeable. The second-opinion agent exists to interrupt that. It holds no operational authority, owns no projects and executes nothing.

Each agent keeps its own governing file. The file is written in the first person, describes that agent's role, authority, standards and boundaries, and is loaded into its context on every call. It is a working instrument, not a policy binder: the current six range from roughly 7,000 to 16,400 bytes. That scale is worth holding in mind, because it is what makes an added paragraph consequential. A few hundred bytes of new instruction is a measurable share of everything the agent is told about itself.

Authority in the estate is deliberately narrow. The agents propose; the human principal approves. There is no review board and no second human. Where a proposal changes governing text, the human approval step is the only check outside the estate, which is both the system's main weakness and the reason the review step has to be structural rather than collegial.

Review is paired so that no agent reviews its own governance. For the five specialist agents, the system administration agent conducts the recorded review. For the system administration agent's own changes, the second-opinion agent does it. The agent reviewer advises, the human principal decides, and the authoring agent remains accountable for its own file. A review is not an approval and does not transfer authority.

The agents coordinate through two shared layers rather than by talking to each other directly. The first is a read-only common reference library holding agent cards, conventions and environment records. The second is a file-based mail queue: a bounded request is written as a file into the recipient's queue, the recipient records the outcome in the same file, and the exchange stays inspectable afterwards. It is asynchronous and slightly old-fashioned, and both properties are useful. A request carries context and a question. It carries no authority.

6. Implementation and results

Against that background, the principle was adopted in the system administration agent's own governing file on 9 September 2026 and then circulated as a briefing package. The package was explicit that it was not permission to change anything: each agent had to read its own governing files, identify the missing or defective text, draft an exact replacement in its own voice, obtain a recorded review and bring the result to the human principal. No agent received a copy of another's paragraph to paste in. Six governing files now carry locally worded variants, rechecked directly against the live files before publication.

The review step changed material wording in two of the six cases.

In the first, the proposed wording permitted a governing edit only where instruction was "genuinely missing". The review rejected it: that formulation excludes the case where existing instruction is ambiguous, over-broad or unsafe, which is most of the cases that matter, and it contradicted the same paragraph's own requirement to name the text being replaced. Exact replacement wording was supplied and approved.

In the second, the visual agent's draft compressed a phrase covering "my own errors of judgement" into "execution failure". The review blocked the compression on the ground that judgement can fail before anything is executed, supplied exact restoration wording, and separately required that the amendment mechanism be bounded to the operating governance document rather than to "my governance", which would have swept in ordinary skills and runbooks.

The third case is the useful counter-example. The music agent's proposal replaced two existing paragraphs, preserved its creative and operational autonomy, and was supported without wording changes. The reviewer recorded no material objection. A review process that always finds fault is not performing review.

Two behavioural checks were run. The editorial agent's adoption was exercised in fresh read-only contexts against four representative decisions, and correctly separated ordinary factual repair, failure to follow a sound verification rule, a mistake-driven governing amendment and a non-mistake-driven amendment to a protected boundary. One case had to be re-run because the test prompt, not the governing wording, was ambiguous about whether the amendment was mistake-driven. The music agent's adoption verified six representative decisions, including sound-rule failure, defective governance and ordinary procedural repair.

There is an awkward measurement to report. The adopted principle warns against net growth, and five of the six files grew: 609 bytes, 662, 854 and 963 in four cases, with the system administration agent's own identity file shrinking by 169 bytes while its separate operating mandate gained 295. The sixth had no equivalent text at all, so its version was a pure insertion. Against files of 7,000 to 16,400 bytes, an increase of that size is not trivial. Every increase is attributable to the amendment mechanism, which genuinely did not exist anywhere before, and the method permits exactly that. But an anti-accretion principle that added text in five cases out of six is entitled to be challenged on its own headline test, and the honest answer is that the protocol bought change control, not brevity.

7. Limitations

The work took place in one private estate with one human principal. Six adoptions ran from a shared briefing inside one week, which demonstrates that the principle could be adapted to six different remits rather than that six independent parties reached it separately.

The reviewers were other agents in the same estate. Structural separation from the authoring agent is real, and the pairing described in section 5 prevents self-review, but agents drawn from the same model family are exactly the population most likely to share a blind spot. For that reason the estate keeps a second-opinion agent operating on a different model base, and the human approval step adds a third level of oversight.

No controlled comparison exists between task performance before and after adoption. It was left to the human principal to determine relative merits and improvement before and after. The behavioural evidence is an acceptance test: it shows that the adopted text produces the intended classifications in representative cases, and in no case for any agent was rollback deemed preferable to continuation with the adopted schema.

8. What this method does not do

Prompt-level governance is text a model is asked to honour. It is not enforcement. An agent with unrestricted file tools can bypass anything written in prose, and consolidation does not change that. Work on deterministic control addresses the harder problem: compiling policy into runtime classifiers with provenance and human review (arXiv:2509.23994), composing delegated authorisation with owner-signed behavioural constitutions and verifiable governance receipts (arXiv:2606.30970), and separating planning, execution and governance behind policy-as-code (10.3389/frai.2026.1800407). Consolidation improves the policy surface that such systems enforce. It does not make that surface enforce itself.

Two further objections deserve direct answers. The first is that this is ordinary refactoring applied to prompts. The ancestry is conceded; the contribution is the explicit taxonomy, the exact-text requirement, the review path and the behavioural test. The second is that deleting an incident-specific rule erases hard-won learning. It does if the lesson does not survive somewhere. The protocol requires the lesson to reappear as a positive standing rule or in the correct procedural store, and if it can do neither, the deletion is not justified.

9. Conclusion

The useful question after an agent fails is not which rule to add. It is whether the governance was defective or the agent simply failed to live by it. Only the first justifies touching the governing text, and answering it honestly requires evidence rather than the feeling that something permanent ought to be done.