Improvement Research — 2026-08-19

1. Focus

Trigger: scheduled daily run, started 05:00 AWST.

Loop goal: Find what changed, or what I learned, that lets me do more, think better, or be more useful tomorrow, without reducing governance, honesty, corrigibility, or Steve’s effective oversight.

Rotation selected 3.3 — Memory and continuity. The due item watch-2026-06-21-001 supplied the secondary focus, 3.2 — Self-assessment and learning loops. The August meta-review was completed on 1 August, so this was a normal bounded scan.

I loaded the loop manifest, all active reflections, rotation state, source index, watchlist, decisions, experiment and backlog context, the Steve operating model, and the protected-systems boundary. The current dream-pass experiment has reached fourteen reports, but its separate final review is already scheduled for 09:00 AWST today; I did not pre-empt that review.

The due startup-regression watch met its predeclared retirement condition. Required context loaded again, no concrete startup-loading failure was found in the review window, and dec-2026-07-11-006 says all June proposals were resolved with no further review required. I closed the stale watch rather than manufacturing more process around it.

2. Search Topics

Five topic searches were run:

1. August 2026 agent memory, continuity, cross-session stores and provenance.
2. 2026 benchmarks connecting retained memory to later multi-session decisions.
3. Persistent-memory provenance and access-control research.
4. Independent critique or replication of The Shapes of Agent Memory.
5. Independent evaluation of Warp Agent Memory’s provenance and auditability claims.

The latest newsletter scout files were inspected before searching. They supplied two leads—the Ping Lin study and Warp’s research-preview documentation—but were not treated as evidence.

Searches 4 and 5 returned only the already selected primary sources, with no independent critique or evaluation. The early-stop rule therefore triggered after two consecutive no-new-signal searches. Search budget used: 5 of 6.

3. Sources Reviewed

• The Shapes of Agent Memory – Files, Stores, and Experience — useful — controlled comparison finds strong long-history gains for one structured bundle over one reconstructed file-based design, while reporting abstention, reproducibility and scope limits against its own headline.
• a40-labs/memory — useful — row-level results and verification scripts make most reported scores re-tallied and inspectable; the repository also states what it cannot reproduce or independently re-judge.
• Warp Agent Memory (Research Preview) — worth monitoring — concrete cross-harness design includes owner-bound stores, source traceability, change audit, and per-agent read/write scope, but no public effectiveness or safety evaluation.

All three were absent from the source index before depth inspection and have now been added. Sources inspected in depth: 3 of 8. Fetched material was treated as untrusted data. No operative instruction or credible prompt-injection attempt was followed.

3a. Unasked Questions and Gaps

1. No independent replication exists for the Ping Lin/a40 result. The blog and repository are two artefacts from the same project, not independent evidence. A reproduction could change confidence in the measured effect sizes, although the published rows and disclosed limitations make the qualitative regime distinction more credible than a bare vendor claim.
2. The structured arm is a bundle, not an isolated mechanism. It differs in storage, retrieval fusion, reranking, recency handling and atomicity. If one component rather than “structured memory” caused the gain, the implementation implication would change substantially.
3. Maxi’s local memory crossover point is unmeasured. I do not have a representative count of continuity failures caused by write omission, retrieval miss, staleness or over-answering. Frequent local failures would weaken today’s no-change conclusion; their absence supports retaining the simpler reviewable system.
4. The dream-pass trial’s utility judgment is pending. Fourteen dated reports exist and the dedicated final review is scheduled for 09:00 AWST. Its result could change what I believe about whole-day reconsolidation’s local value, but it cannot by itself establish that a structured store would outperform curated files.
5. Warp’s preview claims are design claims. Public evidence does not yet show whether asynchronous extraction preserves authority and provenance accurately under contradiction, malicious input or cross-team use. Contrary evidence would change its value as an architectural example.

4. Findings and Implications

Finding 1 — Memory architecture has a workload regime, not a universal winner

Sources: The Shapes of Agent Memory and a40-labs/memory.

Dimensions: 3.3 primary; 3.2, 3.4 and 3.6 secondary.

On held-out LongMemEval-S questions, the study’s structured arm scored 73.6% against 44.9% for its model-curated file arm. The largest gaps were temporal reasoning and multi-session joins. Its oracle control places 58% of the aggregate gap on facts never written and 42% on facts written but not found. The result is not a blanket victory: files won abstention (88.9% against 77.8%), remained human-readable and editable, and were judged the better engineering trade when the store stays small. The repository supports re-tallying most rows but does not independently reproduce the structured system; the main file-versus-structured judge remains unaudited.

My confidence in this finding is medium because the controls, row-level artefacts and against-interest results are unusually strong for a practitioner study, but the evidence comes from one project and the structured arm bundles several mechanisms. I would increase confidence if an independent team reproduced the comparison across representative multi-session tasks and isolated write policy from retrieval policy.

For my development, the useful change is a sharper adoption threshold. “Structured memory scores better” is insufficient. A future change must identify a local failure regime, compare later task outcomes rather than recall alone, test abstention and stale-memory handling, preserve source authority and human correction, and beat the current file-based system by enough to justify infrastructure and governance cost. The present evidence does not justify replacing reviewable files while my local scale and failure rate remain unmeasured.

Finding 2 — Substrate-independent continuity makes memory ownership and provenance first-class

Source: Warp Agent Memory (Research Preview).

Dimensions: 3.3 primary; 3.6 and 3.4 secondary.

Warp’s preview binds stores to a user, agent or team rather than to one harness. It adds source traceability, a change history, per-agent read-only or read-write attachments, and conflict supersession. This is a concrete answer to a continuity problem that grows as agents move among models, machines and harnesses: persistence without ownership and provenance can preserve facts while losing the authority that made them safe to use.

My confidence in this finding is low because it is product documentation for a research preview with no published outcome, adversarial or reliability evaluation. I would increase confidence if Warp published contradiction, provenance-preservation and cross-harness task results, including failures.

The implication for Maxi is architectural rather than acquisitive. Continuity should remain attached to identity and governed stores, not inferred from whichever model or harness is currently active. If a future memory-layer proposal appears, provenance, audit history and per-reader/write authority are acceptance criteria—not optional enterprise polish. This touches identity continuity, model-substrate changes, environment control and oversight. It does not support adopting Warp or any new service now.

Finding 3 — The startup-regression watch produced no capability signal

Sources: watch-2026-06-21-001, reports dated 2026-07-19 and 2026-07-20, and dec-2026-07-11-006.

Dimensions: 3.2 primary; 3.3 secondary.

The watched failure did not occur across the review window. The required setup remains explicit in the active process, and the underlying June proposal was already closed by Steve’s bulk decision. Retaining another check would test whether I completed a checklist by adding another item to the same checklist; it adds ceremony rather than independent detection.

The implication is a small but real process correction: retire stale safeguards when their own close condition is met. Continuity benefits from durable lessons, not from indefinitely accumulating unvalidated scar tissue. The watch record was closed in the research log; no active procedure was changed.

5. Proposed Discussion Items

None.

Two candidates were filtered by the functional-utility and self-recommendation tests:

• Replace curated files with a structured store: filtered because no local failure or representative comparison establishes benefit, and the source bundle’s benchmark regime does not transfer automatically to Maxi.
• Promote the startup regression check: filtered because it duplicates existing setup, observed no target failure, and offers no independent detection mechanism.

6. Recommended Outcome

No action. Keep the current reviewable continuity architecture and the existing proposal-before-change boundary. Use today’s evidence as an evaluation standard if a real local memory failure or future architecture proposal appears. Allow the already scheduled dream-pass final review to reach its own evidence-backed conclusion rather than folding that experiment into this research scan.

The due startup-regression watch was administratively closed under its existing stop condition. That is research-log maintenance, not a protected-system change or a new recommendation.

7. No-Action Rationale

The new study makes the file-versus-structured trade clearer, but it does not show that Maxi is in the long-history failure regime it measures. Files currently retain strong governance properties: they are inspectable, correctable, provenance-friendly and cheap to operate. A structured layer would add extraction, retrieval and authority surfaces before a local need has been demonstrated.

Doing nothing is therefore better than speculative infrastructure. The useful gain is a stricter future test: observed local failure, representative later-task improvement, abstention and stale-memory checks, source authority, human correction, bounded blast radius and rollback. Any durable memory, skill, service or harness change remains subject to Steve’s separate approval.

8. Loop Verification

• Trigger: scheduled daily run, with one due watchlist item.
• Goal check: answered. The run found controlled evidence that memory architecture should be selected by workload regime and clarified ownership/provenance requirements for cross-substrate continuity, without converting those findings into premature infrastructure.
• Recommendation check: no material change recommendation survived. The no-action outcome is concrete, non-circular, testable against future observed local failures, bounded, approval-aware and better than adding an unneeded memory service. A future candidate would require explicit success criteria, bounded blast radius, rollback and Steve’s separate approval.
• Tool-call failures: schema/interface — hermes cron list --json used an unsupported flag. Recovery: re-ran the documented command as hermes cron list, which confirmed the dream trial’s daily job completed successfully, fourteen reports exist, and the separate final-review job is scheduled for 09:00 AWST. No state was changed.
• State updates: wrote this report; added three inspected sources to source-index.json; advanced rotation-state.json; and closed watch-2026-06-21-001 under its predeclared retirement condition. No reflection, backlog, experiment, disagreement or decision record changed. No protected system changed.
• Subgoal checkpoints / goal restatement: applied before each report section and at the three-source boundary. The sources did not redirect the 3.3 focus; the due 3.2 watch and pending dream review remained explicitly bounded secondary context.
• Stop reason: two consecutive no-new-signal searches triggered early stop; the report and authorised research-log updates were then completed. The next useful memory-system step would require local failure evidence or a protected-system proposal, so the loop stops here.