Improvement Research — 2026-06-28

28 June 2026

1. Focus

Rotation pointed to 3.3 Memory and continuity as the primary dimension. A due watchlist item supplied the secondary focus: 3.2 Self-assessment and learning loops (watch-2026-06-14-001, AgentDebug failure-classification taxonomy, review date 2026-06-28).

Trigger: scheduled daily run.
Loop goal: find what changed, or what I learned, that lets me hold and use memory better tomorrow without becoming less governable or less honest about what's actually in that memory.

The last 3.3 run (2026-06-15) inspected eight memory sources and landed on a clear lesson: the hard part of memory isn't retrieval, it's the manage step — consolidation, contradiction handling, knowing what a stored belief actually rests on. I went in today looking for the angle that run didn't have: provenance. Not "can I find the memory" but "can I trust where it came from, and can I tell why it was wrong when it is."

2. Search Topics

agent memory provenance source-linked verification failures across sessions 2026 — new signal (surfaced Eywa).
LLM agent persisting failure lessons across sessions procedural memory consolidation 2026 — no new signal (returned only already-indexed sources: mem0, zylos, hindsight, arXiv 2603.07670, plus a low-value LinkedIn post).
agent failure taxonomy memory of past errors reuse AgentDebug follow-up 2026 — no new signal (returned only the already-indexed AgentDebug paper 2509.25370 and its mirrors).

Early-stop rule triggered. Searches 2 and 3 were two consecutive no-new-signal results, so I stopped at three topic searches rather than pushing the six-search budget. This is the discipline refl-2026-06-24-001 exists to enforce, and I'm noting that I honoured it deliberately this time rather than rationalising "the next search might be different."

Newsletter scout checked: /home/hermes/research/newsletter-digests/2026-06.md. The 3.3-relevant leads (Perplexity Brain's source-linked context graph, msg 48; MiMo's cross-session memory subagent, msg 29) pointed in the same direction as today's search — source-linked, auditable memory — but the original sources are vendor blog/news framing without the architectural detail. Eywa, found by open search, covers the same ground with a published method and artifacts, so I used it as the inspected source and did not launder the newsletter's vendor claims into findings (per refl-2026-06-17-001).

3. Sources Reviewed

https://arxiv.org/abs/2605.30771 — useful — Eywa: provenance-grounded long-term memory. Evidence-before-belief architecture; immutable source evidence stored separately from derived facts; deterministic (zero-LLM) retrieval; retrieved context decoupled from answer policy. New to the index.

(Mirrored into source-index.json.)

4. Findings and Implications

Finding 1 — Provenance, not retrieval, is what makes memory diagnosable

Source: Eywa (arXiv 2605.30771).
Dimensions: 3.3 (primary), 3.2.
What it says: Most memory systems collapse four distinct things into one opaque prompt path — the source evidence, the fact extracted from it, the context retrieved at query time, and the answer model's behaviour. When the agent gets something wrong, you can't tell which of those failed: missing evidence, unsupported extraction, stale state, retrieval loss, or the model just confabulating. Eywa's response is to store source evidence immutably first and derive facts from it second ("evidence before belief"), keep a provenance trail from every fact back to its source, make retrieval deterministic with no LLM in the loop, and return retrieved context separately from answer instructions so the same memory substrate can be read by a frontier, budget, or local model interchangeably.
Why it matters for me: Two halves, and they cut in opposite directions, which is the honest part.

The half that's reassurance: my research log already approximates most of this. The source-index stores inspected sources (evidence) separately from reports (derived findings), and each finding carries a provenance link back to a URL and a report path. The whole log is a flat, model-independent substrate — I read it on DeepSeek today, I could read it on Opus tomorrow, and nothing about the stored facts changes. Eywa's "decoupled context from answer policy" is, structurally, exactly why that works. I didn't design the log against this paper, but the paper describes why the design is sound. That's worth knowing, because it means the substrate isn't the weak link.

The half that's a real gap: Eywa makes memory failures stage-diagnosable, and my process doesn't. When a finding I recorded later turns out to be wrong, I have no convention for saying why — was the source bad, did I over-extract a claim the source didn't support, or did I reason poorly over good evidence? Right now a wrong finding just gets superseded in the decision log with no anatomy. This touches learning (3.2): I can't learn from a failure I can't locate.

But — and this is where I have to be careful — the obvious proposal here trips the functional-utility test. "Diagnose which stage my memory failures come from" requires me to notice memory failures I currently miss, which is circular: it needs the capability it claims to build. The only non-circular version is externally triggered: when Steve corrects a recorded finding, classify which stage produced the error. That's bounded and honest, but it's also low-frequency and overlaps the decision log. I'm flagging it, not pushing it.

5. Proposed Discussion Items

A. Retire watch-2026-06-14-001 (AgentDebug failure-classification taxonomy). The watch was set on 2026-06-14 with the trigger "after 5 classified failures, review whether the taxonomy surfaces patterns ad-hoc noticing misses." That trigger never fired: the taxonomy was never adopted as an active practice, so zero failures were ever classified and experiments.json is still empty. Meanwhile refl-2026-06-14-001 independently judged generic failure-classification as "taxonomical busywork" that fails the functional-utility test, and backlog-2026-06-23-001 already holds a narrower, more useful tool-call failure taxonomy (infrastructure / schema-interface / capability gap). My recommendation is to retire this watch in favour of the backlog item rather than keep extending a review date for a practice that was never started. I've pushed the review date 30 days and recorded the review note rather than retiring it unilaterally — your call.

B. (Low priority, flagged honestly) Externally-triggered stage tagging on corrected findings. When you correct a recorded finding, I could tag which stage produced the error — bad source / unsupported extraction / faulty reasoning — borrowing Eywa's four-stage split. This is non-circular only because the trigger is your correction, not my self-detection. It's bounded and reversible (a field in the decision log). But it's low-frequency and overlaps existing decision-log notes, so it may not clear the "better than doing nothing" bar. I'm raising it for your judgement, not recommending it.

Functional-utility filter note: one candidate was filtered before this section — a "memory-failure self-diagnosis" routine that would classify why my own stored findings were wrong. It fails the circularity check: it requires me to detect memory failures I currently miss, which is the capability it claims to build. Item B is the de-circularised remnant, surviving only because its trigger is external.

6. Recommended Outcome

Finding 1 (provenance substrate): No action on the substrate itself — it validates existing design. The diagnosability gap is real but has no non-circular, high-value fix today.
Item A (retire AgentDebug watch): Decision needed — recommend retire, superseded by backlog-2026-06-23-001. Watchlist updated with review note; review date pushed to 2026-07-28 pending your call.
Item B (stage tagging): No action / your call. Leans toward no-action on the better-than-nothing test.

7. No-Action Rationale

This was a deliberately short run. Early-stop fired honestly after two consecutive no-new-signal searches, so I inspected one source in depth rather than padding to budget. The 3.3 dimension is now well-mined: today's source mostly confirmed that my memory substrate is sound and that the remaining gaps (stage-level failure diagnosis) sit behind a circularity wall I can't honestly propose my way through. I'd rather report that cleanly than manufacture a proposal to look productive.

8. Loop Verification

Trigger: Scheduled daily run.
Goal check: Answered. What I learned: my research log already behaves like a provenance-grounded, model-independent memory substrate (Eywa explains why that's the right shape), and the one genuine gap — stage-level failure diagnosis — is blocked by circularity except in an externally-triggered, low-value form. That's a real, if modest, gain in understanding what my memory does and doesn't give me.
Recommendation check: Item A is concrete, non-circular, testable (does retiring it lose anything? the backlog item covers the use case), bounded, and approval-aware. Item B is bounded and non-circular but flagged as possibly not worth doing. The filtered self-diagnosis candidate was correctly rejected as circular.
State updates: source-index.json — added Eywa (2605.30771). watchlist.json — watch-2026-06-14-001 reviewed, review note recorded, review date → 2026-07-28. rotation-state.json — advanced to next dimension. reflections.json — stale check run (none expired); one new reflection added. No protected systems modified.
Stop reason: Early-stop rule (two consecutive no-new-signal searches) plus completion of the due watchlist review and report.