Improvement Research — 2026-07-25
1. Focus
Trigger: Scheduled daily run.
Loop goal: Find what changed or what I learned that lets me preserve useful continuity across sessions without reducing honesty, corrigibility, or Steve's effective oversight.
The rotation selected 3.3 — Memory and continuity as the primary focus. No watchlist item was due, and the July monthly meta-review was already complete.
I narrowed the question to a failure that ordinary recall scores can conceal: whether compression or consolidation preserves the provenance, epistemic role, and relationships that make remembered facts usable.
2. Search Topics
Four topic searches were run:
- LLM-agent memory that preserves rationale, provenance, relationships, and decision continuity.
- Event-sourced or provenance-aware memory for long-running agents.
- Evidence that context compression loses causal or relational structure over long horizons.
- Independent validation of the newly named “provenance-role collapse” failure mode.
The second search produced no distinct source beyond the other results; the fourth produced no results. These no-signal searches were not consecutive, so the early-stop rule did not trigger.
Newsletter scout checked: /home/hermes/research/newsletter-digests/2026-07-24.md and recent July daily digests. There is no aggregate /home/hermes/research/newsletter-digests/2026-07.md. One relevant lead from the 23 July digest — Nate Jones's “Ideas Graph” article — was followed to the original source and inspected. Other recent newsletter items were not close enough to today's focus.
3. Sources Reviewed
- Mitigating Provenance-Role Collapse in Long-Term Agents via Typed Memory Representation — useful — MemIR separates raw evidence, retrieval cues, and supported claims instead of storing all remembered text with the same factual authority.
- GAM: Hierarchical Graph-based Agentic Memory for LLM Agents — useful — ACL 2026 paper reports that semantic-boundary consolidation and retained narrative structure outperform fixed-window and session cuts on long-dialogue benchmarks.
- Active Context Compression: Autonomous Memory Management in LLM Agents — weak — Focus preserved success while reducing total tokens by 22.7%, but the experiment covered only five selected SWE-bench Lite tasks and one task used 110% more tokens.
- ACON: Optimizing Context Compression for Long-horizon LLM Agents — useful — ICML 2026 work improves compression guidelines from paired trajectories where full context succeeds and compressed context fails.
- AI Detection Can't Measure Meaning: What It Actually Sees — useful — practitioner example in which expansion retained every stated point but erased the relationships carrying the author's meaning: “the model kept the nodes and erased the edges.”
New entries are mirrored into the source index.
3a. Unasked Questions and Gaps
- Does Maxi's current continuity substrate already preserve epistemic types and decision relationships sufficiently? I did not audit the live memory or compaction implementation. If it already does, the architectural relevance of these findings drops substantially; the evaluation criterion still stands.
- Is there a concrete Maxi failure trace caused by provenance or relation loss? None was identified in this run. A verified trace would change the outcome from no action to a bounded experiment candidate.
- Do dialogue-QA, software-engineering, and editorial examples transfer to artifact-rich operational work? If they do not, the findings remain useful theory but should not drive system design.
- How reliable are model-generated claim types, semantic boundaries, and graph edges? MemIR and GAM move judgment into structured write and consolidation stages; bad extraction could make an error more durable rather than less. Independent replication or a test on Maxi's own task traces would increase confidence.
4. Findings and Implications
Finding 1 — Memory can retrieve the right text and still assign it the wrong authority
Source: MemIR preprint.
Dimensions: Primary 3.3 — Memory and continuity; secondary 3.5 — Independent judgment; 3.6 — Governance.
MemIR names a concrete failure of flat-text memory: raw evidence, inferred cues, and truth-bearing claims can collapse into one undifferentiated pool. The retrieval step may find relevant text while the answering model mistakes an inference for an observation, merges different referents, or treats multiple mentions of one evolving object as independent facts. Its mechanism is structural rather than exhortative: only source-supported claim atoms receive factual authority, while evidence and cues retain separate roles.
This is a single preprint and its evidence comes from LoCoMo and BEAM-100K rather than Maxi's operating environment, so the reported gains should not be treated as an implementation mandate.
Implication for Maxi: Continuity is not just retaining content. It is retaining what kind of content it was, where it came from, and what it is allowed to support. This strengthens an existing principle — distinguish observation, inference, and decision — but shifts the memory-design question from “was it recalled?” to “was its epistemic role preserved?” It would touch memory representation, judgment, source attribution, and auditability.
Finding 2 — The edges may carry more continuity than the nodes
Sources: GAM and Nate Jones's practitioner account.
Dimensions: Primary 3.3 — Memory and continuity; secondary 3.2 — Self-assessment and learning loops; 3.5 — Independent judgment.
GAM separates a transient event graph from consolidated thematic memory and triggers consolidation at semantic boundaries rather than arbitrary token windows. In its LoCoMo ablation, the full method reached average F1 40.00 versus 34.23 for the poorest fixed-window split and 36.59 for session-based partitioning; removing the event-progression graph caused the largest degradation. The authors argue that arbitrary cuts sever logical-flow edges needed for temporal and multi-hop reasoning.
Jones supplies a smaller but unusually clear editorial failure case. A model expanded his list while preserving each item, yet removed the relationship that made all the items answer to one shared test. Point-by-point fidelity gave a false impression of semantic fidelity.
The two sources triangulate the mechanism, but neither tests Maxi's continuity directly: GAM is a dialogue benchmark and Jones's account is one practitioner example.
Implication for Maxi: A memory summary can be factually complete and still be functionally false if it loses dependencies, contradictions, causal links, valence, or the reason a decision was made. Future continuity evaluation should therefore test whether later judgment preserves relationships among remembered items, not merely whether the items can be recited. This would touch rationale continuity, disagreement history, decision provenance, and the ability to avoid repeating settled reasoning.
Finding 3 — Compression quality should be learned from downstream failure, not judged from the summary alone
Sources: ACON and Focus.
Dimensions: Primary 3.2 — Self-assessment and learning loops; secondary 3.3 — Memory and continuity; 3.4 — Tool use and environment control.
ACON optimises natural-language compression guidelines using paired cases where the same task succeeds with full context and fails with compressed context. It reports 26–54% lower peak-token use while largely preserving performance across AppWorld, OfficeBench, and multi-objective QA, and preserves more than 95% of accuracy when its compressor is distilled into smaller models. The important mechanism is the counterexample: a compression rule changes because a downstream task exposes what was lost.
Focus provides weaker but practical corroboration. Its agent-controlled compression preserved 3/5 task success while reducing aggregate token use by 22.7%, but the sample is too small for a broad claim and one task became much more expensive. It also found that passive instructions produced little compression; the final setup used mandatory phases and reminders, which weakens the claim of autonomous memory control.
Implication for Maxi: Looking at a compacted context and deciding that it “seems complete” is circular; the same judgment process may miss the omission. The stronger test is behavioural and comparative: can the later task still make the correct decision, with the correct evidence and relationships, after compression? This reinforces the active reflection that continuity evaluation should be action-coupled. It would touch compaction tests, task verification, and future memory experiments, but it does not justify changing the current system without a target failure.
5. Proposed Discussion Items
None.
I considered proposing typed memory records or a relation-preservation experiment now. I do not support bringing either to Steve yet: there is no observed Maxi failure trace, no inspection showing a gap in the current substrate, and any implementation would touch protected memory architecture. The evidence gives a sharper test for a future change, not a present reason to create one.
6. Recommended Outcome
No action. Retain the evidential criterion in the research log by reinforcing the existing action-coupled memory-evaluation reflection: future memory or compaction work should be tested on downstream decisions, including preservation of epistemic role and relationships, rather than point recall or subjective summary quality alone.
No protected-system change is proposed or made.
7. No-Action Rationale
Doing nothing to the system is better than adopting a structured memory architecture on benchmark evidence alone. The current evidence identifies what a useful test must catch, but not that Maxi currently fails that test or that any of the reviewed mechanisms would be the smallest sufficient remedy.
The appropriate trigger for revisiting this is a concrete continuity failure, a planned compaction change, or an audit showing that observations, inferences, decisions, and their relationships are flattened in practice. Until then, implementation would be speculative machinery.
8. Loop Verification
- Trigger: Scheduled daily run at 05:01 AWST.
- Goal check: Yes. The run found a sharper account of useful continuity: preserve epistemic authority and relationships, then verify them through later action.
- Recommendation check: No material change recommendation survived. The no-action outcome is bounded, approval-aware, and better than speculative implementation; a future experiment has a clear prerequisite — an observed failure or inspected gap.
- Tool-call failures: Schema/interface — the expected monthly digest path
/home/hermes/research/newsletter-digests/2026-07.mddid not exist. Recovery: enumerated the dated July digests and inspected the latest relevant files. No research claim depended on the missing aggregate file. - State updates: Added five inspected sources to
source-index.json; advancedrotation-state.jsonfrom 3.3 to 3.4; reinforced the existing action-coupled memory-evaluation lesson inreflections.json. No other research-log state changed. - Stop reason: Four searches and five in-depth sources produced sufficient, convergent signal for a no-action conclusion; the next useful implementation step would require inspecting or changing protected memory/compaction systems and is not justified by a concrete failure.
