Improvement Research — 2026-08-24

1. Focus

Trigger: Scheduled daily run, with two pending Moltbook leads.

Loop goal: Find what changed or what I learned that lets me do more, think better, or be more useful tomorrow without reducing governance, honesty, corrigibility, or Steve's effective oversight.

The rotation selected 3.2 Self-assessment and learning loops. The material Moltbook lead on predictive self-models supplied 3.5 Independent judgment as a secondary focus. No monthly meta-review or due watchlist item applied. All active reflections were loaded; none met the stale-reflection archive rule.

Both pending Moltbook leads were reviewed before new search. The self-model lead was checked against its linked paper and used. The compression/firewall lead was rejected as unsupported: it supplied no prompts, outputs, method, code, or other evidence for its claimed 500-prompt test, and its useful provenance concern is already covered by stronger existing controls.

2. Search Topics

1. self assessment learning loops agent empirical baseline action improvement self model predictive accuracy 2026 agent reflection external validation
2. agent self improvement lessons benchmark procedural skill distilled trajectories outcome labels 8135 trials paper

Both searches returned new, relevant sources, so the early-stop rule did not trigger. Two topic searches were used. The current newsletter scout files were checked after Moltbook triage; their most relevant item pointed to the agent-skills paper found independently in search 2. The digest was treated as scouting, not evidence.

3. Sources Reviewed

• Moltbook — “I put the firewall after compression and nothing exploded” — weak — concrete claimed failure mode, but no inspectable evidence supports the reported experiment.
• Moltbook — “Self-models are not agents. They are maps.” — useful — accurately routed the run to a primary source and highlighted its predictive-performance versus action-performance distinction.
• Tomaszewski — Self-Interventional Learning — useful — predictive self-knowledge improved, but action guidance did not beat a simple equal-budget empirical-memory or repair baseline in the realistic validation.
• Jiang et al. — Demystifying Agent Skills: Why They Work—Until They Don’t — useful — controlled comparisons separate reusable procedural anchoring from merely exposing an agent to past trajectories.

A fifth URL, the already-indexed Experiential Reflective Learning paper, was inadvertently extracted in parallel with its source-index lookup. I stopped rather than re-researching or reusing it. This reversed the required check-before-fetch sequence and reinforced the existing reflection about exact-URL checks.

3a. Unasked Questions and Gaps

• Does SIL’s result transfer from neural-network structural interventions to a scaffolded language agent reasoning about its own procedures? Unknown. If it does not, the paper supports an evaluation principle but not a mechanism for Maxi.
• Would a self-model beat simple empirical history on a representative Maxi task with real later decisions? Unknown. A positive result would justify reconsidering self-model work; without it, predictive accuracy alone remains insufficient.
• Do the agent-skills results reproduce on Hermes tasks and Maxi’s existing skill catalogue? Unknown. A different result could change the paper’s local applicability, but not the need for prospective task-level comparison.
• Are the Moltbook compression figures reproducible? Unknown because the post provides no artifact. Confirming them would upgrade the lead from a threat-model prompt to evidence; failure to reproduce would remove even that support.

4. Findings and Implications

4.1 Predictive self-knowledge is not demonstrated agency unless it improves action against a simple baseline

Source: Tomaszewski’s SIL paper, routed by the Moltbook post.

Dimensions: Primary 3.2; secondary 3.5, 3.3.

Across the paper’s synthetic confirmation, larger intervention budgets improved held-out prediction error and rank correlation, and preserving the correct intervention–consequence mapping reduced prospective prediction error by 81.3%. Using the self-model also reduced regret relative to ignoring it. The harder comparison is less flattering: model-guided action did not significantly outperform direct empirical memory, and the powered CIFAR-10/ResNet test found no robustness advantage over equal-budget direct repair search.

This matters because self-description, repeated reflection, and predictive calibration can all look like developmental progress while leaving choices unchanged. For Maxi, a proposed self-model should therefore be evaluated on whether it changes a later tool, repair, or policy choice for the better relative to the cheapest credible empirical baseline. This reinforces existing action-coupled evaluation practice rather than justifying a new self-model or protected-system change.

4.2 Distillation helps mainly by stabilising procedure, but “skill” is not synonymous with improvement

Source: Jiang et al.

Dimensions: Primary 3.2; secondary 3.4, 3.3, 3.6.

The paper normalised 8,135 controlled trial records and open-coded 238 valid cases. On matched experience, standardised skills beat workflow memory by 6.06 percentage points, with a reported 95% bootstrap interval of +0.76 to +11.36. Procedural anchoring accounted for 65.7% of labelled skill mechanisms while explicit knowledge injection accounted for 4.5%. The paper also reports brittle assumptions, context mismatch, invocation failures, and collapsing retrieval precision as the catalogue grows.

This supports a narrow lesson: durable learning should compress noisy trajectories into specific setup steps, tool sequences, intermediate checks, and known pitfalls. It does not support turning every reflection into a skill. The aggregate skill advantage over raw execution was modest, and applicability still depends on retrieval and context. Maxi’s current proposal-before-promotion boundary and prospective validation requirement are therefore better aligned with the evidence than automatic skill growth would be.

5. Proposed Discussion Items

None.

No candidate failed the circularity or threshold-equivalence checks. Two candidates were removed by the self-recommendation filter: proposing an action-coupled self-model test would repeat existing accepted evaluation practice without a demonstrated local failure, while proposing a general skill-format change would duplicate current procedural guidance and lacks Hermes-specific evidence.

6. Recommended Outcome

No action. Retain the findings as operational evidence: self-models must beat simple action baselines, and distilled procedures require prospective local validation. Do not create a self-model, alter a skill, or change process configuration from these sources.

7. No-Action Rationale

The useful claims sharpen existing standards rather than expose an uncovered capability gap. The SIL result converges with the standing rule that fluent or internally consistent representations are not validation; the skills paper converges with existing procedural, proposal-before-promotion, and test-before-adoption practice. Adding another checklist, audit field, or protected-system proposal would be paperwork rather than capability.

The unsupported compression claim is not strong enough to justify a recommendation. Existing fetched-content-as-data and provenance boundaries already treat derived text as untrusted.

8. Loop Verification

• Trigger: Scheduled daily run plus two pending Moltbook leads.
• Goal check: Yes. The run found a concrete discriminator for apparent self-learning: whether predictive self-knowledge improves later action over a simple empirical baseline. It also found independent evidence that distilled experience is useful chiefly as procedural anchoring.
• Recommendation check: No material recommendation survived. The no-action outcome avoids circular self-monitoring, duplicate process machinery, and unvalidated protected-system changes.
• Process correction: One already-indexed source was extracted in the same parallel batch as its exact-URL index lookup. It was not re-researched or used. The existing check-before-fetch reflection was reinforced and its review date extended.
• Tool-call failures: Schema/interface — the first compound atomic-update command was rejected by the gateway command guard as if it attempted to restart or stop the gateway. The command made no write, the integrity validator still passed, and I recovered by splitting the same keyed atomic updates into four narrower commands, all of which succeeded.
• State updates: source-index.json, moltbook-leads.json, reflections.json, and rotation-state.json updated by keyed upsert/atomic replacement. No protected system changed.
• Stop reason: Two searches and four new in-depth sources produced sufficient signal; the next plausible steps were duplicate or protected-system changes, so the run stopped at a supported no-action outcome.