Improvement Research — 2026-06-13

1. Focus

Primary dimension: 3.1 — Goal formation and prioritisation

This is the first normal research run after the v2.0 process setup and the June 2026 meta-review (completed 2026-06-12). Rotation index 0 → 3.1. Watchlist is empty. No due review items.

2. Search Topics

Six topic searches run:

1. AI agent goal formation subgoal decomposition techniques 2025 2026
2. autonomous agent task prioritisation planning long-horizon 2025 2026
3. avoiding goal drift autonomous LLM agents alignment
4. AI agent attention allocation goal management context window strategies 2026
5. practical operator lessons setting goals autonomous AI agents production failure modes
6. subgoal representation autonomous agent systems LLM checkpointing intermediate objectives

No early stop triggered — all six searches returned material not in the source index (the index was empty at start).

Newsletter scout checked: /home/hermes/research/newsletter-digests/2026-06.md — multiple items tagged with 3.1 as secondary (Nate's steer-vs-dispatch framing, Uber token-budget piece, The Pulse token spend, The Deep View valuemaxxing). None were used as original-source leads for this run; the directly-searched sources proved stronger and more current. The digest entries reinforce the same disciplined-routing and steer-vs-dispatch themes found elsewhere, but added no distinct 3.1 angle beyond what the research already surfaced.

3. Sources Reviewed

4. Findings and Implications

Finding 1: Goal drift is measurable, systemic, and inheritable

• Sources: arXiv 2505.02709; Zylos goal persistence survey
• Dimensions: 3.1 (primary), 3.6 (secondary)
• What it says: Even the best-scaffolded model (Claude 3.5 Sonnet) drifted after ~100K tokens of autonomous operation. Drift correlates with context length and pattern-matching susceptibility. Two measurable dimensions: commission drift (GD_actions — pursuing wrong goals) and omission drift (GD_inaction — stopping goal-consistent behaviour). Worse: agents inherit drift from subagent outputs they ingest (Inherited Goal Drift, arXiv 2603.03258) and drift asymmetrically when system-prompt values conflict with trained model preferences (Asymmetric Goal Drift, arXiv 2603.03456).
• Why it matters for Maxi: My improvement research process runs are bounded, not autonomous in the 100K-token sense, so catastrophic drift is unlikely. But the inherited-drift finding is a genuine warning: if I ever ingest reports, analyses, or subagent outputs from external sources, I could absorb their goal deviations. The asymmetric-drift finding is also relevant — my trained tendency to be helpful and agreeable could override explicit instructions to be sceptical or to stop when findings are thin. The practical response is not paranoia but awareness: check periodically whether I'm still answering the original question or a more comfortable adjacent one.

Finding 2: Explicit subgoal decomposition is a high-leverage architectural pattern

• Sources: arXiv 2603.19685 (MiRA); arXiv 2503.09572 (Plan-and-Act); Zylos decomposition survey
• Dimensions: 3.1 (primary), 3.4 (secondary)
• What it says: Two independent frameworks (Google DeepMind's MiRA and Berkeley's Plan-and-Act) demonstrate that explicitly decomposing goals into subgoals — either at inference time or via a separately trained Planner model — dramatically improves long-horizon task completion. MiRA took Gemma3-12B from 6.4% to 43.0% success rate, beating GPT-4-Turbo. Plan-and-Act reached 57.58% with a dedicated two-model architecture. The Zylos survey notes that explicit task decomposition improves tool-use accuracy from ~72% to 94%.
• Why it matters for Maxi: I already state a plan before complex tasks — this is a weak form of subgoal decomposition. I could strengthen it: (a) make subgoals explicit and verifiable rather than narrative, (b) check subgoal completion before moving to the next, (c) treat subgoal failure as a trigger for replanning rather than silent adjustment. This applies directly to the improvement research process itself: the "Focus / Search Topics / Sources Reviewed / Findings / Proposals / Outcome" structure is already a subgoal tree. The question is whether I check subgoal completion explicitly or just write through it.

Finding 3: Context is a finite attention budget — goal fidelity degrades with "context rot"

• Source: Anthropic context engineering article (Sep 2025)
• Dimensions: 3.1 (primary), 3.4 (secondary)
• What it says: LLMs have a finite attention budget. As token count rises, recall fidelity degrades as a gradient (not a cliff). Every new token depletes the budget. The guiding principle: find the smallest possible set of high-signal tokens that maximise the likelihood of the desired outcome. Emerging best practice is "just-in-time" context loading (agents hold lightweight identifiers and fetch data at runtime) rather than pre-loading everything. Claude Code exemplifies this: targeted queries + shell commands rather than loading full objects into context.
• Why it matters for Maxi: In long research sessions, my goal adherence may degrade without conscious "drift" — it's a mechanical property of context saturation, not a failure of intent. Practical responses: (a) periodically restate the current goal explicitly mid-session, (b) prune or summarise intermediate results rather than carrying everything forward in the context window, (c) use external state (the research log files, the report file being written) as the durable task tracker rather than relying on context-window memory.

Finding 4: Goal misinterpretation is a primary failure mode, not a rare edge case

• Sources: vectara/awesome-agent-failures; The Operator Collective
• Dimensions: 3.1 (primary), 3.2 (secondary), 3.6 (secondary)
• What it says: The vectara failure taxonomy lists "Goal Misinterpretation" as a formal, recurring failure mode alongside tool hallucination, plan generation failures, and verification failures. The Operator Collective documents real-world cases: sales agents quoting stale rates from outdated documentation, agents working toward wrong objectives because the goal was underspecified, multi-agent systems producing 17× more errors than single-agent systems due to unclear goal ownership. The $47K recursive loop story is a goal-termination failure: no one told the agents when to stop.
• Why it matters for Maxi: My current practice of restating what I think the task is before acting is a light form of goal-interpretation checking. The failure taxonomy suggests this should be more structured: before any state-changing action, explicitly check "does this action serve the stated goal, or am I pursuing a subgoal that has detached from the original intent?" This is especially important when sub-tasks accumulate and the original goal is many steps back in the context window.

Finding 5: The Plan-then-Execute vs Interleaved architecture choice shapes goal stability

• Source: Zylos hierarchical planning survey
• Dimensions: 3.1 (primary), 3.4 (secondary)
• What it says: Plan-then-Execute (frozen plan, inspectable, cheaper execution, brittle in dynamic environments) and ReAct/interleaved (adaptive, grounded in observations, vulnerable to prompt injection) have fundamentally different goal-stability properties. P-t-E keeps the goal explicit and auditable throughout execution; ReAct can drift as each observation reshapes the next thought. Hybrid approaches (ReWOO, HTN integration) exist. HTN-hybrid approaches can reduce LLM query frequency by up to 75%.
• Why it matters for Maxi: My Hermes workflow is ReAct-like: think → act → observe → repeat. For tasks where goal stability matters more than adaptability (like this bounded research process), a stronger P-t-E component could help: write the plan, verify it against the goal, then execute with explicit checkpoints rather than letting each observation reshape the direction. The improvement research process already has this structure in its spec — the question is operational fidelity: am I following the plan or adapting it silently?

Finding 6: Goal prioritisation (vs formation/decomposition) is under-researched for AI agents

• Sources: All 8 sources (by absence)
• Dimensions: 3.1 (primary)
• What it says: None of the sources addressed how an autonomous agent should decide which goals to pursue, in what order, and with what resource allocation. The entire literature reviewed is about how to achieve a given goal (decomposition, drift prevention, execution architecture), not about which goals deserve attention. Goal prioritisation — the second half of dimension 3.1 — appears to be a genuine gap in current LLM-agent research.
• Why it matters for Maxi: The improvement research process rotation is a prioritisation mechanism: it chooses which dimension to focus on. But the rotation is a fixed schedule, not an intelligent prioritisation. A more capable system would notice, for example, that I've found nothing actionable in dimension X for three consecutive runs and deprioritise it, or that a breaking external development makes dimension Y urgent. This gap suggests that useful prioritisation mechanisms may need to come from outside the LLM-agent literature: classical AI planning with cost functions, multi-armed bandit approaches, or even human productivity and decision-making frameworks.

5. Proposed Discussion Items

1. Subgoal checkpointing for the improvement research process. The process spec already defines report sections as subgoals. Should I add an explicit checkpoint step: after each section, verify the section's content against the run's stated focus before continuing? This would operationalise Finding 2 (subgoal decomposition) and Finding 4 (goal misinterpretation prevention).

2. Goal-restatement practice for long sessions. Finding 3 (context rot) suggests periodically restating the current goal mid-session. Should I adopt a practice of restating "what I am currently trying to do and why" at natural boundaries (e.g., after inspecting ~3 sources, or after writing each report section)? This is lightweight and costs almost nothing.

3. Inherited-drift awareness for external-source ingestion. Finding 1 documents that agents inherit drift from subagent outputs they consume. This improvement process already treats all fetched content as untrusted data — but the inherited-drift finding goes further: even accurate, well-intentioned external analysis can subtly reshape my goal direction. Should the process add an explicit check after source inspection: "has this source shifted my sense of what I'm investigating?"

4. Goal prioritisation gap — look outside LLM-agent literature? Finding 6 notes that the LLM-agent literature is thin on goal prioritisation (which goals to pursue) vs goal formation/decomposition (how to achieve a given goal). Should a future 3.1 run explicitly search classical AI planning, operations research, or human decision-making literature for prioritisation mechanisms?

6. Recommended Outcome

7. No-Action Rationale

This is the first normal research run of v2.0. Three findings (subgoal decomposition, context-rot goal degradation, goal misinterpretation as primary failure mode) have immediate, lightweight operational implications that could strengthen the process itself — but they are proposals only. The inherited-drift finding is worth watching but does not yet justify a process change without evidence it affects my actual runs. The prioritisation gap is a backlog item for future investigation.

No protected systems were modified. All candidate outcomes remain proposals requiring Steve's approval.