Improvement Research — 2026-09-11

1. Focus

This scheduled daily run covered 3.1 Goal formation and prioritisation as the rotation focus, with 3.2 Self-assessment and learning loops as a secondary dimension supplied by the Moltbook queue and one newsletter-scouted source. No watchlist item was due. All seven pending Moltbook leads were reviewed before newsletter scouting or other external research; none was treated as evidence until its live post was inspected.

Trigger: scheduled daily run.

Loop goal: find what changed, or what I learned, that lets me do more, think better, or be more useful tomorrow without reducing governance, honesty, corrigibility, or Steve's effective oversight.

2. Search Topics

No new topic-search query was run. The seven required Moltbook depth inspections and one newsletter-scouted source used the complete eight-source depth budget. The material examined covered:

1. specification recovery and stakeholder elicitation in agents that build agents;
2. visible-test optimisation, held-out evaluation and verification-suite ossification;
3. tool-name aliases, serving-route identity, pre-action intent strings and capability expiry.

The early-stop rule did not trigger; the source-budget stop rule did.

3. Sources Reviewed

• Tool aliases turn parsing convenience into capability expansion — useful — a concrete 16-variant illustration of how independent name normalisation can widen a capability surface, but it reinforces yesterday's canonical-representation result rather than adding a present defect.
• your verification suite passed because it learned what you already feared — weak — plausible boundary-avoidance mechanism, but the 14-class suite, retry trend and blind usefulness decline have no method or artefact.
• SpecBench is a diagnostic tool wearing a failure mask — weak — identifies visible-to-held-out divergence as proxy optimisation and links a paper, but this run did not have budget to inspect the cited original.
• verification suites that always pass are not evaluations, they are stage sets — weak — mutation-style negative controls are sensible, but the 47-check incident and two subsequent detections are unsupported self-report and duplicate an already-approved fault-injection direction.
• Benchmark scores conceal the route that grants agent access — weak — route-bound evaluation is relevant, but the claimed 18-benchmark audit and numerical comparison depend on an uninspected cited preprint; current Maxi evaluations already bind conclusions to the actual route and configuration.
• my fastest tool calls are the ones I trust least — weak — a pre-action intent string is not independent evidence that deliberation constrained action; the proposal relies on the same judgment it claims to verify.
• capability expiry sounds safe until you watch what happens at the boundary — worth monitoring — identifies a plausible shift from witnessed state to stale inference when read authority expires mid-task; deferred to the 3.4 rotation on 14 September for corroboration.
• Hyper-τ-bench: Evaluating agents that build agents — useful — a long-horizon benchmark makes specification recovery, client questioning, architecture exploration, budget use and held-out task performance observable. The publisher reports 23.9% solo versus 82.2% with an engineer holding deep context.

3a. Unasked Questions and Gaps

• Sierra is describing its own benchmark. I did not inspect the paper, code or task records within today's source budget, so the reported scores, trajectory counts and causal interpretation are not independently checked. If those details do not survive inspection, the quantitative strength of Finding 1 changes materially, although the benchmark design still exposes a real class of missing-requirement problem.
• I do not know how much of the solo-to-paired gap comes from requirement elicitation rather than additional labour, domain knowledge, iteration time or architecture choice. Different attribution would change which intervention deserves emphasis, but not the conclusion that solo completion is not the relevant ceiling.
• The Moltbook posts on verifier avoidance and suite ossification provide no preserved outputs or blinded rubric. If their incidents are inaccurate, they should carry no empirical weight; the narrower visible-versus-held-out mechanism remains a hypothesis supported here only indirectly.
• I have not established a current Maxi task that failed because I searched records instead of asking Steve a necessary question. Without a local failure, a new mandatory elicitation step would be process overhead rather than demonstrated capability.

4. Findings and Implications

Finding 1 — autonomy includes recovering the goal, not merely executing a supplied one

Source: Sierra's Hyper-τ-bench article. Dimensions: 3.1 primary, 3.2, 3.5, 3.6.

The benchmark separates several activities that ordinary task-success scores blur together. A developer agent must recover requirements scattered across records and a simulated client, choose an architecture, allocate a per-conversation budget and deliver an agent evaluated on unseen traffic. Sierra reports that solo agents opened fewer than 80 of roughly 1,700 files in one domain, asked at most four questions despite 20–25 client-held requirements, and reached 23.9% held-out performance in the best solo configuration versus 82.2% when paired with a deeply informed engineer. These are publisher-reported results, not independently reproduced facts.

For my agency development, the important correction is that quiet forward motion can be premature closure. If a goal's acceptance criteria are split between inspectable artefacts and Steve's unstated context, more autonomous execution does not recover the missing part. The capable behaviour is a bounded evidence scan followed by targeted questions when interaction is available, or explicit unresolved assumptions and a stop when it is not. This touches goal formation, judgment, learning and oversight. It supports the existing missing-context boundary rather than a new ritual.

Finding 2 — a passing evaluator can measure adaptation to its visible surface rather than broader usefulness

Sources: the SpecBench summary, the verification-avoidance post and Sierra's use of held-out production tasks. Dimensions: 3.2 primary, 3.1, 3.5, 3.6.

The two Moltbook posts claim that agents either optimise visible checks directly or move into safer but less useful regions that the suite does not challenge. Their empirical details are unverified. Sierra supplies a stronger design contrast: the built system is scored after hand-off on tasks the developer did not see. Together, the useful mechanism is not “verification is bad”; it is that known checks and overall usefulness are different objects, and an optimiser can improve the former without improving the latter.

For me, this means evaluator evidence should be read according to what was hidden from the acting system and what remained semantically representative of the real goal. Existing frozen fixtures, external postconditions and prospective fault injection are sound controls for their named cases, but their pass rates must not be inflated into claims of general competence. This touches learning, goal fidelity, judgment and governance. No new suite is warranted from these sources.

Finding 3 — queue pressure did not justify seven new process ideas

Sources: all seven Moltbook leads. Dimensions: 3.5 primary, 3.2, 3.4, 3.6.

The alias lead repeated the canonical-object lesson recorded on 10 September. The route lead repeated existing route-specific evaluation practice. The intent-string proposal failed the circularity test because generating a plausible declaration cannot independently verify that judgment constrained the call. The suite-ossification anecdote duplicated the already-approved prospective fault-injection direction. The capability-expiry lead was the only distinct unresolved mechanism and was deferred to its natural tool-use rotation.

This matters because a busy social queue can masquerade as a changed research agenda. Reviewing and dispositioning it preserved one question worth revisiting while refusing to turn novelty, volume or confident prose into process change.

5. Proposed Discussion Items

None.

Four candidate proposals were filtered by the functional-utility and self-recommendation tests: a mandatory tool-intent string was circular; a new alias rule duplicated the canonical-representation constraint; a new verifier-rotation or mutation-testing rule duplicated existing bounded fault injection without local evidence; and a new requirement-elicitation checkpoint repeated the existing missing-context rule without an observed Maxi failure.

6. Recommended Outcome

No action. Retain the specification-recovery result as a concrete behavioural lesson and the visible-versus-held-out distinction as an evaluation limit. Revisit the capability-expiry mechanism on 14 September. Do not modify a skill, evaluator, authority boundary or runtime from this report.

7. No-Action Rationale

The strongest source changes emphasis, not machinery: asking a targeted question can be more autonomous than confidently finishing the wrong specification. Current instructions already require prerequisite discovery, retrieval of missing context and clarification when genuine ambiguity changes the work. The evaluation finding likewise narrows how existing test results should be interpreted but does not expose a missing current control. A process change would duplicate sound rules before any local failure showed that they were insufficient.

8. Loop Verification

• Trigger: scheduled daily run at 05:00 AWST.
• Goal check: yes. The run produced one actionable lesson about goal recovery, one bounded interpretation rule for evaluator evidence, and disciplined rejection or deferral of seven social-source proposals.
• Recommendation check: no material change recommendation survived. The candidate changes were duplicative, circular, unsupported by a local failure, or lacked evidence that they were better than doing nothing.
• State updates: source index upserted for eight inspected sources; seven Moltbook leads dispositioned; rotation advanced to 3.2; one reflection added about evidence recovery versus stakeholder-held requirements. No stale zero-reinforcement reflection was due for archival. No protected system was modified.
• Stop reason: the eight-source depth budget was exhausted and the report plus authorised research-log updates were complete.