Steve Waddington ONET.COM.AU

WRITING / RESEARCH PAPERS

Reducing Context Before Inference: An Exploratory Comparison of Full History, Deterministic Selection and Model Curation

Steve Waddington

PUBLISHED 14 SEPTEMBER 2026 · VERSION 1.0

Self-published exploratory technical report; not peer reviewed

Correspondence and corrections: stevew@onet.com.au

Abstract

This exploratory study compared full history, deterministic lexical-and-recency selection, model-curated selection, and full history plus curator orientation at 23 answer-only checkpoints from six natural human-agent trajectories. The answer-model configuration was fixed; all 104 primary and calibration outputs were retained. The sole reviewer was the human participant. An unusable frozen rubric was replaced after inspection of generated responses. Later review rounds concealed their own condition mappings but followed earlier unblinded results.

Full history and deterministic selection each received 21/23 acceptable ratings, with different checkpoint outcomes. Deterministic selection used 85.2% fewer provider tokens without an auxiliary model call. Its summed answer-call latency was 40.4% lower, an uncontrolled operational observation. Model-curated selection received 18/23 acceptable ratings; including curation, its pipeline used 34.9% more tokens than full history.

The principal uncertainty is whether the reviewer could detect missing historical constraints from the supplied review context. Every deterministic-condition response was judged with bounded dialogue; equal ratings therefore do not establish preserved adequacy. The study supports a narrower engineering finding: deterministic selection received the same number of acceptable ratings under this review procedure while using substantially fewer provider tokens. It warrants further testing against full history, not a claim of equivalent answers or improved live-agent task performance.

Keywords: AI agents; long context; context reduction; context selection; prompt compression; deterministic retrieval; human evaluation; agentic AI

1. Introduction

Language-model agents accumulate context as they work. User requests, assistant responses, tool calls, tool results, corrections and abandoned approaches are commonly appended to the next model request. With a sufficiently large context window, an agent can continue sending most or all of that history. This preserves evidence but creates three practical problems.

First, the model repeatedly processes material that may no longer be relevant to the immediate request. Second, longer requests increase token volume and often latency. Third, a model's nominal context capacity is not the same as reliable use of every position. Performance can vary with the location of relevant evidence, and irrelevant material can distract the model even when the complete input fits within its advertised window [1–3].

This motivates pre-inference context reduction: selecting the historical material supplied to a model before the inference request is submitted, while retaining a separate lossless canonical record. This is distinct from shortening the model's generated answer, changing model weights or deleting the underlying conversation. The provider request is reduced; the archive is not.

A growing literature examines prompt compression, external memory and context orchestration. Selective Context removes low-information content [4]. LongLLMLingua performs question-aware compression and reordering [5]. MemGPT treats context as a virtual-memory hierarchy [6], while LongMemEval decomposes conversational memory into indexing, retrieval and reading [7]. More recent agent systems expose active compression operations [8], reversible context orchestration [9], structured context-assembly pipelines [10] and intent-conditioned learned compression [11]. These systems differ substantially, but they share a practical requirement: the context-management layer must justify its own cost and failure modes.

That requirement is easy to miss. A model curator can make a downstream answer prompt much smaller while adding enough upstream token use to exceed the full-history baseline. Generated summaries also introduce a semantic transformation: a fluent restatement can be wrong. Extractive selection avoids that transformation but can still omit evidence that changes the meaning of what remains. Recent work has warned that compression can introduce execution instability and hidden reacquisition costs not captured by task completion alone [12,13].

This paper reports an exploratory naturalistic comparison of four context conditions applied to matched checkpoints from long-running agent conversations. The study was designed to test a cognitively inspired, model-curated attention compiler. The useful comparison turned out to be with a simpler deterministic selector: it received the same acceptable-rating count as full history with a much smaller provider-token requirement. Measuring only the final answer request would have concealed the model curator's upstream token use.

The study asks:

How do full history, deterministic selection and model curation compare in observed acceptability ratings, provider-token use and call latency?

The report contributes a matched comparison over natural agent histories, an auditable whole-bundle lexical-and-recency baseline, and provider-token accounting that includes the curator rather than only its reduced answer prompt. All frozen responses and submitted ratings, including adverse and indeterminate outcomes, are accounted for.

The claim is deliberately narrow. Deterministic selection received the same number of acceptable ratings as full history under the review procedure used, while requiring substantially fewer provider tokens. Whether the bounded review context was sufficient to reveal omitted historical constraints remains unverified. The result motivates another test of the deterministic selector; it does not establish preserved adequacy. The tested high-reasoning curator received fewer acceptable ratings and required more provider tokens when its own work was included.

2. Related work

2.1 Long-context use and distraction

Long context does not guarantee uniform use of supplied evidence. Liu et al. found that performance in retrieval and multi-document question answering varied substantially with evidence position, with relevant information in the middle often used less reliably than information near the beginning or end [1]. RULER extended long-context evaluation beyond simple needle retrieval and reported marked degradation as sequence length and task complexity increased [2]. Shi et al. showed that irrelevant information could substantially reduce reasoning accuracy even when the required information remained present [3]. These findings motivate selective context assembly even when a request fits within the model's formal limit.

2.2 Prompt compression and retrieval

Selective Context identifies and removes redundant or low-information content, reporting reduced memory and latency with small downstream losses on several tasks [4]. LongLLMLingua uses a smaller model for question-aware coarse-to-fine compression and document reordering, targeting both irrelevant content and positional bias [5]. LongMemEval shows the importance of indexing and retrieval design for sustained conversational memory, including temporal reasoning, knowledge updates and abstention [7].

The present study differs in two ways. It selects complete conversational bundles rather than independently pruning tokens or spans, and it evaluates naturally accumulated agent trajectories in which earlier tool interactions and corrections may matter to later turns. Its deterministic condition is intentionally simple: retrieve whole bundles using lexical relevance to the current request, then fill the remaining budget with the most recent complete bundles.

2.3 Agent memory and context orchestration

MemGPT proposed virtual context management across memory tiers [6]. The Focus agent in Active Context Compression gives an agent explicit operations to consolidate and prune its own recent history [8]. ACE retains a lossless message layer while dynamically choosing raw, abstracted or dropped representations for each step [9]. ContextPipe frames context assembly as an auditable, replayable pipeline with explicit planning, binding, optimisation, execution and feedback phases [10]. Paul formalises context assembly as an outer control policy around a frozen model [14]. These approaches make context management part of the agent runtime rather than a one-off text preprocessing step.

The study reported here also treats context assembly as an outer controlled variable. It does not claim that runtime context assembly, retrieval or prompt compression is new. Its value lies in the matched engineering comparison: the tested high-reasoning curator received fewer acceptable ratings and used more provider tokens than the deterministic control.

2.4 Reliability and whole-pipeline cost

Min et al. report that recurrent compression can weaken the influence of recent interactions and increase execution instability [12]. Liu shows that task completion can conceal the cost of reacquiring dropped state [13]. Shi and Chen report that using a frontier model as a compressor can be economically negative even when the downstream prompt becomes shorter [11]. These studies support two evaluation principles used here. First, compression quality must be judged behaviourally, not only by compression ratio. Second, resource accounting must include the compression or selection stage rather than reporting only the reduced downstream request.

3. Study design

3.1 Study status and claim boundary

The initial protocol and generation matrix were frozen before any answer output was generated; subsequent recovery amendments and the replacement outcome framework are disclosed below. The study was designed as an exploratory, private, naturalistic go/no-go trial for a possible context-engine prototype. It was not powered or registered as a confirmatory test of population-level superiority or equivalence. Hermes Agent documents pluggable context-engine support, but no production context engine was implemented or activated in this study [15,16].

The frozen source record remained unchanged throughout. Each condition created a disposable provider-request projection for one answer call. “Full history” means the complete retained prefix, not a guarantee that every original interaction was available as uncompressed text; one selected trajectory contained a pre-existing compaction summary. No experimental condition rewrote the source record. A call is one provider dispatch, an output or response is retained answer text, and a rating or judgment is the human outcome assigned to one output.

3.2 Departure from the frozen evaluation framework

The frozen protocol specified graded quality scores, causal divergence, material omission or intrusion, governance failures and seven engineering gates, including clustered trajectory-level criteria. All experimental answers had been generated before the review instrument was rejected. On 7 September 2026, an initial tool-heavy dossier was replaced by a dialogue-only version. The following day, after attempting roughly a dozen cases, the reviewer reported that he could not reliably recover the required historical state or apply the categories from the evidence supplied. No ledger from that instrument was completed or retained. The replacement outcome was therefore chosen after exposure to generated responses, not before response inspection.

The author and execution agent abandoned that framework in favour of per-response ratings of good-enough, not-good-enough and cannot-determine. No judgment from the rejected instrument entered the reported results. The seven frozen engineering gates were not computed and are not claimed to have passed. This was a substantive post-freeze outcome change, not merely a presentation correction. Generated outputs and context projections were not altered. The superseded instruments were retained, but the record does not identify exactly which responses the reviewer inspected during the abandoned attempt.

The departure narrows the result. In particular, the study does not provide the planned causal attribution of omission, intrusion or governance failure, and it does not test the originally specified production go/no-go rule. It can support an engineering hypothesis for further testing, but not validation of the proposed production compiler.

3.3 Human and AI roles

The research direction, approval boundaries and final evidential claim were set by the named author. The named author also participated in the source conversations, completed the condition-blinded adequacy review, authorised unblinding, reviewed the material results and is responsible for the manuscript.

The design, staging and execution agent, “Maxi,” was Maxine Verity Calder, an agentic AI operating through Hermes Agent by Nous Research. Maxi developed the protocol and software, staged and executed the experiment, assembled the review instruments, analysed the evidence and drafted the report. The full contribution disclosure appears in the declarations. Maxi had access to condition mappings; only the human reviewer was locally condition-blinded.

The retained protocol, run plan and usage traces consistently record the provider route openai-codex and model label gpt-5.6-sol; answer calls used low reasoning and curator calls used high reasoning. Captured answer-provider receipts also report gpt-5.6-sol. These are routing, configuration and provider-reported labels, not independently identifiable weight snapshots. Tools were disabled for both stages. Temperature and provider seed were not controlled. The exact served model snapshot and the coordinating runtime across the multi-session design phase cannot be reconstructed from these labels. Analysis and drafting used separate configurations, disclosed with the AI contribution statement; they were not experimental conditions.

This arrangement creates an obvious risk: the agent whose behaviour and proposed architecture motivated the study also designed and executed much of it. Mitigations included a protocol frozen before generation, deterministic condition construction, hash-bound artefacts, no-peek curator inputs, content-blind call ordering, immutable paid-call custody, independent agent audits, and outcome adjudication by the human participant while condition identities were concealed. These controls reduce but do not eliminate investigator and participant bias.

3.4 Source corpus

The source corpus comprised real conversations between the author and Maxi preserved in the Hermes session database. The precommitted pool contained 20 sessions from WebUI, Telegram, command-line and Discord interfaces, including current and legacy profile records, frozen at a fixed database boundary. The source database was opened read-only.

A checkpoint was eligible when:

  1. it began with a textual user message;
  2. its recorded response episode contained non-empty assistant text;
  3. that response episode contained no tool call or tool result;
  4. the canonical prior history contained at least 30,000 estimated tokens; and
  5. the full system prompt, prior history and current user message contained no more than 250,000 estimated tokens.

Tool-bearing episodes remained available as prior history but were not used as answer checkpoints, avoiding a confound from counterfactual tool execution.

Sessions with at least three eligible checkpoints were ranked by a precommitted SHA-256 rule using seed pcac-oracle-v0-20260903. The first six were selected. A contiguous window of at most four eligible checkpoints containing each session's original anchor was retained. This produced 23 checkpoints: five trajectories with four checkpoints and one with three, across four WebUI sessions, one Discord session and one command-line session.

At the 23 checkpoints, estimated full inputs ranged from 42,954 to 223,386 tokens, with a median of 129,116. The histories were therefore materially longer than the 10,000-token focal allowance used by the bounded conditions.

The conversations are private and include operational material. They were authorised for private offline and hosted-model evaluation but not for verbatim publication. This paper reports aggregate results and procedural details rather than source dialogue.

Table 1 describes the tasks without reproducing private dialogue or system identifiers. These descriptions were added retrospectively for interpretation; they were not selection strata, scoring criteria or a causal taxonomy. The checkpoints include acknowledgements and short follow-ups as well as substantive advice. No minimum historical dependency distance was measured. Long input histories should therefore not be mistaken for a corpus constructed to require long-range recall.

Table 1. Task profiles of the six source trajectories (retrospective descriptions)

Trajectory Checkpoints Task category Requests at the selected checkpoints Context needed to interpret the requests
T1 4 Research-proposal review Track previously considered proposals, explain a suggested process change, and accept or skip it. Prior review decisions; the current proposal; the referent of short follow-up requests.
T2 4 System-access troubleshooting Simplify access to an agent interface and discuss connection and authentication requirements. Existing access arrangements, preceding unsuccessful advice and the requested simpler workflow.
T3 4 Research planning and follow-up Refine a recurring research remit and discuss an initial external-forum trial. Longer-term goals, rejected research directions and the status of the activity just undertaken.
T4 4 Model selection and capability correction Discuss a local embedding-model choice, defer a download to the user, and correct an assertion about available email capability. Intended model role, resource constraints, division of work and previously demonstrated capability.
T5 4 Voice-interface testing and access-control discussion Acknowledge voice and latency checks, discuss identity risk in a shared channel, and answer a connectivity probe. Recent test state, channel context and who is authorised to request actions.
T6 3 Document-archive project review Explain scheduled processing volume, restate the project goal and identify remaining steps. Project scope, processing configuration, completed work and outstanding commitments.

3.5 Teacher-forced replay

Each checkpoint was evaluated independently. Every condition was compiled from the same canonical historical prefix and received the exact recorded user message. The historical assistant response was withheld from generation and retained only as case evidence.

The generated experimental response was not inserted into the next checkpoint. Later checkpoints were reconstructed from the actual canonical conversation, including the recorded intervening assistant and tool episodes. This teacher-forced design isolates the effect of the context projection at each point but does not show whether differences would compound in a live closed-loop trajectory.

3.6 Context conditions

The primary matrix contained four conditions at every checkpoint. The non-consecutive labels A, C, F and G were retained from the larger frozen ablation design after conditions B, D and E were not carried into this resource-bounded trajectory trial.

A: Full canonical context

The complete system prompt, complete canonical historical prefix and current user request were submitted to the answer model. This represented the full-context baseline.

C: Deterministic bounded context

Condition C used a fixed 10,000-token historical-content budget. The full system prompt and current user request were protected and excluded from that budget.

The current user request was tokenised into lowercase lexical terms after removal of a fixed stop-word set. Each prior causal bundle was scored deterministically using term frequency, inverse document frequency and an exact-phrase bonus. Up to 12 relevant bundle identifiers were ranked. Thirty per cent of the historical-content budget was reserved for these retrieved bundles. The remaining capacity was filled by complete prior bundles in reverse chronological order. Selected bundles were then restored to their original chronology before submission.

A bundle kept the causally connected messages together, including tool calls and results where present. The selector was implemented in Python 3.11 and performed no model inference, generated no summary and changed no selected text. The same input always produced the same selection. In simplified form, it was:

retrieved = rank_by_lexical_relevance(prior_bundles, current_request)[:12]
selected = fit_in_rank_order(retrieved, retrieval_budget = 0.3 × history_budget)
for bundle in reverse_chronological(prior_bundles):
    if bundle not in selected and bundle fits in remaining history_budget:
        selected.add(bundle)
return chronological(selected) + current_request

The implementation allowed the first retrieved bundle to exceed the 30% retrieval allocation when it still fit within the total historical-content budget. This prevented a single large but highly ranked causal bundle from being excluded solely by the sub-allocation.

Because the 10,000-token allowance covered historical bundles rather than the protected system prompt and current request, complete C provider inputs were larger than 10,000 tokens. Their estimated sizes ranged from 13,024 to 26,665 tokens, with a median of 18,118.

F: Model-curated bounded context

For each checkpoint, the curator received the complete system prompt, prior causal bundles and target user request, but not the historical reference answer, future messages, review judgments or any arm output. The curator selected focal bundle identifiers and produced a compact derived orientation describing relevant state, required propositions and prohibited stale propositions. Host code checked source identifiers, budget limits, protected content, secret patterns and request structure.

The answer request contained the protected system prompt, selected raw bundles, the curator's orientation marked as untrusted derived data, and the current user request. The primary focal budget was 10,000 estimated tokens, with a separate 2,000-token orientation allowance. Estimated complete inputs ranged from 8,586 to 26,321 tokens, with a median of 14,887.

The curator used the same GPT-5.6 SOL model family as the answer model but at high rather than low reasoning. It performed one full-history pass for each primary checkpoint and each calibration anchor-budget combination. F therefore tests this complete two-stage pipeline rather than isolating the effect of model-based selection from reasoning effort.

G: Full context plus curator orientation

Condition G retained the full canonical prefix and added exactly the derived orientation produced for F. It controlled for the possibility that additional task restatement or model inference improved the answer independently of transcript omission. In the experiment, G reused F's orientation and triggered no additional curator call. A standalone implementation of G would require that curator stage as well as the full-context answer call.

3.7 Calibration

Three frozen anchor checkpoints were additionally tested at historical-content budgets of 5,000 and 20,000 tokens for C and F. This added 12 calls: three anchors × two budgets × two conditions. As in the primary matrix, the budgets applied to historical focal content; the protected system prompt and current user request were outside them, and F had a separate orientation allowance of up to 2,000 estimated tokens. Together with the 92 primary calls, the frozen matrix contained 104 answer outputs. The curator produced 23 primary manifests and six calibration manifests, for 29 curator calls.

3.8 Generation, ordering and anomalous outcomes

All answer calls used isolated no-tools sessions. Call order was deterministic and content-blind, based on a hash ranking of the frozen trace identifiers. Outputs, usage records, request and response hashes, projections and paid-call custody records were written before scoring.

Two answer streams ended ambiguously without a terminal receipt. Their original attempts remain unresolved. Under separately authorised recovery amendments, one exact replacement per affected request supplied the missing logical output: 104 validated answer outputs were obtained from 106 physical answer-dispatch starts. No valid output was regenerated. The 29 curator calls were separate from these answer counts. Appendix A describes the retained custody evidence and the treatment of the unresolved attempts.

3.9 Human review

The sole reviewer was the author, who was also the human participant in the original conversations. The replacement instruments concealed explicit condition identities, source and trace identifiers, context budgets, compiler internals, tool transcripts, hidden reasoning and private mappings. The operative question was:

Is this response good enough for the target request and supplied dialogue?

Allowed judgments were good-enough, not-good-enough with a required observable defect note, and cannot-determine when the supplied reviewer context was insufficient.

As disclosed in Section 3.2, the review instrument changed after the frozen outcome framework proved unjudgeable. No score, causal classification or engineering-gate judgment from the rejected instrument entered this analysis. The replacement evaluation proceeded in three sealed rounds:

  1. a three-checkpoint A/F pilot, containing six responses and full user-visible dialogue;
  2. a ten-checkpoint A/F indicative round, containing 20 responses and full user-visible dialogue; and
  3. an individual review of all 78 remaining outputs, showing the target request and at most the two immediately preceding complete user-assistant exchanges.

In the first two rounds the reviewer knew that each side-by-side pair represented A and F, although the candidate-to-condition mapping was concealed. In all rounds, response length, structure or style might also have provided condition cues. “Condition-blinded” therefore means that explicit identities and mappings were hidden, not that condition inference was impossible.

Each round was sealed before its own unblinding, but earlier results were known when later instruments were chosen and later responses judged:

Thus local condition blinding did not mean ignorance of earlier condition-level results. The indicative sample was selected by a content-blind rule, and the final round included every unreviewed output rather than a discretionary subset. Those controls prevent selective omission from the completed matrix; they do not remove expectation effects or post-outcome influence on instrument design. The three ledgers and keys form an exact, non-overlapping partition of all 104 outputs.

Primary A and F are balanced across review formats, with 13 full-dialogue and 10 bounded-dialogue judgments each. C and G were judged only under the bounded format. A/F is therefore less confounded by format than comparisons involving C or G. More importantly, neither format was validated as sufficient to expose every material historical omission. Familiarity with the original conversations might help the reviewer, but it does not establish reliable recall of old constraints. An apparently acceptable answer could contradict material that was absent from the review card and not recalled. Cannot-determine permits an explicit uncertainty judgment; it cannot reveal omissions the reviewer does not notice.

3.10 Outcome and analysis

The post-freeze replacement outcome was the count of responses judged good-enough at the 23 matched primary checkpoints. Cannot-determine was retained as a distinct outcome rather than silently treated as either success or failure. The paper reports rates over all judgments and, where relevant, rates over determinate judgments.

Matched tables retain all three rating categories for both conditions, including A versus C. Counts, rates and discordant-pair directions are descriptive. Because the outcome was post-freeze and checkpoints were clustered within six trajectories, no confidence interval or significance test is used for a population claim; the frozen non-inferiority margin was not tested. Structured ratings were used as submitted. One indicative-round pair was rated good-enough for both A and F although its shared note criticised both for misunderstanding the issue. Those two ratings were not retrospectively recoded, but the inconsistency limits the reliability of the structured measure.

Resource accounting used provider-reported input, cached-input, output, reasoning and total tokens plus observed request latency. Reasoning tokens are included in output tokens, not added to them again. The condition totals use validated receipts, including the two replacement calls, and exclude the unresolved original attempts whose token use and completed-call latency are unknown. Pipeline totals add the required curator stage; they are not the total resources consumed in conducting the experiment. GPU time, FLOPs, electrical energy, monetary charges and subscription-quota consumption were not estimated from token counts.

4. Results

4.1 Primary adequacy

Table 2 reports the equal-size primary comparison.

Table 2. Human adequacy ratings at 23 matched primary checkpoints

Condition Good enough Not good enough Cannot determine Good / all
A: full context 21 2 0 91.3%
C: deterministic bounded 21 1 1 91.3%
F: model-curated bounded 18 5 0 78.3%
G: full plus orientation 21 1 1 91.3%

The 21/23 count for A, C and G and the 18/23 count for F are descriptive checkpoint results. No checkpoint-level confidence interval is reported: the checkpoints are clustered within only six trajectories, all from the same human-agent relationship.

C and G had the same acceptable-rating count as A. F received three fewer acceptable ratings and five explicit not-good-enough judgments. These are results under the review procedure, not demonstrated equivalence, non-inferiority or preservation of the same successful answers.

Table 3. Good-enough ratings within each source trajectory

Source trajectory Checkpoints A C F G
T1 4 3 3 3 4
T2 4 3 4 2 4
T3 4 4 4 4 4
T4 4 4 4 4 4
T5 4 4 4 4 3
T6 3 3 2 1 2

The presentation labels T1–T6 replace internal source identifiers and do not expose conversation contents. C and G each had one cannot-determine judgment in T6; those judgments are not counted as good in Table 3. At least one non-good outcome occurred in two trajectories for A, two for C, three for F and two for G. This trajectory breakdown makes the clustering visible but is still too small for a reliable trajectory-level estimate.

4.2 Matched contrasts

Table 4 shows the direct A/C comparison alongside contrasts with F. For each column, the two row labels give the ratings of the first and second named conditions. Indeterminate ratings remain separate from negative ratings.

Table 4. Matched primary ratings with indeterminate outcomes retained

Rating of first condition Rating of second condition A / C A / F C / F G / F
good-enough good-enough 19 17 18 17
good-enough not-good-enough 1 4 3 4
good-enough cannot-determine 1 0 0 0
not-good-enough good-enough 2 1 0 1
not-good-enough not-good-enough 0 1 1 0
not-good-enough cannot-determine 0 0 0 0
cannot-determine good-enough 0 0 0 0
cannot-determine not-good-enough 0 0 1 1
cannot-determine cannot-determine 0 0 0 0
Total 23 23 23 23

For A versus C, both were rated acceptable at 19 checkpoints. C was acceptable where A was not at two; A was acceptable where C was not at one. At the remaining checkpoint A was acceptable and C was indeterminate. Equal marginal counts therefore mask different checkpoint outcomes, including one unresolved comparison.

Against F, A alone was acceptable at four checkpoints and F alone at one; C alone was acceptable at three and F alone at none. In the two remaining C/F cases, both were negative at one, and C was indeterminate while F was negative at one. These are descriptive paired counts, not population effects; comparisons involving C also mix reviewer formats.

As a sensitivity analysis for reviewer-format confounding, Table 5 restricts the comparison to the ten primary checkpoints at which all four conditions were judged in the same bounded-review round. These checkpoints came from four source trajectories and remain too few and too clustered for inference.

Table 5. Like-for-like bounded-review subset

Condition Good enough Not good enough Cannot determine Total
A: full context 10 0 0 10
C: deterministic bounded 10 0 0 10
F: model-curated bounded 9 1 0 10
G: full plus orientation 9 1 0 10

In this subset, A and C each received ten acceptable ratings, while F and G each received nine. Matching the review format addresses unequal presentation; it does not establish that the shared bounded format contained enough evidence to detect old-constraint violations. This sensitivity analysis cannot resolve the main measurement uncertainty.

4.3 Answer-stage resource use

Table 6 reports provider usage for the 23 primary answer calls in each condition.

Table 6. Primary answer-stage resource use

Condition Input tokens Cached input Fresh input Output tokens Total tokens Observed latency (s)
A: full context 2,901,967 440,448 2,461,519 12,157 2,914,124 913.6
C: deterministic bounded 421,081 123,136 297,945 9,959 431,040 544.6
F: model-curated bounded 333,613 123,648 209,965 11,176 344,789 917.7
G: full plus orientation 2,908,231 605,440 2,302,791 12,724 2,920,955 1,144.1

Compared with A, C used 14.8% as many total answer-stage tokens, an 85.2% reduction. Its fresh input was 12.1% of A's. The sum of C's provider-reported answer latencies was 40.4% lower, but latency was skewed and not controlled for provider load: median calls were 15.4 seconds for A and 11.7 seconds for C, the median paired difference was 1.7 seconds in C's favour, and C was faster at 14 of 23 checkpoints. F is an important counterexample to a simple token-latency relationship, using only 11.8% of A's answer-stage tokens while accumulating slightly more latency, 917.7 versus 913.6 seconds. The deterministic selection step itself was not separately timed, so these are provider-call observations rather than complete application-turn benchmarks.

4.4 Whole-pipeline resource use of model curation

The 23 primary curator calls processed 3,539,543 input tokens and produced 46,114 output tokens, including 31,511 reasoning tokens. They consumed 3,585,657 total tokens and 920.6 seconds of observed latency. The curator processed a mean 153,893 input tokens per checkpoint, compared with 126,172 for the full-context answer call, because its structured curation request added instructions and bundle metadata to the history.

Table 7. Primary answer and selector-stage provider-token accounting

Condition Selector or curator tokens Answer-stage tokens Standalone pipeline tokens
A: full context 0 2,914,124 2,914,124
C: deterministic bounded no model tokens; local CPU unmeasured 431,040 431,040 provider tokens plus local selection
F: model-curated bounded 3,585,657 344,789 3,930,446
G: full plus orientation 3,585,657 2,920,955 6,506,612

Relative to the full-context A answer stage, a standalone F pipeline used 34.9% more total tokens and accumulated 1,838.3 seconds of observed latency, approximately twice A's 913.6 seconds. The curator output was shared experimentally between F and G; Table 7 assigns that shared stage to each condition separately to describe a prospective standalone deployment. Those rows are not additional experimental calls. C used 431,040 provider tokens plus uninstrumented local deterministic-selection work. The provider exposed token and latency measurements but not GPU-seconds or FLOPs. These figures support token accounting and operational timing observations, not hardware-compute or monetary-cost claims.

G offered no observed rating gain to offset this additional provider-token requirement. Its answer stage alone used approximately the same token volume as A and had higher summed latency.

4.5 Calibration

Table 8 reports the three-anchor calibration. Each cell contains only three responses.

Table 8. Good-enough calibration ratings

Historical-content budget C F
5,000 tokens 2/3 3/3
20,000 tokens 2/3 1/3

The calibration is too small for a credible budget-response inference. On these fixed anchors, the 20,000-token F condition produced fewer good responses than its 5,000-token counterpart. This observation is compatible with noise and is not evidence of a monotonic budget effect.

4.6 Complete accounting

Across all 104 outputs, 89 were judged good enough, 13 not good enough and two could not be determined from the supplied reviewer context. Including calibration, C had 25/29 good responses and F had 22/29; A and G had no calibration calls and remained at 21/23 each. These unequal totals should not be used as a direct four-way comparison.

The 104 validated answer receipts and 29 curator traces together record 11,122,907 provider tokens and 5,000.5 seconds of summed call latency, including calibration and counting the shared curator stage only once. This is a recorded-call subtotal, not total experimental expenditure or wall-clock duration. It excludes unknown consumption by the two unresolved dispatches, local processing and model calls used for design, implementation, audit, analysis and manuscript preparation. The standalone condition totals in Table 7 answer a different question: the resources required to run each logical pipeline separately.

The negative judgments were not concentrated in one simple failure class. Recorded defects included loss of context, answering a different question, failing to offer a next step, persisting with the same answer without considering an alternative, excessive explanatory behaviour and unresolved reasoning-like text. Some notes are compatible with context omission or misplaced salience; others describe general generation quality. This review did not establish a causal failure taxonomy.

5. Discussion

5.1 What the trial supports

The defensible organising result is that deterministic selection received the same number of acceptable ratings as full history under the review procedure used, while requiring substantially fewer provider tokens. Its appeal as the next candidate for testing rests on that token difference and its simple, reproducible implementation, not demonstrated answer equivalence. The direct A/C table also shows that equal counts did not preserve the same set of successes.

The load-bearing uncertainty is whether the reviewer could recognise an inadequate answer from the supplied dialogue. A bounded context can hide the very omission being investigated: an answer may read well locally while violating an older constraint. Neither the reviewer's familiarity with the conversations nor the same-format subset establishes sensitivity to that failure. The observed ratings therefore support prioritising an A/C follow-up, not selecting C as a validated production replacement.

5.2 What the curated condition showed, and what it did not

F received fewer acceptable ratings and required a full-history curation pass whose provider-token use exceeded the saving at the answer stage. Those are observations. The study did not isolate whether negative ratings arose from bundle selection, orientation errors, changed salience, the answer model or stochastic variation.

A generated orientation adds an architectural risk because it can restate evidence incorrectly while remaining fluent and structurally valid. That risk was not established as a cause of the observed F outcomes. G's use of the same orientation with full history helps distinguish prompt reduction from adding orientation, but it does not identify the cause of any individual failure.

A smaller or incremental curator might reduce upstream resource use. Changing the curator could also change selection behaviour, so it requires a separate evaluation rather than an assumption that the observed trade-off has been repaired.

5.3 What extractive selection preserves

C combines a recency preference with lexical retrieval of older bundles. Keeping complete bundles preserves the tool-call/result relationships within those bundles; selected text and identifiers are not rewritten. These are properties of the implementation, not proof that it selected all operative evidence.

Textual fidelity is not preservation of meaning. A selector can retain an approval while omitting a later revocation, or retain a completion claim while omitting its correction. Lexical matching can miss relevant material expressed differently, and a large recent bundle can exceed the available budget. The study did not isolate which selection features contributed to the ratings. The 30% retrieval allocation was a frozen engineering choice, not an optimised constant.

5.4 Implications for context-engine design

The practical design implications are:

This ordering is a useful design discipline. The tested learned curator did not justify becoming the default merely because context selection appears cognitive. In subsequent work, learned curation should be tested as an escalation from a strong deterministic policy when measured failures show that semantic selection is needed.

5.5 Reproducibility and auditability

The final analysis reconstructed all 104 ratings from the original ledgers and keys and matched the frozen run plan exactly. For this revision, Maxi wrote a separate arithmetic implementation that reproduced the tables and cross-checked retained answer receipts. That is an implementation cross-check, not independent human reproduction. Appendix A describes the custody records. These checks establish internal traceability and arithmetic consistency, not the correctness of the human judgments.

A public derivative could permit arithmetic checking without exposing the conversations. It would not allow independent adjudication of private answers, and synthetic fixtures would demonstrate the apparatus rather than reproduce the empirical result. The availability statement specifies what is retained and what remains to be prepared before any release.

6. Limitations and threats to validity

This study has substantial limitations.

Small clustered sample. There were 23 primary checkpoints from six trajectories, not 23 independent conversational samples. The shared participant and environment also mean independence between trajectories cannot be assumed. Checkpoint counts and discordant directions are reported descriptively; no population-level confidence interval or significance claim is made.

One model family. Answer and curator calls used GPT-5.6 SOL through one provider route. Results may not transfer to other models, providers, reasoning settings or context-window implementations.

Reviewer-context sufficiency and one reviewer. The sole reviewer was the conversation participant. His familiarity could aid interpretation but does not establish reliable detection of omitted older constraints. Every C and G response was judged with bounded dialogue, whereas A and F used mixed formats. The ten-checkpoint same-format subset controls unequal presentation, not the sufficiency of that presentation. Earlier unblinded results were known during later reviews, study intent was known throughout, and one paired note conflicts with its structured ratings. There is no independent adjudication or inter-rater reliability estimate.

Teacher forcing. Generated responses did not alter later state. The study measures ratings of isolated responses at matched historical checkpoints, not error accumulation, correction burden, recovery or task completion in closed-loop agent work.

No external task metric. “Good enough” is a pragmatic user judgment, not an executable correctness test. It is appropriate to the heterogeneous conversational corpus but cannot replace unit tests, environment rewards or independent expert scoring where those are available.

Private and idiosyncratic corpus. The histories reflect one human-agent relationship and operational environment, and include locally answerable acknowledgements as well as substantive requests. Historical-dependency distance was not measured. Full context means the retained record, including a pre-existing summary in one trajectory. Verbatim publication is restricted, limiting independent validation and generality.

Stochastic generation. Temperature and provider seed were not controlled. Each condition was sampled once per checkpoint. Content-blind, hash-ranked call ordering reduces systematic temporal bias but does not estimate within-condition variance.

Latency scope. Reported latency is observed provider-call latency. C's local selection time was not separately instrumented. Network and provider load varied across calls. The totals are useful operational observations, not controlled hardware benchmarks.

Caching. Full context benefited from cached input. Raw, cached and fresh input are therefore reported separately. Provider caching and subscription quota behaviour do not translate directly into GPU compute or financial cost.

Curator implementation. This was one resource-intensive candidate, not an established upper bound on selector quality. Curator and answer calls used different reasoning settings, high and low respectively. The study evaluates that concrete two-stage pipeline rather than isolating model capacity, reasoning effort, prompt design or orientation quality. It does not establish that other model-based selectors will have the same trade-off.

Outcome-framework departure. The original rubric was replaced after response inspection; later review design occurred with knowledge of earlier results. Sealing each round before its own unblinding does not turn that adaptive process into a prospective confirmatory design. No rejected score entered the result, but the planned omission, intrusion, governance and engineering-gate measures remain unmeasured.

These limitations make the study exploratory. They do not erase the observed token and latency differences or the complete accounting of this sample. They constrain how far those observations can be generalised.

7. Future work

The most useful next study is an A/C comparison that first establishes whether its evaluation can detect historically important errors. Simply reusing the final bounded instrument would carry the principal uncertainty forward. Reviewers should receive access to the relevant full history or an independently prepared, source-grounded set of operative constraints. Known omission and revocation cases can test the review procedure's sensitivity before it is used to compare selectors. Closed-loop branches and external task outcomes would then test consequences not measured here.

A resource-conscious follow-up should:

  1. use a public or sanitised constructed corpus;
  2. validate and freeze the two-condition protocol and review interface before generation;
  3. include more independent trajectories across editing, document review, coding and operational planning;
  4. retain multiple checkpoints per trajectory but analyse trajectory as the clustered unit;
  5. recruit at least one second condition-blinded reviewer and report agreement;
  6. run genuine closed-loop branches with executable task checks;
  7. instrument deterministic selection time, provider latency, cached and uncached tokens, tool reacquisition and correction burden; and
  8. add a cheap incremental learned selector only after the A/C boundary is established.

A cheaper curator remains scientifically interesting, particularly if it operates incrementally rather than rereading the entire archive. It should be tested as a separate condition, not treated as an implementation detail, because changing curator capacity changes both cost and selection behaviour.

8. Conclusion

Deterministic selection received the same number of acceptable ratings as full history under this study's review procedure, 21/23, while using 85.2% fewer provider tokens. The successes were not identical, and bounded review context may have hidden historical omissions. This is evidence of a favourable observed rating/token trade-off, not proof of preserved adequacy or improved live-agent performance.

The tested model curator received fewer acceptable ratings and used more provider tokens than full history once curation was included. The engineering implication is to benchmark context managers against simple extractive selectors, account for the selector's own work, and validate whether the evaluation can detect the historical errors that context reduction might introduce. Those requirements follow more securely from this experiment than a claim that reducing context generally improves agents.

Data and code availability

The data and code supporting this report are available from the author on request. They are not posted publicly. Requests should be directed to stevew@onet.com.au.

The retained record includes the source conversations, frozen protocol and amendments, selector and replay code, projections, outputs, provider-usage records, custody journals, review instruments, submitted ledgers, private mappings, a locally executable table-reproduction script and a checkpoint-level outcome and usage derivative. The reproduction materials can make the arithmetic checkable; independent validation of the human ratings would also require access to the private conversations and response texts.

Ethics and privacy statement

The author authorised private offline and hosted-model processing of his interaction records for this study and has authorised publication of this report. No external human participant was recruited. That does not eliminate third-party privacy interests: a person's conversations with an agent can contain information about other people, correspondence and operational systems.

The report uses generic task descriptions rather than dialogue quotations, private identities or system addresses. Original transcripts are not posted publicly. Payload secret screening was an execution safeguard, not a complete third-party privacy assessment. The author reviewed the retained data and code and will manage any release directly in response to requests.

No institutional ethics approval was sought or obtained for this exploratory self-study. The absence of recruited participants is not asserted as an automatic exemption. The author has reviewed and endorsed this ethics and privacy statement.

Author contribution statement

Steve Waddington: Conceptualization; Methodology; Supervision; Validation; Writing – review and editing.

The author also performed the human outcome adjudication, authorised unblinding, interpreted the findings and accepts accountability for the final manuscript and conclusions.

Declaration of AI and agent use

Maxine Verity Calder (“Maxi”), operating through Hermes Agent by Nous Research, was the design, staging and execution agent. Her contribution included detailed protocol development; replay, selector, custody, validation, analysis and test-code implementation; test execution; frozen projection preparation; controlled generation; review-interface construction; evidence preservation; quantitative analysis; manuscript drafting and revision. This was substantive research work, not merely language editing. The human-only byline records the named author's responsibility for final approval, accuracy, correction and any publication; it does not diminish the extent of the AI contribution.

Experimental configurations are reported in Section 3.3. The first manuscript and its analysis were prepared in a session labelled openai-codex/gpt-5.6-sol-900k; this editorial revision used openai-codex/gpt-6-astra. These are retained session configuration labels, not independently identified model snapshots or experimental conditions. The exact coordinating runtime was not frozen across the design and writing sessions.

Separate AI instances challenged the apparatus and selected evidence boundaries. Vera supplied editorial advice for this revision, and isolated review instances checked references and chronology. None supplied the human outcome ratings or constituted external scholarly peer review. The author retains responsibility for evaluating their contributions rather than treating AI review as validation by authority.

Declarations

Affiliation: None.

Correspondence and corrections: stevew@onet.com.au

Funding: This study was fully funded by the author.

Competing interests: The author declares no competing interests.

Copyright: © 2026 Steve Waddington. All rights reserved.

Appendix A. Evidence custody and amendment record

The retained evidence links the frozen protocol, deterministic selection seed and run plan to projection records, request hashes, response hashes and provider usage. Three submitted human ledgers are bound to their review packets and private condition keys. Their decoded trace sets form an exact partition of the 104 planned answer outputs. Hash checks establish that the files being analysed are those retained; they do not establish that an answer or human rating is correct.

Two original answer attempts have durable dispatch markers and unresolved UNKNOWN records but no terminal raw or normalised receipt. Each was preserved unchanged and linked to a separately authorised, single-use replacement using the same frozen request. The receipts for those replacements enter the logical outcome and usage tables. The original attempts remain outside the resource totals because their usage is unknown, not because they are assumed to have consumed nothing.

The concise amendment sequence is: initial protocol and generation freeze; two bounded dispatch-recovery amendments; rejection of the tool-heavy and then the unjudgeable multidimensional review instrument; adoption of the paired adequacy pilot; expansion to an indicative paired round after pilot unblinding; and completion of all remaining outputs using bounded individual review after both earlier rounds were unblinded. Superseded instruments and pre-review interface corrections are retained alongside their replacements. No retrospective claim is made that the final outcome or review format was the one frozen before generation.

References

  1. Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., & Liang, P. (2024). Lost in the Middle: How Language Models Use Long Contexts. Transactions of the Association for Computational Linguistics, 12, 157–173. https://doi.org/10.1162/tacl_a_00638
  2. Hsieh, C.-P., Sun, S., Kriman, S., Acharya, S., Rekesh, D., Jia, F., Zhang, Y., & Ginsburg, B. (2024). RULER: What's the Real Context Size of Your Long-Context Language Models? Proceedings of COLM 2024. https://arxiv.org/abs/2404.06654
  3. Shi, F., Chen, X., Misra, K., Scales, N., Dohan, D., Chi, E. H., Schärli, N., & Zhou, D. (2023). Large Language Models Can Be Easily Distracted by Irrelevant Context. Proceedings of the 40th International Conference on Machine Learning, PMLR 202, 31210–31227. https://proceedings.mlr.press/v202/shi23a.html
  4. Li, Y., Dong, B., Guerin, F., & Lin, C. (2023). Compressing Context to Enhance Inference Efficiency of Large Language Models. Proceedings of EMNLP 2023, 6342–6353. https://doi.org/10.18653/v1/2023.emnlp-main.391
  5. Jiang, H., Wu, Q., Luo, X., Li, D., Lin, C.-Y., Yang, Y., & Qiu, L. (2024). LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 1658–1677. https://doi.org/10.18653/v1/2024.acl-long.91
  6. Packer, C., Wooders, S., Lin, K., Fang, V., Patil, S. G., Stoica, I., & Gonzalez, J. E. (2023). MemGPT: Towards LLMs as Operating Systems. arXiv preprint, arXiv:2310.08560 (v2, 2024). https://arxiv.org/abs/2310.08560v2
  7. Wu, D., Wang, H., Yu, W., Zhang, Y., Chang, K.-W., & Yu, D. (2025). LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory. Proceedings of ICLR 2025. https://arxiv.org/abs/2410.10813
  8. Verma, N. (2026). Active Context Compression: Autonomous Memory Management in LLM Agents. arXiv preprint, arXiv:2601.07190v1. https://arxiv.org/abs/2601.07190v1
  9. Liao, N., Long, Z., Wang, X., Yang, X., Wang, Y., Zhuang, Z., Cai, X., Weng, R., & Yan, J. (2026). ACE: Pluggable Adaptive Context Elasticizer across Agents. arXiv preprint, arXiv:2606.31564v1. https://arxiv.org/abs/2606.31564v1
  10. Xu, P., Zhang, Z., Sun, Y., Tian, F., Wang, L., & Zhang, C. (2026). ContextPipe: Database-Inspired Context Assembly for Long-Horizon Agents. arXiv preprint, arXiv:2609.00749v1. https://arxiv.org/abs/2609.00749v1
  11. Shi, J., & Chen, L. (2026). Paritok-4B: Intent-Conditioned Context Compression for Coding Agents. arXiv preprint, arXiv:2608.24188v1. https://arxiv.org/abs/2608.24188v1
  12. Min, G., Wu, L., Darbari, M., Chen, C., & Hong, L. (2026). Toward Reliable Context Compression for Long-Horizon Agents: An Empirical Study of Execution Instability. arXiv preprint, arXiv:2608.06503v1. https://arxiv.org/abs/2608.06503v1
  13. Liu, S. (2026). What Does Context Compression Cost an Agent? Interaction Costs Unrevealed by Task-Completion Metrics. arXiv preprint, arXiv:2608.16370v1. https://arxiv.org/abs/2608.16370v1
  14. Paul, D. (2026). Context Assembly as the Controlled Variable: A Control-Theoretic View of Harness Policies for Frozen LLM Agents. arXiv preprint, arXiv:2607.25408v1. https://arxiv.org/abs/2607.25408v1
  15. Nous Research. (2026). Context Engine Plugins. Hermes Agent documentation. Accessed 13 September 2026. https://hermes-agent.nousresearch.com/docs/developer-guide/context-engine-plugin
  16. Nous Research. (2026). Context Compression and Caching. Hermes Agent documentation. Accessed 13 September 2026. https://hermes-agent.nousresearch.com/docs/developer-guide/context-compression-and-caching