feat(compact): recover context overflow (PR3 phase 2)

This commit is contained in:
Hypatia May
2026-07-15 16:50:44 +08:00
parent e8d066f750
commit 12484104c8
46 changed files with 913 additions and 242 deletions
@@ -16,13 +16,13 @@ Three properties carry the design:
- **Request-only, header-logged.** `deriveMessages()` never returns the prefix; its one durable record is `EpochHeader.messagePrefix` on the instance's anchoring `request/header` snapshot — the channel the reconstructable-requests RFC already owns for the request's non-history half, so no new session event exists. The dev invariant ([dsh-invariants](../../../../packages/support/invariants/src/index.ts)) recomputes `messagePrefix + boundary derivation` against every loop-built request; an unlogged prefix cannot reach the wire.
- **Frozen per instance.** Reuse is structural, not disciplined: the cached product cannot change mid-session, so the provider's prompt cache holds by construction and the prefix extends the cacheable region at zero marginal cost per step. A process restart or `ctx.agents.resume()` is a new instance: it recomposes, and any drift lands attributably on the `'resume'` header snapshot. This is the routing rule the seam creates: session-frozen openers ride the prefix; content that changes mid-session rides the append-only history channels (`agent.inject()`, a `tools/post-execute` decision's `additionalContext`, prompt-submit `additionalContext` — [the interception-seams RFC](2026-06-30-interception-seams.md)), each a durable `context/message` paid once and prefix-cached thereafter.
- **Composed before the pressure gate.** Composition precedes the instance's first `agent/pre-step`, and the seam hands the composed value through: `agent/pre-step` carries a `sessionPrefix` parameter and `CompactService.compactIfNeeded(agent, fullSystemPrompt, sessionPrefix, signal)` counts it in its token-pressure estimate — a gate reading the previous instance's folded prefix instead would under-gate a resumed or forked instance whose contributor grew, skipping compaction and shipping an over-window first request. A composition interrupted by a cancel/dispose landing inside the waterfall is discarded, never cached: an abort-aware listener's degraded fallback cannot leak into later requests, and the next turn recomposes under a live signal.
- **Exact in the durable request envelope.** Composition precedes the instance's first `agent/pre-step` and request boundary. The first routed request logs the current prefix on its header, so post-step token pressure reads the exact prefix together with the actual prompt, tools, and routed model; no compaction-only parameter is carried through the generic pre-step seam. A composition interrupted by cancel/dispose is discarded, never cached: an abort-aware listener's degraded fallback cannot leak into later requests, and the next turn recomposes under a live signal.
Because composition runs before the boundary snapshot, a composing listener's session append joins the CURRENT request's derived history. Compaction structurally cannot touch the prefix (or the system prompt): it rewrites surface nodes, and header state never enters the surface.
## Testing
[Interception tests](../../../../packages/core/agent-loop/tests/interception.spec.ts) pin compose-once reuse with no header deltas, prepend order, empty-prefix omission, immutability, and composition before pre-step; [cancellation tests](../../../../packages/core/agent-loop/tests/cancel.spec.ts) pin discard and recomposition. Session codec, invariant, and compaction tests cover header round trips, request reconstruction, and prefix-aware pressure accounting. Snapshot normalization preserves prefix counts, while the [pinned-header scenario](../testing/2026-07-06-pin-request-header-content-in-one-scenario.md) owns content and the default example remains prefix-free. No prefix-specific e2e is needed because the seam is deterministic and provider-independent; the with-key [request-cache e2e](../../../../packages/core/agent-loop/tests/request-cache.e2e.ts) covers its cache economics.
[Interception tests](../../../../packages/core/agent-loop/tests/interception.spec.ts) pin compose-once reuse with no header deltas, prepend order, empty-prefix omission, immutability, composition before pre-step, and the prefix on the routed header; [cancellation tests](../../../../packages/core/agent-loop/tests/cancel.spec.ts) pin discard and recomposition. Session codec, invariant, token-meter, and compaction tests cover header round trips, request reconstruction, and durable prefix-aware pressure accounting. Snapshot normalization preserves prefix counts, while the [pinned-header scenario](../testing/2026-07-06-pin-request-header-content-in-one-scenario.md) owns content and the default example remains prefix-free. No prefix-specific e2e is needed because the seam is deterministic and provider-independent; the with-key [request-cache e2e](../../../../packages/core/agent-loop/tests/request-cache.e2e.ts) covers its cache economics.
## Alternatives considered
@@ -30,12 +30,12 @@ Because composition runs before the boundary snapshot, a composing listener's se
- **A system-prompt section** (`system-prompt/assemble`) — rejected for this content: the assembly renders to the single `system` string, so message-shaped openers do not fit, and the system prompt is deliberately re-assembled per step (with header deltas when it changes) while the opener wants instance-frozen semantics.
- **A durable history opener** (`inject()` at session start) — rejected: permanent history is the failure mode in the problem statement — replayed everywhere, compactable, stale across resumes.
- **Compose per turn instead of per instance** — rejected: a turn-boundary recompose either desyncs silently from the log or forces a header delta per change, and it busts the provider cache exactly as often as it fires; the legitimate refresh point is the instance boundary, where the `'resume'` snapshot already records drift attributably.
- **Compose lazily at the first request and let compaction read the folded header** (the shape as first merged) — superseded in review: the fold matches the live prefix only from the instance's second request on, so on a resumed/forked instance's first step the pressure gate read the PREVIOUS instance's prefix and could under-gate. Composing before the first pre-step and handing the live value through the seam makes the estimate exact at every step.
- **Carry prompt/prefix through `agent/pre-step` for provisional pressure** — superseded by post-step replay. It coupled a generic lifecycle seam to one consumer and still missed later request routing/tools; the routed header is the exact durable home for all request-envelope fields.
- **A dedicated session event carrying the prefix** — rejected: the header events are the request's non-history record by design; a second event would be a second home for the same fact and another codec to keep total.
## Consequences
- `agent/pre-step` and `CompactService.compactIfNeeded` carry a `sessionPrefix` parameter: every pre-step listener and compaction backend sees the real per-instance value (all in-repo implementations updated in the same change, per the pre-release stance).
- `agent/pre-step` stays a generic `(agent, turn, step, signal)` checkpoint. Compaction receives no prefix parameter; `ctx.tokenMeter` folds the prefix from the canonical routed header at post-step.
- A contributor whose content changes mid-session is not re-read until the next instance — by design. A deployment needing mid-session catalog updates routes the change notice through the append-only history channels and pays one durable `context/message`.
- The dropped `after` slot leaves no request-only channel near the request tail; nothing in the repo needs one, and adding it back would re-open the every-step re-pay cost the design exists to avoid.
- The `request/header-delta` `messagePrefix` arm (whole-array replacement, empty array encoding transition to absence) exists for codec totality; the loop never exercises it, because the cached prefix cannot change within an instance.