Merge remote-tracking branch 'origin/token-meter-service' into compact-post-step-overflow-recovery

# Conflicts:
#	docs/core-data-structures/compaction.md
#	docs/rfc/implemented/architecture/2026-07-15-replay-token-meter-service.i18n.yaml
#	docs/rfc/implemented/architecture/2026-07-15-replay-token-meter-service.md
#	docs/rfc/implemented/architecture/2026-07-15-replay-token-meter-service.zh.md
#	docs/rfc/implemented/feature/2026-06-18-compaction-capability-seam.md
#	examples/coding-agent/cordis.yml
#	packages/compact/compact-basic/README.md
#	packages/compact/compact-basic/src/automatic.ts
#	packages/compact/compact-basic/src/config.ts
#	packages/compact/compact-basic/src/index.ts
#	packages/compact/compact-basic/tests/compact-basic.spec.ts
This commit is contained in:
Hypatia May
2026-07-16 16:18:16 +08:00
36 changed files with 939 additions and 1292 deletions
@@ -6,52 +6,55 @@ English | [中文](2026-07-15-replay-token-meter-service.zh.md)
## Problem
Context pressure is useful outside compaction. A compaction backend, an overflow guard, or a future request-policy plugin can all need the same answer: how much of one model's window does the durable request consume? Keeping that fold inside `dsh-compact-basic` duplicates replay logic, makes measurement unavailable without compaction, and encourages callers to reuse accounting from the wrong model.
Context pressure is useful outside compaction. A compaction backend, an overflow guard, or a future request-policy plugin can all need the same answer: how much of the configured context window does the durable request consume? Keeping that fold inside `dsh-compact-basic` duplicates replay logic, makes measurement unavailable without compaction, and encourages callers to reuse stale accounting.
Provider usage is not a complete answer. It describes one successful call under one exact request envelope, while the current surface can grow, shrink, or be replaced afterward. Sessions also switch models, old logs can lack chunk provenance, and provider fields separate input, cache-read, cache-write, output, and reasoning counts. A useful service therefore combines exact anchors with conservative model-specific repricing and exposes the log revision consumed by each result.
Provider usage is not a complete answer. It describes one successful call under one exact request envelope, while the current surface can grow, shrink, or be replaced afterward. Sessions also switch models, old logs can lack chunk provenance, and provider fields separate input, cache-read, cache-write, output, and reasoning counts. A useful service therefore combines the latest exact anchor with conservative heuristic repricing and exposes the log revision consumed by each result.
## Decision
### One concrete LLM-family service
`@deepseek-ai/dsh-token-meter` is one concrete package under `packages/llm/` and registers `ctx.tokenMeter`. It is not split into an interface and backend before a second implementation exists. Its public entry point resolves an exact model name to a stable `ModelTokenMeter`; unknown names throw `TokenMeterError` with `TOKEN_METER_MODEL_UNCONFIGURED` instead of inheriting a universal window.
`@deepseek-ai/dsh-token-meter` is one concrete package under `packages/llm/` and registers `ctx.tokenMeter`. It is not split into an interface and backend before a second implementation exists. `TokenMeterService` itself exposes `contextWindow`, `measure(session, requestHeader?)`, and `estimateMessage(message)`; consumers call the singleton service directly.
The built-in `deepseek-v4-flash` and `deepseek-v4-pro` profiles use a 128,000-token context window and four characters per estimated token. `models` overrides merge field-by-field. A custom name requires `contextWindow`, while `charsPerToken` defaults to four. Direct construction reports typed profile errors; Loader mounts first apply the package's Schemastery shape validation.
The service has one `contextWindow`, defaulting to 128,000 tokens and configurable as a positive integer. Estimation uses a fixed four-characters-per-token heuristic plus structural overhead. There are no model profiles, density settings, tokenizer backends, or language-specific strategies.
### Model-bound replay folds
### Per-session replay folds
Each model/session pair owns an isolated incremental fold. Active folds advance from `session/event`; every read catches up through the durable tail, so listener ordering, seeded sessions, and service reload do not change the answer. The fold tracks canonical request headers and deltas, step boundaries, surface appends and replacements, assistant usage, and assistant-chunk provenance. A malformed next event fails transactionally and remains unread rather than partially mutating state.
Each session owns one isolated incremental fold. Active folds advance from `session/event`; every read catches up through the durable tail, so listener ordering, seeded sessions, and service reload do not change the answer. The fold tracks canonical request headers and deltas, step boundaries, surface appends and replacements, assistant usage, and assistant-chunk provenance. A malformed next event fails transactionally and remains unread rather than partially mutating state.
`measure(session, requestHeader?)` returns scalar pressure. `measureSurface(session)` returns positional per-node prices for retention and replacement decisions. `estimateMessage(message)` applies the handle's profile without session state. Results are detached, deeply immutable snapshots carrying `logRevision`; a consumer compares scalar and surface revisions before making one decision.
`measure(session, requestHeader?)` synchronizes the fold once and returns scalar pressure together with positional per-node prices. `totalTokens` remains request-and-response pressure; `surfaceTokens` is the surface-only heuristic total and equals the sum of `nodes[].tokens`. A `requestHeader` override changes pressure pricing only, while the surface fields always describe the current session. `estimateMessage(message)` applies the fixed heuristic without session state. Each result is one detached, deeply immutable snapshot carrying one `logRevision`. Every measurement clones the current nodes and is therefore O(surface).
Provider usage is reused only when the handle's model and canonical request envelope equal the successful-call anchor. Any system, prefix, tool, or call-config change causes complete repricing under the requested model. Surface changes remain a signed delta from a matching anchor, including negative values after a shrinking replacement. A success by another model changes the shared surface but never overwrites this model's anchor.
Provider usage is reused only when the measured canonical request envelope equals the latest successful-call anchor. Any model, system, prefix, tool, or call-config change causes complete heuristic repricing. Surface changes remain a signed delta from a matching anchor, including negative values after a shrinking replacement. A later successful request replaces the earlier anchor, including across model switches.
Usage sums the disjoint input, cache-read, cache-write, and output buckets. Reasoning is not added a second time. Every successful model call records an `assistant/message`, including content-less and max-token calls, with its exact earlier chunk seqs. An explicit empty provenance list means a known empty provider stream; absent legacy provenance conservatively treats the durable assistant output as provider output.
### Compact-basic consumes, but does not own, measurement
`dsh-compact-basic` requires `ctx.tokenMeter`; `CompactService` gains no token methods or types. The backend is factored into configuration, automatic triggering, region transaction, and summarizer modules, while `summarize()` remains its sole subclass hook. The conversation model's meter consistently prices pressure, retention, shadowed content, provenance, and non-shrinking-summary rejection.
`dsh-compact-basic` requires `ctx.tokenMeter`; `CompactService` gains no token methods or types. The backend is factored into configuration, automatic triggering, region transaction, and summarizer modules, while `summarize()` remains its sole subclass hook. The singleton service consistently prices pressure, retention, shadowed content, provenance, and non-shrinking-summary rejection.
Every metered model receives a compact policy with defaults: threshold ratio `0.8`, retained tail `floor(contextWindow × 0.16)`, summarization model `''`, maximum summary output `8192`, one extra pressure-compaction attempt, one context-overflow retry, and automatic triggering enabled. Per-model compact overrides merge `thresholdRatio` and `retainTokens`; retention must remain below the resulting threshold. Empty summarization model resolves the latest logged routed model, then `AgentOptions.model`.
Automatic compaction uses one unified measurement for each threshold-and-retention decision. The region transaction measures after appending its durable `compact/start` lock and again after asynchronous summarization; any intervening durable append changes `logRevision` and prevents replacement.
Automatic pressure runs at `agent/post-step` and measures the canonical durable envelope under the model actually selected by `agent/request`. A headerless session has no completed routed request to assess and produces no work; a durable unknown routed model remains an exact typed error. Canonical overflow recovery uses the same meter for forced range selection, and retries only after a proven surface replacement.
Compact policy has service-wide defaults: threshold ratio `0.8`, retained tail `floor(contextWindow × 0.16)`, summarization model `''`, maximum summary output `8192`, one extra pressure-compaction attempt, one context-overflow retry, and automatic triggering enabled. Top-level `thresholdRatio` and `retainTokens` override the pressure policy; retention must remain below the resulting threshold. Empty summarization model resolves the latest logged routed model, then `AgentOptions.model`.
Automatic pressure runs at `agent/post-step` and measures the canonical durable envelope produced under the model actually selected by `agent/request`. A headerless session has no completed routed request to assess and produces no work; any routed model name can use the singleton estimator. Canonical overflow recovery uses the same measurement for forced range selection and retries only after a proven surface replacement.
## Testing
Unit coverage pins profiles, field-wise overrides, custom and unknown models, envelope invalidation, model switching, usage and missing-usage paths, seeded append/replace replay, signed deltas, provenance modes, malformed boundaries, immutable snapshots, listener ordering, reload, compact defaults, actual routing, retention, convergence, forced overflow, and transaction rollback. A real Loader/Include YAML fixture loads the exact zero-config token-meter and compact-basic package names in dependency order.
Unit coverage pins service configuration, fixed estimation, envelope invalidation, latest-anchor replacement across model switches, usage and missing-usage paths, seeded append/replace replay, signed deltas, provenance modes, malformed boundaries, unified snapshot detachment and deep immutability, surface-total equality, listener ordering, reload, compact defaults, actual routing, one-call automatic decisions, retention, convergence, forced overflow, generation proof, and log-revision rollback. A real Loader/Include YAML fixture loads the exact zero-config token-meter and compact-basic package names in dependency order.
## Alternatives considered
- **Keep estimation inside `CompactService`** — rejected because measurement has consumers and replay semantics independent of compaction; it would also force every compactor to expose the same unrelated API.
- **Split a token-meter interface from a heuristic backend immediately** — rejected because only one implementation exists. One concrete service preserves the future seam without speculative packages or configuration.
- **Give unknown models a 128,000-token fallback** — rejected because a plausible but wrong capacity can trigger destructive policy at the wrong point. Unknown routed names fail with their exact name.
- **Copy complete history into each scalar result** — rejected because below-threshold reads are common. Immutable revisioned scalars and a separate surface snapshot preserve consistency without an O(history) copy.
- **Treat provider usage as portable between models or envelopes** — rejected because tokenization, context capacity, tools, prefixes, and call config are model/request facts. Mismatch reprices the whole current request.
- **Keep model-keyed windows and density profiles** — rejected because the deployment currently has one context policy and one estimator. Model registries, unknown-model failures, and configurable density add branches without a second behavior to select.
- **Keep separate scalar and surface measurements** — rejected because callers would need two reads and revision matching for one decision. A scalar-only read could avoid cloning nodes below threshold, but the split API introduces a caller-side race window; the unified snapshot accepts O(surface) cloning in exchange for coherence.
- **Treat provider usage as portable between envelopes** — rejected because model, tools, prefixes, and call config are request facts. Mismatch reprices the whole current request.
## Consequences
- Token pressure has one replay-aware owner that compaction and future plugins can share.
- Defaults make the bundled DeepSeek composition usable with two zero-config plugin entries, while custom models must state the one fact that cannot be guessed safely: context capacity.
- Heuristic density and provider usage remain estimates of provider behavior. Maintainers must update built-in profiles and overflow wording as models evolve.
- The default makes the bundled composition usable with two zero-config plugin entries; deployments override one context capacity when needed.
- Fixed heuristic pricing remains an estimate of provider behavior and is not an exact tokenizer or request serializer.
- Every measurement clones the current positional surface and therefore costs O(surface), including pressure checks that finish below threshold.
- Measurements fail loudly on malformed durable boundaries. This turns corrupted replay into a named integration failure instead of silently drifting pressure.
- Post-step pressure reads the exact logged routing/tools/prefix boundary; provider overflow classification remains the adapter-maintained backstop for requests rejected before a successful usage anchor.