Pine/deepseek-harness
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases 1 Wiki Activity

round 1: implement bracket-first manual compaction

Browse Source
This commit is contained in:
Hypatia May
2026-07-30 17:40:25 +08:00
parent 86b95a3856
commit faac9b4fd5
101 changed files with 3452 additions and 295 deletions
Download Patch File Download Diff File Expand all files Collapse all files
packages/compact/compact/README.md
+13 -6
View File
@@ -10,22 +10,25 @@ This package is the interface tier of the compaction capability, split so each c
|---|---|
| `@deepseek-ai/dsh-compact` (this) | the interface: abstract service + `compact/*` events + `CompactionResult` + canonical checkpoint source + tool-pairing boundary helpers |
| `@deepseek-ai/dsh-compact-basic` | a backend: `ctx.tokenMeter` pressure + token-budget retention + `llm.stream()` summarization |
| `@deepseek-ai/dsh-tool-compact` (deferred) | the model-facing `/compact` tool over `ctx.compact` |
| `@deepseek-ai/dsh-command-compact` | the human `/compact` command over `ctx.compact.compactNow()` |
Unlike the bash seam, this interface depends on `@deepseek-ai/dsh-session` and `@deepseek-ai/dsh-llm` — the contract's verbs are defined over a `Session` and its output is the `ContentBlock` vocabulary, so they cannot be expressed without naming those packages. That deviation from the "interface depends only on cordis" guidance is intentional and recorded in the [compaction capability-seam Agent Note](../../../.agents/notes/implemented/feature/2026-06-18-compaction-capability-seam.md).
## Service API (`ctx.compact`)
Both methods are **abstract** — the backend owns trigger policy, retention, event sequencing, and summarization. Reusable request measurement is a separate service, [`ctx.tokenMeter`](../../llm/token-meter/README.md), rather than part of this interface.
All three operations are **abstract** — the backend owns trigger policy, retention, event sequencing, and summarization. Reusable request measurement is a separate service, [`ctx.tokenMeter`](../../llm/token-meter/README.md), rather than part of this interface.
| Member | Semantics |
|---|---|
| `compactIfNeeded(agent, trigger, signal)` | Consider automatic compaction for `trigger: 'pressure' \| 'context-overflow'`. A pressure trigger may apply the backend's threshold and retained-tail policy; a confirmed overflow may force a useful balanced reduction. Returns the `CompactionResult`, or `null` when no safe range exists. A backend's summarization request is a direct `ctx.llm.stream()` call (not a loop step), so per-call interception happens at `llm/stream`. |
| `compactNow(agent, signal)` | Explicitly compact one useful balanced older span even below automatic pressure. It synchronously reserves idle turn admission before yielding, writes nothing when no useful span exists, records a standalone `compact/* { turn: null }` attempt before summarization, and awaits its durability checkpoint before release. Expected operational failures use `ManualCompactionError`; cancellation rethrows the exact abort reason. |
| `compactRegion(start, end, agent, signal?)` | Forcibly summarize surface nodes `[start, end]` (inclusive seqs) from `agent.session` into a single replacement node whose source is `COMPACT_CHECKPOINT_SOURCE`. **Throws** if a compaction is already in progress, if `start`/`end` aren't surface nodes, or if `start` is positioned after `end` on the surface. The range is a SURFACE-POSITION span, not a numeric seq interval — after a prior replace lands a fresh high-seq summary node at the shadowed range's position, surface order no longer tracks seq order. |
`CompactionResult` keeps the raw summary and bookkeeping-event seqs available to callers alongside the shadowed range and token accounting; its drift-checked shape lives in the [compaction data-structure reference](../../../docs/core-data-structures/compaction.md#compactionresult).
`compactIfNeeded` takes a required `signal`; `compactRegion`'s is optional. A backend that summarizes via `ctx.llm.stream()` **must** forward it into the call's `GenerateOptions.signal`, so an abort or fiber dispose tears down the in-flight summarization instead of leaving an orphaned model call running past the cancellation. The turn that the `compact/*` events belong to is recoverable from the owned session's log (the currently-open turn), so the backend stamps it from the log rather than trusting a caller-supplied value.
`compactIfNeeded` and `compactNow` take a required `signal`; `compactRegion`'s is optional. A backend that summarizes via `ctx.llm.stream()` **must** forward it into the call's `GenerateOptions.signal`, so an abort or fiber dispose tears down the in-flight summarization. Automatic and explicit-region brackets recover their numeric owner from the currently open turn. Manual brackets require no open turn and stamp `turn: null`.
`ManualCompactionError.code` is the closed set `busy | changed | summary | commit | persistence`. `changed` and `summary` mean the selected conversation surface was not replaced, but their failed attempt is still recorded in the session log. `commit` is deliberately neutral about partial mutation, and `persistence` means the in-memory bracket closed but its explicit flush failed.
## Tool-pairing boundaries
@@ -45,11 +48,15 @@ The private per-session cache is keyed by `session.surface.replaceGeneration` an
The surface mutation (step 4) sits **inside** the lock bracket: `compact/end` is the last event, so the lock is never released before the mutation lands. A crash between `compact/start` and `compact/end` therefore leaves a detectable orphaned lock (a `compact/start` with no matching `compact/end`) rather than a `compact/end` that falsely claims compaction finished while the surface was never shadowed.
The marker pair names lock acquisition and release, not an exclusive event container. An idle `inject()` may append unrelated context between a manual start and end while summarization is pending. Manual stability therefore revalidates the selected span rather than demanding whole-surface equality; the positional replacement leaves that injected context visible after the checkpoint. Automatic compaction keeps whole-surface equality inside its active turn.
`deriveMessages()` then renders the summary as a user-role message followed by the retained nodes. The shadowed events remain in the raw log, so replay is deterministic.
## Blocking
Compaction is serialized via a log-recorded lock: `compactRegion` refuses to start if the last `compact/start` has no matching `compact/end` after it. The lock is the log (not an in-memory mutex), so it survives replay and a persistence backend can detect an orphaned `compact/start` on reload. The lock brackets the **whole** operation — summarization, the `compact/summary` provenance record, *and* the `user/message` surface replacement all happen before `compact/end` — so a `session/event` listener firing on `compact/end` never observes the lock free while the surface mutation is still pending. The basic backend revalidates the selected surface after summarization: a surface change rejects, while an unrelated log-only append does not invalidate the replacement. `compact/end` is appended even when summarization throws, so a failure can never wedge the lock.
Compaction is serialized by one log-recorded lock shared by all entry points. Tail inspection independently finds the latest unmatched `compact/start` and the newest `session/end-seed`. An unmatched start after that boundary is live and reports `busy`; an older unmatched start is stale evidence from a prior process lifecycle and does not block. The same end-seed transition clears the invariant companion's replay trace.
The lock is the durable bracket, not a `WeakSet`, wrapper mutex, or client-side anchor. `compact/start` is appended synchronously before summarization yields. Every later failure makes exactly one `compact/end { error }` attempt; if that close append itself fails, the unmatched start remains the intentional busy signal and no flush is attempted. A successfully closed manual attempt is flushed even when it reports `changed` or `summary`, preserving the recorded attempt before turn admission is released.
## Events
@@ -57,7 +64,7 @@ The `compact/*` events extend `SessionEventMap` (merge-extensible) via declarati
## Implementing a backend
Subclass `CompactService`, implement `compactIfNeeded` and `compactRegion`, and load the subclass as a plugin — it registers as `ctx.compact`. Every successful backend uses `COMPACT_CHECKPOINT_SOURCE` on its replacement user message; `isCompactCheckpointSource()` recognizes the marker after persistence or cloning without depending on backend identity. A template- or model-backed implementation can live as a sibling package without changing callers or the shared token meter.
Subclass `CompactService`, implement `compactIfNeeded`, `compactNow`, and `compactRegion`, and load the subclass as a plugin — it registers as `ctx.compact`. Every successful backend uses `COMPACT_CHECKPOINT_SOURCE` on its replacement user message; `isCompactCheckpointSource()` recognizes the marker after persistence or cloning without depending on backend identity. A template- or model-backed implementation can live as a sibling package without changing callers or the shared token meter.
## Recognizing a checkpoint outside the host program (`./checkpoint`)
@@ -81,6 +88,6 @@ A successful backend replacement invalidates reuse from the first shadowed histo
## Known Limitations and Deferred Work
- **No model-facing consumer tier yet** — `@deepseek-ai/dsh-tool-compact` (the `/compact` tool) is deferred; compaction is reachable only via direct `ctx.compact` calls or a backend's auto listener.
- **Human command, not a model tool** — `@deepseek-ai/dsh-command-compact` exposes argument-free `/compact` through `ctx.commands`; no model-facing compaction tool is registered.
- **Some single-unit overflow is out of contract** — balanced summary compaction cannot split one indivisible unit. The optional pruning companion can still repair a closed tool pair when text-bearing tool-result bulk is removable; a large non-tool node or a tool unit whose non-prunable remainder is oversized cannot be compacted.
- **An envelope that alone approaches the window is not surface-compaction work** — compaction shrinks derived history, never the system prompt, tools, or session prefix.