Merge worktree-hooks-b-bash-seam into worktree-hooks-c-interception
Bring the interception-seams branch onto current master (via A→B). The substantive reconciliation is master's compaction `agent/pre-step` serial seam meeting C's interception seams: - types.ts: keep BOTH master's `agent/pre-step` AND C's new interception events (`agent/prompt-submit`, `agent/session-start`, `agent/turn-continuation`→ `ContinuationDecision`); drop the turn-mirror declarations (removed on A). - loop.ts: the merged per-turn order is `turn/start` → per queued msg `agent/prompt-submit` (rewrite/inject/block) → (fully-blocked ⇒ zero-step `rejected`) → per step: drain steering → assemble system prompt → `agent/pre-step` (compaction, OUTSIDE the step) → `step/start` → single `deriveMessages()` → model → tools/pre-execute·dispatch·post-execute. No turn-mirror emits; `closeTurn()` is the A-simplified single-call form. - Docs (architecture, core.md, agent/agent-loop READMEs, catalog) reconciled to show C's interception seams alongside `agent/pre-step`, no turn/step mirrors. - rfc/README: dropped the stale `proposed/` compaction row (master moved that RFC to implemented/); kept C's new `pre-tool-input-rewrite` proposed row. - interception.spec.ts: migrated its two `agent/turn-end` reason collectors to the `turn/end` session event, and ADDED a cross-test proving a `prompt-submit` rewrite + additionalContext is VISIBLE to an `agent/pre-step` listener on the same turn — pinning the merged seam ordering (compaction sees the post-prompt-submit surface, not stale history).
This commit is contained in:
Tianyi Cui
@@ -27,7 +27,7 @@ The mapping between ACP and existing harness seams — each row names the seam a
|
||||
| `session/new {cwd, mcpServers, additionalDirectories}` → `{sessionId}` | the `dsh-agent` create factory (see Dependency note + Plan) | the seam must accept `{ sessionId, meta }` so the ACP-generated `sessionId` becomes the live/persisted session id and the validated `cwd` is attached as the `SessionHeader` (today `AgentLoop.create(id)` hardcodes `${id}-session` and takes no metadata); reject a 2nd session (single-session MVP, see [ACP multi-session](2026-06-14-acp-multi-session.md)); `cwd` validated (require absolute) — any absolute cwd is honored: it becomes the session's `SessionHeader.cwd` and the default bash workdir (per-session cwd, see § Deferred → RESOLVED), so the server need not launch in the workspace; non-empty `mcpServers` and `additionalDirectories` are rejected for the MVP because silently ignoring requested servers/roots would desync the client's tool and filesystem-scope UI |
|
||||
| `session/load {sessionId, cwd, mcpServers, additionalDirectories}` | the `dsh-agent` resume factory ([session persistence](../../implemented/architecture/2026-06-14-session-persistence.md) + Dependency note) | load `{ meta, events }`, seed the session, re-derive history via `deriveMessages()`, replay prior turns to the client as `session/update` per the ACP load contract; `mcpServers` and `additionalDirectories` rejected as in `session/new` |
|
||||
| `session/prompt {prompt}` | `agent.send()` (idle) | text blocks → `TextBlock`; reject image/audio per advertised capabilities; one in-flight prompt per session |
|
||||
| resolve `session/prompt` → `{stopReason}` | `agent/turn-end` (extended, see Plan) | map the harness kebab `TurnEndReason` to the ACP snake_case `StopReason` wire enum: `completed`→`end_turn`, `max-tokens`→`max_tokens`, `aborted`(cancel)→`cancelled`, plus `refusal`/`max_turn_requests` when applicable; honor the batch-into-one-turn and send-not-synchronously-running settle semantics |
|
||||
| resolve `session/prompt` → `{stopReason}` | the `turn/end` `session/event` (its `reason`) | map the harness kebab `TurnEndReason` to the ACP snake_case `StopReason` wire enum: `completed`→`end_turn`, `max-tokens`→`max_tokens`, `aborted`(cancel)→`cancelled`, plus `refusal`/`max_turn_requests` when applicable; honor the batch-into-one-turn and send-not-synchronously-running settle semantics |
|
||||
| `session/update: agent_message_chunk` | `agent/stream-chunk` `text-delta` only | do NOT also emit on `block-end(TextBlock)` — it carries the fully-assembled block and would duplicate the streamed text |
|
||||
| `session/update: agent_thought_chunk` | `agent/stream-chunk` `reasoning-delta` | |
|
||||
| `session/update: tool_call` (pending→in_progress) | `session/event` `tool/call` | demux via a Session→sessionId map; `kind` inferred from the tool name |
|
||||
@@ -46,7 +46,7 @@ Lifecycle and disposal: the connection, listeners, and in-flight permission prom
|
||||
1. Package scaffold `packages/ui/acp/` per [the cookbook](../../../cookbook/adding-a-package.md); add `@agentclientprotocol/sdk` and `zod`. Add the abstract create/resume factory to `dsh-agent` (the interface) so the bridge can `inject: ['agents', 'sessions', 'tools', 'sessionPersistence']` without depending on the concrete loop; `sessionPersistence` is required because `session/load` advertises `loadSession: true`. (Fallback only if the factory is judged not worth it: inject `agentLoop` directly and record the architecture-rule exception in `docs/architecture.md`.)
|
||||
2. Connection plus `initialize`/`session/new`: wire `AgentSideConnection` to stdin/stdout; protocolVersion negotiation; the single-session guard; create the live session through the new `{ sessionId, meta }` factory seam (so the ACP `sessionId` and validated `cwd` become the session's id and header); the `sessionId↔agent` and `Session↔sessionId` maps.
|
||||
3. Internal edit — turn-end reason fidelity (sanctioned: edit internals to fit ACP). Extend `TurnEndReasonMap` in the proper places: (a) declaration-merge a `max-tokens` variant in the owning package (`packages/core/session/src/types.ts`, alongside `completed|aborted|error|disposed`) — add `max-tokens` because `FinishReasonMap` produces it (DeepSeek maps `length` → `max-tokens`); do not add `refusal`, since no current adapter produces it (unknown DeepSeek finish reasons collapse to `error`), but leave a comment in `TurnEndReasonMap` noting `refusal` should be added when an adapter first emits it (`FinishReasonMap` is merge-extensible); (b) make `agent-loop`'s `loop.ts` populate the reason from the model `finish` chunk — `assembler.finish` lives inside `runStep`, so `runStep` must return it up to `runTurn`, and the rule is "the last step's finish reason wins, but any `max-tokens` in the turn surfaces as `max-tokens`"; (c) no consumer exhaustively switches over `TurnEndReason` today (the invariants plugin switches on `SessionEventType`, and `deriveMessages` ignores `turn/end`), so adding `max-tokens` is a non-breaking extension — but recheck before landing; (d) update [docs/architecture.md](../../../architecture.md) (the CI-verified loop-lifecycle/event-taxonomy doc) and the affected package READMEs/JSDoc (`dsh-session`, `dsh-agent`, `dsh-agent-loop`) per the repo doc-sync policy. This replaces a fragile "observe the finish chunk in the bridge" hack with a real, documented contract.
|
||||
4. Prompt-turn streaming plus load: translate `agent/stream-chunk` and `session/event` into `session/update`; resolve `session/prompt` on settle, mapping the harness `TurnEndReason` to the ACP `StopReason` wire enum (`completed`→`end_turn`, `max-tokens`→`max_tokens`, `aborted`→`cancelled`) — a small total function with a test asserting the exact wire strings, since the SDK rejects an unknown `stopReason`. Concrete correlation, since the loop batches queued messages into one turn and `send()` does not synchronously flip to running: install listeners before `send()`; gate on an observed `agent/turn-start` (confirms work was accepted) then resolve on the next `agent/turn-end`; reject an empty/whitespace prompt up front rather than calling `send()` (no turn would ever start, so the RPC would hang). Implement `session/load` on the session-persistence resume seam.
|
||||
4. Prompt-turn streaming plus load: translate `agent/stream-chunk` and `session/event` into `session/update`; resolve `session/prompt` on settle, mapping the harness `TurnEndReason` to the ACP `StopReason` wire enum (`completed`→`end_turn`, `max-tokens`→`max_tokens`, `aborted`→`cancelled`) — a small total function with a test asserting the exact wire strings, since the SDK rejects an unknown `stopReason`. Concrete correlation, since the loop batches queued messages into one turn and `send()` does not synchronously flip to running: install the `session/event` listener before `send()`; capture the prompt's owning turn from its `turn/start` record, then resolve on that turn's `turn/end` (with `agent/status` idle/disposed as a fallback); reject an empty/whitespace prompt up front rather than calling `send()` (no turn would ever start, so the RPC would hang). Implement `session/load` on the session-persistence resume seam.
|
||||
5. Permission gate: a single `tools/execute` listener registered with `prepend: true`, owning a `WeakMap<Agent, sessionId>` of bridge-created agents; no-op (`next()`) for unowned/no-agent calls; for owned calls → `session/request_permission` → allow (`next()`) / veto; settle the stored resolver exactly once on outcome, cancel, or connection close.
|
||||
6. Example wiring (extract a shared base). `@cordisjs/plugin-include` is itself a plugin entry that resets `ctx.baseUrl` and loads a path, so a child `cordis.yml` can nest-include a shared base; the extraction is safe because every dependent plugin declares `inject` (loader groups initialize via `Promise.all`, so YAML order is NOT the dependency mechanism — never rely on it). Extract the provider/tool core (`llm, sessions, system-prompt, tools, agents, invariants, llm-deepseek, bash-local, tool-bash`) into `examples/base.yml`; have both `coding-agent` and a new `examples/acp-agent/` include it and add their own UI plugin plus logger. Keep `agent-loop` per-example (NOT in the base): `AgentLoop` creates its configured agents in its constructor, and the two examples disagree — `coding-agent` needs a pre-created `main` (its `stdio-chat` calls `ctx.agents.get('main')`), while `acp-agent` must pre-create none (ACP `session/new` creates agents). So `coding-agent` declares `agent-loop` with `agents: [{ id: main, … }]` and `acp-agent` with `agents: []`. `acp-agent` loads `dsh-session-persistence-jsonl` (from [session persistence](../../implemented/architecture/2026-06-14-session-persistence.md) — required for `session/load`), omits the stdout logger (see Risks), and adds `pnpm run demo:acp` plus the Zed `agent_servers` snippet.
|
||||
7. Tests (the repo cares a lot here): a property-based test for the protocol shape (precedent: [property-based testing](../../implemented/testing/2026-06-11-property-based-testing.md)) — fuzz arbitrary harness event sequences and assert ACP-stream invariants (never a `tool_call_update` before its `tool_call`; exactly one `session/prompt` resolution per prompt; monotonic, well-formed ordering; `stopReason` in the legal set); codec unit tests over an in-memory `Duplex` pair (drive `AgentSideConnection` without a subprocess; assert exact frames for `initialize`, `session/new`, a full prompt turn); the mandatory HMR-safety test (dispose the fiber; assert the connection closed, all `ctx.on` listeners gone, any in-flight `request_permission` settled); failure-path tests (connection closes mid-stream; closes with a permission pending; a notification `send()` rejects but the turn survives; `finish{kind:'error'|'aborted'}`; a `tools/execute` throw with no `tool/result`; a second `session/new` rejected; a `session/prompt` while one is in flight; an empty prompt rejected without hanging; a `session/load` re-derives identical history and replays it); and an e2e (`*.e2e.ts`, self-skips without `DEEPSEEK_API_KEY`) that boots `examples/acp-agent`, connects a `ClientSideConnection`, sends a real prompt, owns and disposes the harness in `afterEach`, and verifies the world (files on disk), not the agent's self-report.
|
||||
|
||||
@@ -1,59 +0,0 @@
|
||||
# RFC: Compaction as a capability seam (abstract contract + basic backend)
|
||||
|
||||
Status: proposed (2026-06-18)
|
||||
|
||||
## Context
|
||||
|
||||
A long-running agent conversation grows without bound. As the event log accumulates turns, the derived message history eventually approaches the model's context window — the model then truncates mid-response (`max-tokens`) or degrades. **Compaction** is the mitigation: replace a run of older history with a concise summary, keeping recent context intact.
|
||||
|
||||
The [session surface](../../implemented/architecture/2026-06-18-session-surface.md) was built as the foundation for exactly this — a linked list over the event log with a `surfaceOp: { op: 'replace', start, end }` operation purpose-built to shadow a range of nodes and insert a replacement, with `sourceEventSeqs` recording provenance so the decision replays deterministically. What remained was the plugin that *decides what to compact and produces the summary*.
|
||||
|
||||
Two forces shape the design. First, compaction is **swappable**: token counting can be a char/4 heuristic or a real tokenizer, and summarization can be a model call, a template, or a remote service — these vary independently of *when* and *which range* to compact. Second, a later commit (`ce43c25`) closed `SurfaceEventType` to five event types (`user/message`, `assistant/message`, `tool/result`, `context/message`, `steering/message`); only those may carry `surfaceOp`. A bespoke `compaction/*` event therefore **cannot** itself appear on the surface — the compiler rejects `surfaceOp` on it and the invariants plugin rejects it at runtime.
|
||||
|
||||
## Decision
|
||||
|
||||
### Compaction is a capability seam, split interface / implementation
|
||||
|
||||
Per the [capability-seams RFC](../../implemented/architecture/2026-06-13-capability-seams.md), compaction ships as separate packages so the contract, the algorithm, and (later) the consumer surface evolve independently:
|
||||
|
||||
1. **Interface** — `@deepseek-ai/dsh-compact`: an abstract `CompactService` owning the `ctx.compact` key, the `CompactionResult` vocabulary, and the `compact/*` session events. It declares `compactIfNeeded()` and `compactRegion()` as **abstract** — the contract states *what* compaction does, not *how*.
|
||||
2. **Implementation** — `@deepseek-ai/dsh-compact-basic`: a concrete `BasicCompactService` that owns the entire algorithm — token estimation (char/4 + per-block overhead), the tail→head retention walk, summarization via `ctx.llm.generate()`, the surface replacement, the lock, and the `agent/request` auto-compaction listener. A tokenizer-based or template-based backend is a sibling package (or a subclass overriding the two protected estimation/summarization hooks).
|
||||
3. **Consumer** — deferred. A `/compact` tool and slash command will `inject: ['compact']` and call the contract; they are intentionally out of scope here so the seam settles first.
|
||||
|
||||
### The contract depends on `dsh-session` and `dsh-llm` — a deliberate deviation
|
||||
|
||||
The capability-seams RFC states the interface package "depends only on cordis" (true of `dsh-bash`, whose vocabulary is self-contained). Compaction **cannot** honor that: its verbs are defined *over* a `Session` (`compactRegion(session, start, end)`) and its output *is* the content vocabulary (`CompactionResult.summary: ContentBlock[]`). There is no way to express the contract without naming `Session`/`SessionEvent` (from `dsh-session`) and `ContentBlock` (from `dsh-llm`).
|
||||
|
||||
This is not a coupling smell — it is the contract's domain. The "only cordis" guidance was always shorthand for "the interface depends only on what the contract genuinely names, and never on an implementation." `dsh-session` and `dsh-llm` are themselves interface/vocabulary packages, not implementations; `dsh-compact` still imports no backend. The seam's real invariant — *consumers and implementations evolve independently behind an abstract service* — holds intact. We record the deviation here so a future reader doesn't mistake it for an accident or "fix" it by smuggling `Session` behind an opaque handle.
|
||||
|
||||
### Abstract `compactIfNeeded` / `compactRegion`, algorithm in the backend
|
||||
|
||||
An earlier draft put the full algorithm (the retention walk, token-summing, text extraction) as concrete methods on the interface, with only `estimateContentTokens()` and `summarize()` abstract. That recouples the contract to one strategy: a backend that wants a different retention policy (e.g. turn-count instead of token-budget) or a different event-sequencing would have to fight inherited concrete code. Making both core methods abstract puts every *how* decision in the backend, where it belongs, and keeps the interface a pure statement of *what*. The backend remains internally factored — `estimateContentTokens()` and `summarize()` are `protected` hooks a sub-backend can override without reimplementing the walk — but that factoring is the backend's private concern, not the contract's.
|
||||
|
||||
### Surface replacement: `compact/*` events are log-only; one `user/message` carries the summary
|
||||
|
||||
Because `SurfaceEventType` is closed, the summary cannot ride on a `compact/*` event. The backend instead appends a **single `user/message`** with `surfaceOp: { op: 'replace', start, end }` whose `content` is the summary `ContentBlock[]` and whose `sourceEventSeqs` covers the shadowed nodes *and* the bookkeeping events. The `compact/*` events are pure log records (lock + provenance), never on the surface. The surface mutation sits **inside** the lock — `compact/end` is the last event appended:
|
||||
|
||||
```
|
||||
compact/start → log-only. Acquires the lock.
|
||||
[summarize older range via the backend]
|
||||
compact/summary → log-only. Provenance: summary, range, shadowed seqs, token count.
|
||||
user/message → surfaceOp { op:'replace', start, end }. THE surface mutation.
|
||||
deriveMessages() renders it as a user-role message.
|
||||
compact/end → log-only. Releases the lock.
|
||||
```
|
||||
|
||||
Ordering the surface mutation **before** `compact/end` is deliberate: `session.append()` commits one event at a time, so there is no multi-event transaction to make the sequence atomic. Releasing the lock last converts the crash window from *silent corruption* (a `compact/end` that claims compaction finished while the surface was never shadowed) into a *detectable orphaned lock* (a `compact/start` with no matching `compact/end`), which a persistence backend already detects on reload. A `session/event` listener on `compact/end` likewise never sees the lock free before the replacement has landed.
|
||||
|
||||
`deriveMessages()` then yields `[summary_as_user_message, ...retained_nodes]`. An alternative — extending `SurfaceEventType` to admit a `compact/*` type — was rejected: the closed union is a deliberate safety boundary (only message-producing events reach the model), and a summary genuinely *is* user-role context, so reusing `user/message` is honest rather than a workaround.
|
||||
|
||||
### Blocking via a log-recorded lock, not a mutex
|
||||
|
||||
Compaction must be serialized: no second compaction starts before the first finishes, and no ordinary events interleave the slow summarization. Rather than an in-memory mutex (invisible to replay, lost on crash), the lock **is** the log: `compactRegion` refuses to start if the last `compact/start` has no matching `compact/end` after it. `compact/start` is appended first (fast, synchronous), the slow model call runs, then the `compact/summary` and `user/message` replacement land, and only then is `compact/end` appended — in a `catch` that records the error, so a failed summarization can never wedge the lock. Because the backend runs compaction synchronously inside the `agent/request` waterfall, the loop is single-threaded for that window; the lock additionally gives observability and lets a persistence backend detect an orphaned `compact/start` on reload.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **New packages**: `packages/compact/compact` (interface) and a sibling `compact-basic` (backend) under `packages/compact/`, wired into the three root tsconfigs. The consumer tier is deferred.
|
||||
- **`SessionEventMap`** gains `compact/start` / `compact/summary` / `compact/end` by declaration merging (merge-extensible); `SurfaceEventType` is **not** touched. These are session events, not cordis `Events`, so the event-taxonomy gate needs no entry.
|
||||
- **No changes** to `dsh-session`, `dsh-invariants`, or `dsh-agent-loop`: the surface replace op, the surface-metadata runtime guard, and the `agent/request` waterfall all already exist. Compaction is a pure plugin on documented seams.
|
||||
- The capability-seams convention gains a second reference beyond bash, and a documented case where "interface depends only on cordis" relaxes to "depends only on interface/vocabulary packages the contract genuinely names." On acceptance, [AGENTS.md](../../../../AGENTS.md) § Conventions and [architecture.md](../../../architecture.md) § "Capability seams" should note this relaxation.
|
||||
Reference in New Issue
Block a user