Merge newest origin/master into token-meter-service
This commit is contained in:
Tianyi Cui
@@ -1,4 +1,4 @@
|
||||
# RFC: Session surface — a linked list over the event log for LLM message derivation
|
||||
# RFC: Session surface — an ordered projection over the event log
|
||||
|
||||
Status: implemented
|
||||
|
||||
@@ -8,7 +8,7 @@ The event log is authoritative, but history manipulation had no durable shared m
|
||||
|
||||
## Decision
|
||||
|
||||
Add a **surface** — a derived, cached linked list of "surface nodes" (the subset of events that produce LLM messages) — maintained by `surfaceOp` markers in the event log.
|
||||
Add a **surface** — a derived, cached order of event sequences (the subset of events that produce LLM messages) — maintained by `surfaceOp` markers in the event log.
|
||||
|
||||
### Two new top-level fields on `SessionEvent`
|
||||
|
||||
@@ -25,13 +25,13 @@ export type SurfaceOp =
|
||||
| { op: 'replace'; start: number; end: number } // shadow [start, end] inclusive
|
||||
```
|
||||
|
||||
1. **Append** — add a new node to the tail. Used by `user/message`, `assistant/message`, `tool/result`, `context/message`, `steering/message`. The loop passes `surfaceOp: 'append'` on all such appends and records `sourceEventSeqs` where applicable: every successful `assistant/message` records its complete `assistant/chunk` source set, including `[]`, while `tool/result` records its `tool/call` source.
|
||||
1. **Append** — add the new event seq to the tail. Used by `user/message`, `assistant/message`, `tool/result`, `context/message`, `steering/message`. The loop passes `surfaceOp: 'append'` on all such appends and records `sourceEventSeqs` where applicable: every successful `assistant/message` records its complete `assistant/chunk` source set, including `[]`, while `tool/result` records its `tool/call` source.
|
||||
|
||||
2. **Replace** — remove nodes from `start` through `end` (both inclusive) and insert a new node in their place. Both `start` and `end` must be valid surface node seqs in the current surface; `start === end` replaces a single node. The node's `sourceEventSeqs` must contain every shadowed surface node. The shadowed events remain in the log but are no longer on the surface.
|
||||
2. **Replace** — remove entries from `start` through `end` (both inclusive) and insert the new event seq in their place. Both `start` and `end` must be present in the current surface; `start === end` replaces one entry. The event's `sourceEventSeqs` must contain every shadowed surface seq. The shadowed events remain in the log but are no longer on the surface.
|
||||
|
||||
### SurfaceManager: delta-based, not full rebuild
|
||||
|
||||
A `SurfaceManager` class (private to `Session`) maintains the cached linked list. It tracks `_lastProcessedSeq` and processes only the **delta** (new events since the last access) rather than rescanning the entire log. Because the log is append-only, prior events never change; a seeded log is simply the initial delta folded on first access.
|
||||
A `SurfaceManager` class (private to `Session`) maintains one ordered `number[]` of event seqs. It tracks `_lastProcessedSeq` and processes only the new events since the last access rather than rescanning the entire log. Because the log is append-only, prior events never change; a seeded log is simply the initial suffix folded on first access. Replace locates its inclusive endpoints by array position and splices the replacement seq into that range; no link objects or seq-to-node map duplicate the order.
|
||||
|
||||
Delta processing is O(1) when no new events and O(new events) when new events arrive.
|
||||
|
||||
@@ -54,16 +54,17 @@ Every surface-eligible event must carry `surfaceOp` or it would disappear from d
|
||||
## Alternatives considered
|
||||
|
||||
- **Per-plugin `agent/request` wrapping** (the pre-surface pattern for history manipulation) — listener-ordering fragility, no durable record of what was changed, and every new manipulation forces another change to core `deriveMessages()`.
|
||||
- **Half-open `[start, endExclusive)` replace ranges** — rejected: the surface is a doubly-linked list whose ends are naturally named by node seqs, and single-node replacement (`start === end`) reads naturally with inclusive semantics.
|
||||
- **Half-open `[start, endExclusive)` replace ranges** — rejected: endpoints are named by surface event seqs, and single-entry replacement (`start === end`) reads naturally with inclusive semantics.
|
||||
- **Linked node objects plus a seq map** — rejected: production did not read predecessor links, the only successor use was the next array position, and replacement already required linear `indexOf` lookup. A single seq array preserves the same asymptotic behavior with one representation to validate.
|
||||
- **Full rebuild behind a dirty flag** instead of delta processing — O(N²) over a session's lifetime: every single-event append would rescan all prior events.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **`packages/core/session`**: New `surface.ts` (`SurfaceManager`), new types (`SurfaceOp`, `SurfaceIntent`), new fields on `SessionEvent`, modified `append()` (third required `SurfaceIntent` param), refactored `deriveMessages()` (walks the surface as the sole derivation path), surface-aware `repair.ts`. The seed constructor rejects a surface-eligible seed event missing its `surfaceOp` marker (see § Invariants).
|
||||
- **`packages/core/session`**: `surface.ts` (`SurfaceManager`) maintains one ordered seq array; `SurfaceOp`/`SurfaceIntent` and the top-level session-event fields record how entries join it. `append()` requires a `SurfaceIntent` for surface events, `deriveMessages()` walks the surface as the sole derivation path, and `repair.ts` emits surface-aware closers. The seed constructor rejects a surface-eligible seed event missing its `surfaceOp` marker (see § Invariants).
|
||||
- **`packages/core/agent-loop`**: All surface-capable appends pass surface opts. Chunk seqs are collected for `assistant/message` provenance; `tool/call` seqs are captured for `tool/result` provenance.
|
||||
- **`packages/session-persistence/session-persistence-sqlite`**: Two new nullable TEXT columns (`source_event_seqs`, `surface_op`) on the `events` table; `SCHEMA_VERSION` bumped (bump-and-reject, no migration).
|
||||
- **`packages/support/invariants`**: Surface-related validation rules.
|
||||
- **`packages/session-persistence/session-persistence-jsonl`**: No changes required.
|
||||
- **`packages/session-persistence/session-persistence`**: Abstract interface unchanged.
|
||||
|
||||
The surface is the foundation for future history manipulation. A compaction or tool-result-prune plugin appends one of the existing message-producing event types (a `user/message` carrying the summary, say) with `surfaceOp: { op: 'replace', start, end }` and `sourceEventSeqs` covering the shadowed nodes — the new node takes the range's place on the surface while the plugin's own trace events (e.g. `compaction/start`, `compaction/end`) stay off it. Replay preserves the decision deterministically.
|
||||
The surface is the foundation for future history manipulation. A compaction or tool-result-prune plugin appends one of the existing message-producing event types (a `user/message` carrying the summary, say) with `surfaceOp: { op: 'replace', start, end }` and `sourceEventSeqs` covering the shadowed entries — the new event takes the range's place on the surface while the plugin's own trace events (e.g. `compaction/start`, `compaction/end`) stay off it. Replay preserves the decision deterministically.
|
||||
|
||||
@@ -18,13 +18,13 @@ Prefix-cache stability is corollary #1, not the headline: an append-only log pro
|
||||
|
||||
### The mechanism
|
||||
|
||||
**Messages.** `Session.deriveMessages()` is cached: each surface node is projected exactly once, when first seen, through the public per-node function `deriveEventMessage(event)`; a surface rewrite (a compaction `replace` — `SurfaceManager.replaceGeneration`) rebuilds. Callers get a fresh array per call over shared, deep-frozen messages: mutating logged history through a projection is unrepresentable (it throws), replacing the old clone-per-call isolation. External reconstructors fold the same public function over a log prefix, so no two paths can disagree.
|
||||
**Messages.** `Session.deriveMessages()` is cached: each surface entry is projected exactly once, when first seen, through the public per-event function `deriveEventMessage(event)`; a surface rewrite (a compaction `replace` — `SurfaceManager.replaceGeneration`) rebuilds. Callers get a fresh array per call over shared, deep-frozen messages: mutating logged history through a projection is unrepresentable (it throws), replacing the old clone-per-call isolation. External reconstructors fold the same public function over a log prefix, so no two paths can disagree.
|
||||
|
||||
`EpochHeader` records the request's non-history state: call config, rendered system prompt, tool schemas, and session prefix, with empty values canonicalized to absence. `request/header` writes a full initial, resume, or fallback snapshot. `request/header-delta` encodes system changes by common-prefix/suffix line trim, tools by name-keyed additions/removals/changes, and config or prefix by full replacement. `foldRequestHeader`, `diffHeader`, and `applyHeaderDelta` are the pure codec. Each loop instance writes a snapshot on its first request to anchor process boundaries. Deltas are only an optimization: the writer verifies round-trip equality and falls back to a full snapshot for unrepresentable changes such as pure tool reordering.
|
||||
`EpochHeader` records the request's non-history state: call config, rendered system prompt, tool schemas, and session prefix, with empty values canonicalized to absence. `request/header` always writes a full snapshot: the first loop instance uses reason `initial`, later instances use `resume`, and an in-instance change uses `change`. `foldRequestHeader` selects the latest snapshot. Legacy `request/header-delta` events and the removed `fallback` reason are rejected when appended or loaded.
|
||||
|
||||
Each step rebuilds prompt assembly. On the instance's first step, `agent/session-prefix` extends a frozen empty seed with request-only opener messages; the result is frozen and cached for that loop instance. `agent/pre-step` then receives the composed prefix before messages are snapshotted immediately ahead of `step/start`. The first call config starts from explicit `AgentOptions`, preserving fork overrides and resume reconfiguration; later calls start from the folded header. `agent/request` may replace only that frozen config seed, while model-visible content enters through logged channels. The loop records the owed header event—the prefix's only durable home—builds `GenerateOptions` from prefix, snapshot, and header, and deep-freezes it while leaving `AbortSignal` live. Per-instance state is only the cached prefix and whether its anchoring snapshot has been written.
|
||||
|
||||
**`step/start` is the reconstruction boundary.** A step derives messages from events before that sequence. Injection after the snapshot joins the next request, and reentrant appends are rejected during event publication. `agent/pre-step` is the seam for content needed by the current request. Header reconstruction folds through the step's own `request/header*` event, or carries the prior fold when no new header is written.
|
||||
**`step/start` is the reconstruction boundary.** A step derives messages from events before that sequence. Injection after the snapshot joins the next request, and reentrant appends are rejected during event publication. `agent/pre-step` is the seam for content needed by the current request. Header reconstruction selects the step's `request/header`, or carries the prior snapshot when no new header is written.
|
||||
|
||||
**Enforcement.** In development, `dsh-invariants` independently rebuilds each loop request through a fresh `Session`, so the live cache cannot vouch for itself, then compares messages and folded header fields at `llm/stream`. Loop requests are identified by their frozen shape and session id; direct one-shots are excluded. Correctness depends on sequence-bounded reconstruction rather than listener order. A with-key e2e requires positive cache-read tokens after the first request; per-step usage is the production signal, and a header change or compaction appears as a cache-read drop on the next step.
|
||||
|
||||
@@ -39,15 +39,16 @@ Like MiniCode, the conversation advances append-only and resets only when model-
|
||||
- **Per-call request scalars** (a freely mutable config handed to each `agent/request` dispatch): a listener flips the model per call with zero accounting, silently abandoning the provider cache this design exists to protect. Config is per-conversation logged state; the waterfall proposes, the log records.
|
||||
- **Detect-and-report** (compare consecutive requests, warn on divergence): catches violations after the fact; a violating request is still constructible and ships. Rejected for interface-level unrepresentability.
|
||||
- **Event-driven assembly** (re-render only on change signals): a missed-signal bug class — a tool registered mid-session emits `tools/change`, not `system-prompt/change`, and a third-party provider may emit nothing. Per-step render + value compare is robust with zero signal discipline.
|
||||
- **Narrative fields on the header events** (a `reason`/`changed` list on deltas): derivable by diffing consecutive events — one home per fact; snapshots carry a reason because an anchor's cause is NOT derivable from the data.
|
||||
- **A custom header-delta codec** (system line edits, name-keyed tool edits, whole config/prefix replacements): reduced repeated bytes but duplicated the representation and its diff/apply/fallback machinery. Full snapshots retain one replay representation.
|
||||
- **Narrative changed-field lists on header snapshots**: derivable by comparing consecutive snapshots. The `reason` remains because an instance boundary is not derivable from the snapshot values.
|
||||
|
||||
## Consequences
|
||||
|
||||
- A request that is not explained by the log cannot be constructed by accident — not by the loop, not by a listener; mutating a built request throws; every header change is a durable, diffable log event.
|
||||
- Choosing between the advisory channels is a change-frequency decision, and the design makes the stable one structural: an `agent/session-prefix` contribution is composed once per loop instance and reused verbatim, so it extends the cacheable prefix at zero marginal cost and CANNOT bust the provider cache mid-session; content that changes mid-session flows through the append-only history channels — `agent.inject()` and tool/prompt-submit `additionalContexts` — each a durable `context/message` paid once and prefix-cached thereafter, at the price of accumulating in history and the log. Route session-frozen openers to the prefix and change notices to the history channels; a per-step request-only tail slot was deliberately dropped (no consumer, and a durable append covers every current update pattern).
|
||||
- What still costs full price at the provider is inherent and logged: compaction (its `compact/*` events and replace node), a real prompt/tool change (`request/header-delta`), a config switch (ditto), a process boundary with drift (`'resume'` snapshot differing from its predecessor). The provider's own reasoning-content exclusion is managed server-side.
|
||||
- What still costs full price at the provider is inherent and logged: compaction (its `compact/*` events and replacement entry), a real prompt, tool, or config change (`request/header` with reason `change`), or a process boundary with drift (a differing `resume` snapshot). The provider's own reasoning-content exclusion is managed server-side.
|
||||
- The `step/start`-listener behavior change (above) is the one observable semantics change for plugins; `agent/pre-step` is the current-request seam.
|
||||
- Tool-result trimming (planned) needs no new mechanism: a logged single-node surface replace (`start === end`) carrying a trimmed `tool/result` under the same `callId` — compaction-family, replay-correct, cache-bust batched by the same pressure logic.
|
||||
- Session logs grow one `request/header` snapshot per conversation (system + tool schemas: the dominant term), plus deltas on real changes — small next to `assistant/chunk` volume; `SESSION_FORMAT_VERSION` stays `0` (pre-release churn is absorbed, backends reject-not-migrate).
|
||||
- Tool-result trimming (planned) needs no new mechanism: a logged single-entry surface replace (`start === end`) carrying a trimmed `tool/result` under the same `callId` — compaction-family, replay-correct, cache-bust batched by the same pressure logic.
|
||||
- Session logs grow one `request/header` snapshot per loop instance plus snapshots on real changes. This is larger than a delta codec but small beside chunk-heavy logs and retains one replay representation. `SESSION_FORMAT_VERSION` stays `0`; legacy delta events are rejected rather than migrated.
|
||||
- Snapshot goldens changed once (every transcript gains its header events); the fs-writing fixtures are stored in the normalized authored form with cwd-relative tool arguments, because replay only round-trips cwd-independent argument paths.
|
||||
- FIXME(call-config-shape): revisit `LlmCallConfig`'s exact field set — which fields are genuinely epoch-level for cache purposes (`model` certainly; the sampling scalars sit there out of caution), and where provider-specific extras (reasoning options, extra body params) belong when an adapter needs them.
|
||||
|
||||
+2
-2
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-07-15-llm-model-catalog-and-acp-selection.md: bacfe180faf4d600d027a7aab6073130012b6022
|
||||
2026-07-15-llm-model-catalog-and-acp-selection.zh.md: 7e4bbf835a50c31c28bc40e3030425455390b598
|
||||
2026-07-15-llm-model-catalog-and-acp-selection.md: d84fe9fdb75bd2d28a00269c84d29c4223798253
|
||||
2026-07-15-llm-model-catalog-and-acp-selection.zh.md: 019819c4aa5ab4ad281b5b32daa76a004c9d6466
|
||||
|
||||
@@ -36,7 +36,7 @@ The session's current target is added to the displayed options when its adapter
|
||||
|
||||
Agent setup installs scoped `system-prompt/assemble` and `agent/request` listeners. Prompt assembly snapshots the selected pair once per step, overwrites the assembled `provider` and `model` variables after downstream prompt listeners, and the request listener applies that same snapshot after downstream request listeners. A selection during asynchronous assembly therefore starts on the next step rather than splitting prompt text from routing. Other call-config fields remain untouched.
|
||||
|
||||
The request header remains the durable source of truth. When a selected target is actually used, the existing `request/header` or `request/header-delta` event records it. `session/load` initializes the ACP selection from the folded last request header before falling back to bridge config. A selection that is never used by a request is intentionally in-memory only because it never became model-visible state.
|
||||
The request header remains the durable source of truth. When a selected target is actually used, the existing full `request/header` snapshot records it. `session/load` initializes the ACP selection from the folded last request header before falling back to bridge config. A selection that is never used by a request is intentionally in-memory only because it never became model-visible state.
|
||||
|
||||
ACP's experimental `providers/*` capability is not used. That draft surface configures provider base URLs, protocols, and headers, including secrets; it does not enumerate models and would give the UI authority to rewrite deployment-owned adapter configuration.
|
||||
|
||||
|
||||
+1
-1
@@ -36,7 +36,7 @@ ACP 选择还必须保留提供方维度。同一个模型 ID 可能存在于多
|
||||
|
||||
Agent setup 会安装作用域内的 `system-prompt/assemble` 与 `agent/request` 监听器。Prompt 组装为每个 step 只快照一次选中的字段组合，在下游 prompt 监听器完成后覆盖组装结果中的 `provider` 与 `model` 变量；请求监听器则在下游请求监听器完成后应用同一个快照。因此，异步组装期间发生的选择会从下一个 step 生效，不会导致 prompt 文本与路由分裂。其他调用配置字段保持不变。
|
||||
|
||||
请求头仍是持久化事实来源。当选中目标被实际使用时，现有 `request/header` 或 `request/header-delta` 事件会记录它。`session/load` 先从折叠后的最后请求头初始化 ACP 选择，再回退到 bridge 配置。一个从未被请求使用的选择只保留在内存中，因为它从未成为模型可见状态。
|
||||
请求头仍是持久化事实来源。当选中目标被实际使用时，现有的完整 `request/header` 快照会记录它。`session/load` 先从折叠后的最后请求头初始化 ACP 选择，再回退到 bridge 配置。一个从未被请求使用的选择只保留在内存中，因为它从未成为模型可见状态。
|
||||
|
||||
本功能不使用 ACP 的实验性 `providers/*` 能力。该草案接口配置提供方 base URL、协议和 headers，其中可能包含密钥；它不枚举模型，并且会赋予 UI 改写部署所有的适配器配置的权力。
|
||||
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-07-15-replay-token-meter-service.md: 85e68217544ff1bf12d1c24ce94ab35444052ec4
|
||||
2026-07-15-replay-token-meter-service.zh.md: a7feeb99103114a12984cb58d55f4cb8fe1248d6
|
||||
2026-07-15-replay-token-meter-service.md: 34df0383d1b8ae8047c4283eef3800de772c3cae
|
||||
2026-07-15-replay-token-meter-service.zh.md: 51f319f3c473fe247791e69133eef4280b768002
|
||||
|
||||
@@ -20,7 +20,7 @@ The service has one `contextWindow`, defaulting to 128,000 tokens and configurab
|
||||
|
||||
### Per-session replay folds
|
||||
|
||||
Each session owns one isolated incremental fold. Active folds advance from `session/event`; every read catches up through the durable tail, so listener ordering, seeded sessions, and service reload do not change the answer. The fold tracks canonical request headers and deltas, step boundaries, surface appends and replacements, assistant usage, and assistant-chunk provenance. A malformed next event fails transactionally and remains unread rather than partially mutating state.
|
||||
Each session owns one isolated incremental fold. Active folds advance from `session/event`; every read catches up through the durable tail, so listener ordering, seeded sessions, and service reload do not change the answer. The fold tracks canonical full request-header snapshots, step boundaries, surface appends and replacements, assistant usage, and assistant-chunk provenance. A malformed next event fails transactionally and remains unread rather than partially mutating state.
|
||||
|
||||
`measure(session, requestHeader?)` synchronizes the fold once and returns scalar pressure together with positional per-node prices. `totalTokens` remains request-and-response pressure; `surfaceTokens` is the surface-only heuristic total and equals the sum of `nodes[].tokens`. A `requestHeader` override changes pressure pricing only, while the surface fields always describe the current session. `estimateMessage(message)` applies the fixed heuristic without session state. Each result is one detached, deeply immutable snapshot carrying one `logRevision`. Every measurement clones the current nodes and is therefore O(surface).
|
||||
|
||||
|
||||
@@ -20,7 +20,7 @@ Status: implemented
|
||||
|
||||
### 逐会话回放折叠
|
||||
|
||||
每个会话都有一个隔离的增量折叠。活跃折叠通过 `session/event` 前进；每次读取都会追到持久日志尾部，因此监听器顺序、种子会话与服务重载不会改变答案。折叠跟踪规范请求头及其增量、步骤边界、表层追加与替换、assistant usage，以及 assistant 分片来源。下一个畸形事件会以事务方式失败并保持未读，不会让状态只修改一半。
|
||||
每个会话都有一个隔离的增量折叠。活跃折叠通过 `session/event` 前进；每次读取都会追到持久日志尾部，因此监听器顺序、种子会话与服务重载不会改变答案。折叠跟踪规范的完整请求头快照、步骤边界、表层追加与替换、assistant usage，以及 assistant 分片来源。下一个畸形事件会以事务方式失败并保持未读，不会让状态只修改一半。
|
||||
|
||||
`measure(session, requestHeader?)` 只同步一次折叠，并在返回标量压力的同时给出逐位置节点价格。`totalTokens` 仍表示请求与响应压力；`surfaceTokens` 是仅针对表层的启发式总量，并等于 `nodes[].tokens` 之和。`requestHeader` 覆盖只改变压力定价，表层字段始终描述当前会话。`estimateMessage(message)` 不依赖会话状态，直接应用固定启发式规则。每个结果都是一个分离且深度不可变的快照，只携带一个 `logRevision`。每次计量都会复制当前节点，因此成本为 O(surface)。
|
||||
|
||||
|
||||
Reference in New Issue
Block a user