refactor(agent): unify sourced message delivery

2026-07-24 22:38:50 +08:00
parent 009d113e0e
commit 992cf894af
197 changed files with 1890 additions and 2132 deletions
@@ -2,5 +2,5 @@
 # side as of the last confirmed-consistent state. Both languages carry equal authority;
 # after editing either side, bring the other along and re-record with:
 #   pnpm run verify-translation-pairing --write
-2026-07-21-compaction-summary-prefix-cache-reuse.md: 490eb57a5891bf9cd0799c5d49d25d4e9838041f
-2026-07-21-compaction-summary-prefix-cache-reuse.zh.md: 02412ff07e87e12c7e7de00b5c69e1282433f735
+2026-07-21-compaction-summary-prefix-cache-reuse.md: d05d25cfa7c3984ce0ce75c38068a91a0e07dfe8
+2026-07-21-compaction-summary-prefix-cache-reuse.zh.md: edf9de6fe5388d75612946bfb05c4383d1856102
@@ -6,7 +6,7 @@ English | [中文](2026-07-21-compaction-summary-prefix-cache-reuse.zh.md)

 ## Problem

-Automatic compaction fires mid-conversation, right after the loop has warmed the provider's KV cache with the last routed request (`system` + `tools` + `messagePrefix` + derived history). The default summarizer then issued a *separate* auxiliary request whose prefix shared nothing with that warm request: a bespoke summarizer `system` prompt followed by the older history flattened to a single rendered transcript string. A provider caches on the request's leading token sequence, so a first token that differs — a different system prompt — invalidates the entire cached prefix. Every compaction therefore paid full prompt-processing cost for the whole replayed history twice: once for the conversation request that tripped pressure, and again for the summarization call, defeating the cache exactly when the conversation is largest.
+Automatic compaction fires mid-conversation, right after the loop has warmed the provider's KV cache with the last routed request (`system` + `tools` + derived history). The default summarizer then issued a *separate* auxiliary request whose prefix shared nothing with that warm request: a bespoke summarizer `system` prompt followed by the older history flattened to a single rendered transcript string. A provider caches on the request's leading token sequence, so a first token that differs — a different system prompt — invalidates the entire cached prefix. Every compaction therefore paid full prompt-processing cost for the whole replayed history twice: once for the conversation request that tripped pressure, and again for the summarization call, defeating the cache exactly when the conversation is largest.

 ## Decision

@@ -14,7 +14,7 @@ The summarization directive moves from the **front** of the request (a fresh `sy

 ### `SummarizationInput` carries the replayed prefix, not a rendered string

-`summarize()` (and the internal `summarizeWithLlm`) take a `SummarizationInput` — `{ system?, tools?, messages }` — instead of a flat transcript string. `region.ts` builds it from `session.requestHeader()` (the durable `system`, `tools`, and `messagePrefix`) plus the shadowed region mapped through `session.deriveEventMessage`, which yields byte-identical `Message` objects to what `deriveMessages()` folded into the routed request. `summarizeWithLlm` forwards `system` and `tools` onto `GenerateOptions` and sends `[...input.messages, { role: 'user', content: COMPACTION_INSTRUCTION }]`. `tools` ride along even though the summarizer never calls one: dropping them would shorten the token sequence and break alignment with the cached request.
+`summarize()` (and the internal `summarizeWithLlm`) take a `SummarizationInput` — `{ system?, tools?, messages }` — instead of a flat transcript string. `region.ts` builds it from `session.requestHeader()` (the durable `system` and `tools`) plus the shadowed region mapped through `session.deriveEventMessage`, which yields byte-identical `Message` objects to what `deriveMessages()` folded into the routed request. `summarizeWithLlm` forwards `system` and `tools` onto `GenerateOptions` and sends `[...input.messages, { role: 'user', content: COMPACTION_INSTRUCTION }]`. `tools` ride along even though the summarizer never calls one: dropping them would shorten the token sequence and break alignment with the cached request.

 ### The instruction is a trailing user message

@@ -27,7 +27,7 @@ Auto-compaction always anchors at the surface head, so the shadowed region is th
 ## Alternatives considered

 - **Keep the summarizer system prompt but reuse the rest** — rejected: the system slot is the very first token region a provider caches on, so a distinct summarizer system prompt invalidates the whole prefix regardless of what follows. Only moving the directive off the front recovers the cache.
- **Send only the shadowed region without the `system`/`tools`/`messagePrefix` head** — rejected: a shorter or differently-headed sequence still diverges from the cached request at the first token, so it caches no better while losing the framing the summary needs.
+- **Send only the shadowed region without the `system`/`tools` head** — rejected: a differently-headed sequence still diverges from the cached request at the first token, so it caches no better while losing the framing the summary needs.
 - **Omit `tools` from the summarization request** (the model never calls one) — rejected: tool schemas are part of the cached token sequence; omitting them misaligns every following token and defeats reuse.
 - **A dedicated `assistant/chunk`-emitting summarization sub-session for snapshot replay** — out of scope here; the replay gap predates this change and is tracked in the [compaction-seam note](../feature/2026-06-18-compaction-capability-seam.md).

@@ -6,7 +6,7 @@ Status: implemented

 ## Problem

-自动压缩（compaction）在对话中途触发，恰好在循环用最后一个已路由请求（`system` + `tools` + `messagePrefix` + 派生历史）预热了提供方的 KV 缓存之后。随后默认摘要器发出一个*独立的*辅助请求，其前缀与那个已预热请求没有任何共享部分：一个专门的摘要器 `system` 提示词，后接被拍平成单个渲染后 transcript（文本记录）字符串的较早历史。提供方基于请求起始的 token 序列做缓存，因此第一个 token 只要不同（即一个不同的系统提示词），整个已缓存前缀就会失效。于是每次压缩都要为整段回放的历史付出两次完整的提示词处理成本：一次用于触发压力的对话请求，另一次用于摘要调用，恰好在对话最大时让缓存失去作用。
+自动压缩（compaction）在对话中途触发，恰好在循环用最后一个已路由请求（`system` + `tools` + 派生历史）预热了提供方的 KV 缓存之后。随后默认摘要器发出一个*独立的*辅助请求，其前缀与那个已预热请求没有任何共享部分：一个专门的摘要器 `system` 提示词，后接被拍平成单个渲染后 transcript（文本记录）字符串的较早历史。提供方基于请求起始的 token 序列做缓存，因此第一个 token 只要不同（即一个不同的系统提示词），整个已缓存前缀就会失效。于是每次压缩都要为整段回放的历史付出两次完整的提示词处理成本：一次用于触发压力的对话请求，另一次用于摘要调用，恰好在对话最大时让缓存失去作用。

 ## Decision

@@ -14,7 +14,7 @@ Status: implemented

 ### `SummarizationInput` 携带回放的前缀，而非渲染后的字符串

-`summarize()`（以及内部的 `summarizeWithLlm`）接受一个 `SummarizationInput`（`{ system?, tools?, messages }`）而不是一个扁平的 transcript 字符串。`region.ts` 用 `session.requestHeader()`（持久的 `system`、`tools` 和 `messagePrefix`）加上经 `session.deriveEventMessage` 映射的被遮蔽区域来构建它，后者产出与 `deriveMessages()` 折叠进已路由请求的内容字节级一致的 `Message` 对象。`summarizeWithLlm` 把 `system` 和 `tools` 转发到 `GenerateOptions`，并发送 `[...input.messages, { role: 'user', content: COMPACTION_INSTRUCTION }]`。`tools` 会一同带上，即便摘要器从不调用任何工具：丢弃它们会缩短 token 序列，破坏与已缓存请求的对齐。
+`summarize()`（以及内部的 `summarizeWithLlm`）接受一个 `SummarizationInput`（`{ system?, tools?, messages }`）而不是一个扁平的 transcript 字符串。`region.ts` 用 `session.requestHeader()`（持久的 `system` 和 `tools`）加上经 `session.deriveEventMessage` 映射的被遮蔽区域来构建它，后者产出与 `deriveMessages()` 折叠进已路由请求的内容字节级一致的 `Message` 对象。`summarizeWithLlm` 把 `system` 和 `tools` 转发到 `GenerateOptions`，并发送 `[...input.messages, { role: 'user', content: COMPACTION_INSTRUCTION }]`。`tools` 会一同带上，即便摘要器从不调用任何工具：丢弃它们会缩短 token 序列，破坏与已缓存请求的对齐。

 ### 指令是一条尾部 user 消息

@@ -27,7 +27,7 @@ Status: implemented
 ## Alternatives considered

 - **保留摘要器系统提示词但复用其余部分**——否决：system 槽位正是提供方最先做缓存的 token 区域，因此一个不同的摘要器系统提示词无论后面跟着什么都会使整个前缀失效。只有把指令移离前端才能恢复缓存。
- **只发送被遮蔽区域而不带 `system`/`tools`/`messagePrefix` 头部**——否决：更短或头部不同的序列在第一个 token 处仍然与已缓存请求分叉，因此缓存效果并不更好，反而丢失了摘要所需的框架。
+- **只发送被遮蔽区域而不带 `system`/`tools` 头部**——否决：头部不同的序列在第一个 token 处仍然与已缓存请求分叉，因此缓存效果并不更好，反而丢失了摘要所需的框架。
 - **从摘要请求中省略 `tools`**（模型从不调用任何工具）——否决：工具 schema 是已缓存 token 序列的一部分；省略它们会让后续每个 token 失去对齐，破坏复用。
 - **为快照回放专门建立一个发出 `assistant/chunk` 的摘要子会话**——此处超出范围；该回放缺口早于本次改动，记录在 [compaction-seam Agent Note](../feature/2026-06-18-compaction-capability-seam.md) 中。