fix(compact-basic): replay conversation prefix so summarization reuses KV cache
Automatic compaction fires mid-conversation, right after the loop warmed the provider's KV cache with the last routed request. The default summarizer then issued a separate request whose prefix shared nothing with that warm request — a bespoke summarizer system prompt followed by the older history flattened to one rendered transcript string — so a differing first token invalidated the entire cached prefix and every compaction re-processed the whole replayed history twice. Move the compaction directive from the FRONT (a fresh system prompt) to the END (a trailing user message), and replay the last routed request's own system prompt, tools, message prefix, and shadowed-region messages verbatim via session.requestHeader() + deriveEventMessage. The auxiliary call is now a genuine prefix-extension of the warm request, so the provider reuses the cached tokens up to the trailing instruction. SummarizationInput carries the replayed prefix instead of a flat string; the now-unused renderTranscript/renderContentBlocks path is removed with its spec. Cache reuse is best-effort (head compaction guarantees a hit; a mid-range compaction or a differently-routed summarizer forgoes it), correctness is not.
This commit is contained in:
@@ -17,6 +17,7 @@ import type {} from '@deepseek-ai/dsh-compact-tool-result-prune'
|
||||
import { resolveConfig } from './config.ts'
|
||||
import { compactSurfaceRegion, selectCompactableRange } from './region.ts'
|
||||
import { summarizeWithLlm } from './summarizer.ts'
|
||||
import type { SummarizationInput } from './summarizer.ts'
|
||||
import type {
|
||||
BasicCompactConfig,
|
||||
ResolvedConfig,
|
||||
@@ -135,19 +136,21 @@ export class BasicCompactService extends CompactService {
|
||||
}
|
||||
|
||||
/**
|
||||
* Summarize a rendered region through a direct one-shot `ctx.llm.stream()`
|
||||
* call. Override this sole hook for a template or remote summarizer.
|
||||
* @param text - plain-text conversation region to condense.
|
||||
* Summarize the replayed conversation region through a direct one-shot
|
||||
* `ctx.llm.stream()` call whose prefix reuses the conversation's own system
|
||||
* prompt, tools, and messages so the provider's KV cache is not invalidated.
|
||||
* Override this sole hook for a template or remote summarizer.
|
||||
* @param input - replayed conversation prefix (system, tools, and leading messages) to condense.
|
||||
* @param agent - supplies routed-model history, fallback model, and session id.
|
||||
* @param signal - optional cancellation forwarded to the adapter.
|
||||
* @returns safe text summary blocks and exact auxiliary-call provenance.
|
||||
*/
|
||||
protected async summarize(
|
||||
text: string,
|
||||
input: SummarizationInput,
|
||||
agent: Agent,
|
||||
signal?: AbortSignal,
|
||||
): Promise<{ summary: ContentBlock[]; provider: string; model: string; maxTokens?: number }> {
|
||||
return summarizeWithLlm(this.ctx, this.config, text, agent, signal)
|
||||
return summarizeWithLlm(this.ctx, this.config, input, agent, signal)
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -236,7 +239,7 @@ export class BasicCompactService extends CompactService {
|
||||
const session = agent.session
|
||||
return compactSurfaceRegion({
|
||||
meter: this.ctx.tokenMeter,
|
||||
summarize: (text, owner, abort) => this.summarize(text, owner, abort),
|
||||
summarize: (input, owner, abort) => this.summarize(input, owner, abort),
|
||||
}, session, start, end, agent, signal)
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user