feat(compact): recover context overflow (PR3 phase 2)

This commit is contained in:
Hypatia May
2026-07-15 16:50:44 +08:00
parent e8d066f750
commit 12484104c8
46 changed files with 913 additions and 242 deletions
+10 -7
View File
@@ -8,13 +8,14 @@ This is the implementation tier of the compaction capability — see the [interf
This backend owns the compaction policy:
- **Measurement** — the effective conversation model's `ModelTokenMeter` prices the provisional request envelope and current surface at one consumed-log revision. The current prompt and prefix override their logged values; the pre-step boundary reuses logged tools and call config.
- **Measurement** — the latest durable routed request model's `ModelTokenMeter` prices the canonical logged envelope and current surface at one consumed-log revision. Post-step pressure therefore includes the actual system prompt, tools, prefix, routing, assistant completion, tool results, buffered context, and steering.
- **Retention** — compact the oldest whole surface units while preserving a recent tail and balanced tool-call/result cuts through the [`dsh-compact` boundary helpers](../compact/README.md#tool-pairing-boundaries). Turn boundaries do not protect old steps inside a runaway turn. An open indivisible tail declines until it closes; a single unit larger than the budget remains out of scope.
- **Convergence** — retry head-checkpoint compaction up to `compactionRetries`; reject a summary that does not shrink its source, and throw if retries cannot return below threshold.
- **Summarization** — a direct `llm/stream` call uses the configured model and cap without running the loop-only `agent/request` seam. The input transcript preserves non-text blocks as tagged placeholders; only returned text enters the checkpoint, excluding reasoning and tool calls that would leak private reasoning or create an orphaned call.
- **Framing** — the replacement user message marks established checkpoint context with `<compacted-summary>` tags. The raw summary remains on the provenance event, and later automatic cycles merge the prior checkpoint.
- **Lifecycle** — `compactRegion()` requires its agent to own the exact target session and rejects mismatch before resolution or mutation; a valid call records its start, summary, replacement, and end. The serial `agent/pre-step` listener checks pressure before every step, outside an open step, so a tool-heavy turn remains compactable and the loop derives history once after mutation.
- **Failure handling** — an unmatched `compact/start` is an inert crash marker because no replacement landed. Recoverable failure records an error end and leaves the surface unchanged.
- **Lifecycle** — `compactRegion()` requires its agent to own the exact target session and rejects mismatch before resolution or mutation; a valid call records its start, summary, replacement, and end. The serial `agent/post-step` listener checks pressure after successful output and tool work are durable but before `step/end`. Canonical provider overflow is handled through `agent/request-error` after the failed step closes.
- **Overflow recovery** — below-threshold overflow bypasses normal retention and attempts one maximal balanced head reduction while leaving the newest indivisible unit. Retry is authorized only when `surface.replaceGeneration` advances; no range, no replacement, recovery failure, an exhausted cap, cancellation, or an unknown/noncanonical error preserves the original provider failure.
- **Failure handling** — an unmatched `compact/start` is an inert crash marker because no replacement landed. Operational post-step failures warn and continue, while an actually routed model without a meter profile fails the otherwise-successful turn with the typed meter error.
`summarize()` is the sole subclass hook. A template- or remote-summarizer subclass can override it while pressure, retention, provenance, shrink validation, and shadowed-token accounting stay on the conversation model's meter. The hook returns the summary blocks together with the call envelope it used (`{ summary, model, maxTokens? }`), which is logged on `compact/summary`.
@@ -29,7 +30,8 @@ Every common setting is optional. Every model known to `ctx.tokenMeter` receives
| `summarizationModel` | no (default `''`) | Empty resolves the latest logged routed model, then `AgentOptions.model`. |
| `maxTokens` | no (default `8192`) | Provider generation cap for the summarization call; may include reasoning tokens. |
| `compactionRetries` | no (default `1`) | Extra attempts after the first when pressure remains above threshold. |
| `auto` | no (default `true`) | Register the `agent/pre-step` automatic listener. Set `false` for manual-only. |
| `maxOverflowRetries` | no (default `1`) | Maximum retries after canonical context-window overflow; `0` disables recovery only. |
| `auto` | no (default `true`) | Register post-step pressure and overflow-recovery listeners. Set `false` for manual-only. |
## Usage
@@ -39,7 +41,7 @@ import { BasicCompactService } from '@deepseek-ai/dsh-compact-basic'
import TokenMeterService from '@deepseek-ai/dsh-token-meter'
export const name = 'compact-basic'
export const inject = ['llm']
export const inject = ['llm', 'tokenMeter']
export function apply(ctx: Context): void {
ctx.plugin(TokenMeterService)
@@ -53,7 +55,7 @@ Loading the plugin registers `ctx.compact`. With `auto: true` (the default) it c
### Conversation history
**What the model sees**: Before a step whose estimated envelope and history exceed the threshold, the conversation model receives the checkpoint preamble below, a blank line, `<compacted-summary>`, the data-dependent summary, and `</compacted-summary>`. This one checkpoint replaces the selected older range and is followed by the retained recent units.
**What the model sees**: After a successful step crosses the threshold, the next request receives the checkpoint preamble below, a blank line, `<compacted-summary>`, the data-dependent summary, and `</compacted-summary>`. Overflow recovery rebuilds the immediate retry from that replacement. This one checkpoint replaces the selected older range and is followed by the retained recent units.
**Token effect**: The replacement reduces future input history rather than appending a second copy. The summary remains until a later compaction replaces it; one oversized indivisible unit can still exceed the budget.
@@ -115,8 +117,9 @@ Rules:
## Known Limitations and Deferred Work
- **Pre-step sees a provisional request envelope** — the current prompt and prefix are exact, but routing and tool changes made later in `agent/request` are not logged yet. A router-only agent with no provisional model skips that check.
- **Meter accuracy follows the selected profile** — missing provider usage falls back to the token meter's configured character density and structural overhead.
- **Overflow classification is adapter-maintained** — provider wording can change; both DeepSeek adapters normalize currently recognized context-limit failures to `CONTEXT_WINDOW_EXCEEDED`.
- **Single-unit and envelope-only overflow remain outside surface compaction** — recovery cannot split one indivisible message/tool unit or shrink system/tools/prefix.
- **`compactRegion` requires an open turn** — a manual call on a fully-closed session throws ("no open turn") rather than compacting.
- **Summarization failure fails closed with full, over-budget history** — including truncation at the summarization `maxTokens`, which hidden reasoning tokens can consume; the auto path logs a warning and proceeds.
- **The summarization call has no transcript-snapshot coverage** — `dsh-llm-replay` derives calls from `assistant/chunk` events, so this chunk-less direct `ctx.llm.stream()` call cannot replay (named deferred replay infrastructure in [the seam RFC](../../../docs/rfc/implemented/feature/2026-06-18-compaction-capability-seam.md)).
+41 -17
View File
@@ -1,12 +1,12 @@
/**
* Automatic pre-step pressure listener for compact-basic.
* Automatic post-step pressure and context-overflow recovery listeners.
*
* @module @deepseek-ai/dsh-compact-basic/automatic
*/
import type { Context } from 'cordis'
import type { CompactionResult } from '@deepseek-ai/dsh-compact'
import type { Message } from '@deepseek-ai/dsh-llm'
import type { CompactionResult, CompactionTrigger } from '@deepseek-ai/dsh-compact'
import { CONTEXT_WINDOW_EXCEEDED_CODE } from '@deepseek-ai/dsh-llm'
import {
TOKEN_METER_MODEL_UNCONFIGURED,
TokenMeterError,
@@ -14,10 +14,10 @@ import {
import type { Agent } from '@deepseek-ai/dsh-agent'
interface AutomaticCompactor {
readonly config: { readonly maxOverflowRetries: number }
compactIfNeeded(
agent: Agent,
fullSystemPrompt: string,
sessionPrefix: readonly Message[],
trigger: CompactionTrigger,
signal: AbortSignal,
): Promise<CompactionResult | null>
}
@@ -31,30 +31,54 @@ export function registerAutomaticCompaction(
ctx: Context,
service: AutomaticCompactor,
): void {
ctx.on('agent/pre-step', async (
const logResult = (result: CompactionResult, trigger: string): void => {
ctx.logger.info(
`compaction (${trigger}): shadowed ${result.shadowedSeqs.length} surface nodes `
+ `(seqs ${result.shadowedRange.start}-${result.shadowedRange.end}, `
+ `~${result.shadowedTokenCount} tokens)`,
)
}
ctx.on('agent/post-step', async (
agent: Agent,
_turn: number,
_step: number,
fullSystemPrompt: string,
sessionPrefix: readonly Message[],
signal: AbortSignal,
) => {
try {
const result = await service.compactIfNeeded(agent, fullSystemPrompt, sessionPrefix, signal)
if (result !== null) {
ctx.logger.info(
`compaction: shadowed ${result.shadowedSeqs.length} surface nodes `
+ `(seqs ${result.shadowedRange.start}-${result.shadowedRange.end}, `
+ `~${result.shadowedTokenCount} tokens)`,
)
}
const result = await service.compactIfNeeded(agent, 'pressure', signal)
if (result !== null) logResult(result, 'post-step pressure')
} catch (error: unknown) {
// A named routed model without a meter profile is configuration failure,
// not an optional operational compaction miss.
if (error instanceof TokenMeterError
&& error.code === TOKEN_METER_MODEL_UNCONFIGURED) throw error
const message = error instanceof Error ? error.message : String(error)
ctx.logger.warn(`compaction failed: ${message}; proceeding with full history`)
ctx.logger.warn(`post-step compaction failed: ${message}; continuing the turn`)
}
})
ctx.on('agent/request-error', async (agent, _turn, _step, error, retryAttempt, signal, next) => {
if (error.code !== CONTEXT_WINDOW_EXCEEDED_CODE
|| retryAttempt >= service.config.maxOverflowRetries
|| signal.aborted) return next()
let generation: number
let result: CompactionResult | null
try {
generation = agent.session.surface.replaceGeneration
result = await service.compactIfNeeded(agent, 'context-overflow', signal)
} catch (recoveryError: unknown) {
const message = recoveryError instanceof Error ? recoveryError.message : String(recoveryError)
ctx.logger.warn(
`context-overflow compaction failed: ${message}; preserving the original request error`,
)
return next()
}
// eslint-disable-next-line @typescript-eslint/no-unnecessary-condition -- signal can abort while compaction is awaited.
if (signal.aborted || result === null
|| agent.session.surface.replaceGeneration <= generation) return next()
logResult(result, 'context overflow recovery')
return { action: 'retry' }
})
}
@@ -47,6 +47,7 @@ export function resolveConfig(
summarizationModel: '',
maxTokens: 8192,
compactionRetries: 1,
maxOverflowRetries: 1,
auto: true,
}, meter)
}
@@ -56,10 +57,12 @@ export function resolveConfig(
summarizationModel: config.summarizationModel ?? '',
maxTokens: config.maxTokens ?? 8192,
compactionRetries: config.compactionRetries ?? 1,
maxOverflowRetries: config.maxOverflowRetries ?? 1,
auto: config.auto ?? true,
}
assertPositiveInteger('maxTokens', resolved.maxTokens)
assertNonNegativeInteger('compactionRetries', resolved.compactionRetries)
assertNonNegativeInteger('maxOverflowRetries', resolved.maxOverflowRetries)
if (typeof resolved.summarizationModel !== 'string') {
throw new Error('BasicCompactConfig: summarizationModel must be a string')
}
+27 -37
View File
@@ -7,10 +7,9 @@
import { Context } from 'cordis'
import z from 'schemastery'
import { CompactService } from '@deepseek-ai/dsh-compact'
import type { CompactionResult } from '@deepseek-ai/dsh-compact'
import { canonicalHeader } from '@deepseek-ai/dsh-session'
import type { EpochHeader, Session } from '@deepseek-ai/dsh-session'
import type { ContentBlock, Message } from '@deepseek-ai/dsh-llm'
import type { CompactionResult, CompactionTrigger } from '@deepseek-ai/dsh-compact'
import type { Session } from '@deepseek-ai/dsh-session'
import type { ContentBlock } from '@deepseek-ai/dsh-llm'
import type { ModelTokenMeter } from '@deepseek-ai/dsh-token-meter'
import type { Agent } from '@deepseek-ai/dsh-agent'
import { registerAutomaticCompaction } from './automatic.ts'
@@ -36,24 +35,10 @@ function effectiveModel(agent: Agent): string | undefined {
return agent.session.requestHeader()?.config.model ?? agent.options.model
}
/**
* Build the provisional pre-step request envelope. Prompt and prefix are exact;
* tools and non-model call config come from the latest logged request because
* later request middleware has not run yet.
*/
function provisionalHeader(
model: string,
session: Session,
fullSystemPrompt: string,
sessionPrefix: readonly Message[],
): EpochHeader {
const latest = session.requestHeader()
return canonicalHeader({
config: latest === undefined ? { model } : { ...latest.config, model },
...fullSystemPrompt.length === 0 ? {} : { system: fullSystemPrompt },
...latest?.tools === undefined ? {} : { tools: latest.tools },
...sessionPrefix.length === 0 ? {} : { messagePrefix: [...sessionPrefix] },
})
/** Resolve the exact model durably routed for the latest provider request. */
function routedModel(session: Session): string | undefined {
const model = session.requestHeader()?.config.model
return model === undefined || model.length === 0 ? undefined : model
}
/**
@@ -75,6 +60,7 @@ export class BasicCompactService extends CompactService {
summarizationModel: z.string().default(''),
maxTokens: z.number().step(1).min(1).default(8192),
compactionRetries: z.number().step(1).min(0).default(1),
maxOverflowRetries: z.number().step(1).min(0).default(1),
auto: z.boolean().default(true),
})
@@ -106,29 +92,33 @@ export class BasicCompactService extends CompactService {
}
/**
* Check replayed pressure for the provisional pre-step envelope and compact
* a tool-balanced head until it falls below the effective model threshold.
* A genuinely model-less router-first step skips this provisional check;
* naming an unconfigured model throws the token meter's typed error.
* @param agent - agent whose session and provisional model are measured.
* @param fullSystemPrompt - current assembled system prompt override.
* @param sessionPrefix - current request-only prefix override.
* @param signal - live step cancellation signal forwarded to summarization.
* Compact for replayed post-step pressure or one provider-confirmed context
* overflow. Both triggers price the latest durable routed request model;
* overflow bypasses the normal threshold and retained-tail policy so it can
* force one useful balanced reduction.
* @param agent - agent whose latest durable routed request is measured.
* @param trigger - normal post-step pressure or context-overflow recovery.
* @param signal - live turn cancellation signal forwarded to summarization.
* @returns the latest compaction result, or `null` when no check/work applies.
*/
override async compactIfNeeded(
agent: Agent,
fullSystemPrompt: string,
sessionPrefix: readonly Message[],
trigger: CompactionTrigger,
signal: AbortSignal,
): Promise<CompactionResult | null> {
const model = effectiveModel(agent)
if (model === undefined || model.length === 0) return null
const model = routedModel(agent.session)
if (model === undefined) return null
const meter = this.ctx.tokenMeter.resolve(model)
const policy = this._modelConfig(meter)
const requestHeader = provisionalHeader(model, agent.session, fullSystemPrompt, sessionPrefix)
if (trigger === 'context-overflow') {
const surface = meter.measureSurface(agent.session)
const range = selectCompactableRange(agent.session, surface, 0)
if (range === null) return null
return this.compactRegion(agent.session, range.start, range.end, agent, signal)
}
const threshold = Math.floor(policy.contextWindow * policy.thresholdRatio)
let measurement = meter.measure(agent.session, requestHeader)
let measurement = meter.measure(agent.session)
if (measurement.totalTokens < threshold) return null
let result: CompactionResult | null = null
@@ -147,7 +137,7 @@ export class BasicCompactService extends CompactService {
break
}
result = await this.compactRegion(agent.session, range.start, range.end, agent, signal)
measurement = meter.measure(agent.session, requestHeader)
measurement = meter.measure(agent.session)
if (measurement.totalTokens < threshold) return result
}
@@ -106,7 +106,7 @@ export async function summarizeWithLlm(
if (!summary.some(block => block.text.trim().length > 0)) {
throw new Error('summarization produced no text summary content')
}
return { summary, model, maxTokens: config.maxTokens }
return { summary, model: options.model, maxTokens: config.maxTokens }
}
/**
+4 -1
View File
@@ -22,7 +22,9 @@ export interface BasicCompactConfig {
maxTokens?: number
/** Extra attempts after the first compaction when pressure remains above threshold. Defaults to `1`. */
compactionRetries?: number
/** Enable the automatic `agent/pre-step` pressure listener. Defaults to `true`. */
/** Maximum retries after canonical context overflow; `0` disables recovery. Defaults to `1`. */
maxOverflowRetries?: number
/** Enable automatic post-step pressure and overflow-recovery listeners. Defaults to `true`. */
auto?: boolean
}
@@ -32,6 +34,7 @@ export interface ResolvedConfig {
readonly summarizationModel: string
readonly maxTokens: number
readonly compactionRetries: number
readonly maxOverflowRetries: number
readonly auto: boolean
}
@@ -6,9 +6,10 @@ import BasicCompactService, {
} from '@deepseek-ai/dsh-compact-basic'
import type { BasicCompactConfig } from '@deepseek-ai/dsh-compact-basic'
import { selectCompactableRange } from '@deepseek-ai/dsh-compact-basic/src/region.ts'
import { toolPairingBalancedAfter, toolPairingBalancedBefore } from '@deepseek-ai/dsh-compact'
import type { CompactionResult } from '@deepseek-ai/dsh-compact'
import LlmService, { CallId, LlmAdapter } from '@deepseek-ai/dsh-llm'
import type { ContentBlock, GenerateOptions, Message, StreamChunk } from '@deepseek-ai/dsh-llm'
import LlmService, { CallId, CONTEXT_WINDOW_EXCEEDED_CODE, LlmAdapter } from '@deepseek-ai/dsh-llm'
import type { ContentBlock, GenerateOptions, StreamChunk } from '@deepseek-ai/dsh-llm'
import { Session, SessionId } from '@deepseek-ai/dsh-session'
import TokenMeterService, {
TOKEN_METER_MODEL_UNCONFIGURED,
@@ -43,6 +44,12 @@ function conversation(turns = 4, text = 'fixture'): Session {
source: { kind: 'user' },
}, { surfaceOp: 'append' })
session.append('step/start', { turn, step: 1 })
if (turn === 1) {
session.append('request/header', {
header: { config: { model: MODEL } },
reason: 'initial',
})
}
session.append('assistant/message', {
turn,
step: 1,
@@ -68,6 +75,12 @@ function toolConversation(): Session {
source: { kind: 'user' },
}, { surfaceOp: 'append' })
session.append('step/start', { turn, step: 1 })
if (turn === 1) {
session.append('request/header', {
header: { config: { model: MODEL } },
reason: 'initial',
})
}
session.append('assistant/message', {
turn,
step: 1,
@@ -120,11 +133,10 @@ function service(
async function compactIfNeeded(
compact: BasicCompactService,
session: Session,
trigger: 'pressure' | 'context-overflow' = 'pressure',
model: string | undefined = MODEL,
system = '',
prefix: readonly Message[] = [],
): Promise<CompactionResult | null> {
return compact.compactIfNeeded(agent(session, model), system, prefix, SIGNAL)
return compact.compactIfNeeded(agent(session, model), trigger, SIGNAL)
}
describe('compact configuration and defaults', () => {
@@ -140,6 +152,7 @@ describe('compact configuration and defaults', () => {
summarizationModel: '',
maxTokens: 8192,
compactionRetries: 1,
maxOverflowRetries: 1,
auto: true,
})
expect(resolveModelConfig(resolved, ctx.tokenMeter.resolve(MODEL))).toEqual({
@@ -176,6 +189,7 @@ describe('compact configuration and defaults', () => {
const bad = [
[{ maxTokens: 0 }, /maxTokens/],
[{ compactionRetries: -1 }, /compactionRetries/],
[{ maxOverflowRetries: -1 }, /maxOverflowRetries/],
[{ auto: 'yes' }, /auto must be a boolean/],
[{ summarizationModel: 1 }, /summarizationModel must be a string/],
[{ models: null }, /models must be an object/],
@@ -207,19 +221,57 @@ describe('pressure measurement and retention', () => {
models: { [MODEL]: { thresholdRatio: 0.5, retainTokens: 18 } },
}
it('skips the provisional check only when no routed or fallback model exists', async () => {
it('skips when no durable routed model exists instead of using AgentOptions fallback', async () => {
const compact = service(compactConfig)
const session = conversation()
expect(await compact.compactIfNeeded(agent(session), '', [], SIGNAL)).toBeNull()
const session = new Session(SessionId('headerless'))
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
await expect(compact.compactIfNeeded(agent(session, MODEL), 'pressure', SIGNAL))
.resolves.toBeNull()
expect(compact.calls).toHaveLength(0)
})
it('throws for a named unconfigured model instead of swallowing it', async () => {
const compact = service(compactConfig)
await expect(compactIfNeeded(compact, conversation(), 'missing'))
const session = conversation()
session.append('request/header', {
header: { config: { model: 'missing' } },
reason: 'resume',
})
await expect(compactIfNeeded(compact, session))
.rejects.toMatchObject({ code: TOKEN_METER_MODEL_UNCONFIGURED, model: 'missing' })
})
it('declines forced overflow when the whole surface is one indivisible tool pair', async () => {
const compact = service(compactConfig)
const session = new Session(SessionId('single-tool-pair'))
const callId = CallId('single-call')
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
session.append('step/start', { turn: 1, step: 1 })
session.append('request/header', {
header: { config: { model: MODEL } },
reason: 'initial',
})
session.append('assistant/message', {
turn: 1,
step: 1,
content: [{ type: 'tool-call', id: callId, name: 'read', arguments: '{}' }],
}, { surfaceOp: 'append' })
session.append('tool/call', { turn: 1, step: 1, callId, name: 'read', arguments: '{}' })
session.append('tool/result', {
turn: 1,
step: 1,
callId,
content: [{ type: 'text', text: 'result' }],
isError: false,
}, { surfaceOp: 'append' })
session.append('step/end', { turn: 1, step: 1 })
const generation = session.surface.replaceGeneration
await expect(compactIfNeeded(compact, session, 'context-overflow')).resolves.toBeNull()
expect(session.surface.replaceGeneration).toBe(generation)
expect(session.events.some(event => event.type === 'compact/start')).toBe(false)
})
it('does nothing below threshold and compacts a priced head above threshold', async () => {
const compact = service(compactConfig)
expect(await compactIfNeeded(compact, conversation(2))).toBeNull()
@@ -231,7 +283,7 @@ describe('pressure measurement and retention', () => {
expect(session.surface.nodes.length).toBeLessThan(8)
})
it('counts the current prompt and request prefix without putting either on the surface', async () => {
it('counts the durable routed request envelope without putting its prefix on the surface', async () => {
const compact = service({
auto: false,
models: { [MODEL]: { thresholdRatio: 0.7, retainTokens: 9 } },
@@ -239,11 +291,16 @@ describe('pressure measurement and retention', () => {
const session = conversation(2, 'x'.repeat(2_000))
expect(await compactIfNeeded(compact, session)).toBeNull()
const prefix: Message[] = [{
role: 'user',
content: [{ type: 'text', text: 'p'.repeat(10_000) }],
}]
const result = await compactIfNeeded(compact, session, MODEL, 's'.repeat(5_000), prefix)
const prefix = [{ role: 'user' as const, content: [{ type: 'text' as const, text: 'p'.repeat(10_000) }] }]
session.append('request/header', {
header: {
config: { model: MODEL },
system: 's'.repeat(5_000),
messagePrefix: prefix,
},
reason: 'resume',
})
const result = await compactIfNeeded(compact, session)
expect(result).not.toBeNull()
expect(prefix).toHaveLength(1)
expect(session.events.some(event => event.type === 'context/message')).toBe(false)
@@ -264,17 +321,26 @@ describe('pressure measurement and retention', () => {
reason: 'initial',
})
const result = await compactIfNeeded(compact, session, 'fallback')
const result = await compactIfNeeded(compact, session, 'pressure', 'fallback')
expect(result).not.toBeNull()
})
it('declines when envelope pressure is high but the surface has no compactable range', async () => {
const compact = service(compactConfig)
const empty = new Session(SessionId('empty'))
expect(await compactIfNeeded(compact, empty, MODEL, 'x'.repeat(100_000))).toBeNull()
empty.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
empty.append('request/header', {
header: { config: { model: MODEL }, system: 'x'.repeat(100_000) },
reason: 'initial',
})
expect(await compactIfNeeded(compact, empty)).toBeNull()
const retained = conversation(1)
expect(await compactIfNeeded(compact, retained, MODEL, 'x'.repeat(100_000))).toBeNull()
retained.append('request/header', {
header: { config: { model: MODEL }, system: 'x'.repeat(100_000) },
reason: 'resume',
})
expect(await compactIfNeeded(compact, retained)).toBeNull()
})
it('detects scalar/surface revision disagreement', async () => {
@@ -587,7 +653,19 @@ describe('compaction region transaction', () => {
it('requires a conversation model for pricing', async () => {
const compact = service()
const session = conversation(1)
const session = new Session(SessionId('model-less-region'))
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
session.append('user/message', {
content: [{ type: 'text', text: 'history' }],
source: { kind: 'user' },
}, { surfaceOp: 'append' })
session.append('step/start', { turn: 1, step: 1 })
session.append('assistant/message', {
turn: 1,
step: 1,
content: [{ type: 'text', text: 'answer' }],
}, { surfaceOp: 'append' })
session.append('step/end', { turn: 1, step: 1 })
const nodes = session.surface.nodes
await expect(compact.compactRegion(
session,
@@ -679,6 +757,25 @@ describe('default one-shot summarizer', () => {
expect(adapter.lastOptions?.model).toBe('routed')
})
it('records the model actually dispatched after one-shot stream routing', async () => {
const { ctx, compact } = await summarizerHarness([{ type: 'text', text: 'unused' }])
const routedAdapter = new ScriptedAdapter([{ type: 'text', text: 'routed summary' }])
ctx.llm.registerAdapter(['routed-summary-model'], routedAdapter)
ctx.on('llm/stream', (options, next) => {
options.model = 'routed-summary-model'
return next()
})
const session = conversation(3, 'large history '.repeat(500))
const nodes = session.surface.nodes
await compact.compactRegion(session, nodes[0]!.seq, nodes[3]!.seq, agent(session, MODEL), SIGNAL)
expect(session.events.findLast(event => event.type === 'compact/summary')?.data).toMatchObject({
summary: [{ type: 'text', text: 'routed summary' }],
model: 'routed-summary-model',
})
expect(routedAdapter.lastOptions?.model).toBe('routed-summary-model')
})
it('fails clearly when no summarization model can be resolved', async () => {
const ctx = new Context()
await ctx.plugin(LlmService)
@@ -717,21 +814,36 @@ describe('default one-shot summarizer', () => {
})
describe('automatic listener and loader composition', () => {
function preStep(ctx: Context, owner: Agent): Promise<unknown> {
return ctx.serial('agent/pre-step', owner, 1, 1, '', [], SIGNAL)
function postStep(ctx: Context, owner: Agent, signal = SIGNAL): Promise<unknown> {
return ctx.serial('agent/post-step', owner, 1, 1, signal)
}
it('compacts above threshold and remains idle below it', async () => {
function recover(
ctx: Context,
owner: Agent,
error: Error & { code?: string },
retryAttempt = 0,
signal = SIGNAL,
next: () => Promise<{ action: 'fail' | 'retry' }> = () => Promise.resolve({ action: 'fail' }),
): Promise<{ action: 'fail' | 'retry' }> {
return ctx.waterfall('agent/request-error', owner, 1, 1, error, retryAttempt, signal, next)
}
function overflow(message = 'provider overflow'): Error & { code: string } {
return Object.assign(new Error(message), { code: CONTEXT_WINDOW_EXCEEDED_CODE })
}
it('compacts post-step above threshold using the durable routed model and remains idle below it', async () => {
const ctx = createContext()
const compact = new TestCompactService(ctx, {
models: { [MODEL]: { thresholdRatio: 0.5, retainTokens: 18 } },
})
const pressured = conversation(4)
await preStep(ctx, agent(pressured, MODEL))
await postStep(ctx, agent(pressured, 'unconfigured-agent-fallback'))
expect(pressured.events.some(event => event.type === 'compact/summary')).toBe(true)
const small = conversation(1)
await preStep(ctx, agent(small, MODEL))
await postStep(ctx, agent(small, MODEL))
expect(small.events.some(event => event.type === 'compact/start')).toBe(false)
expect(compact.calls).toHaveLength(1)
})
@@ -746,7 +858,7 @@ describe('automatic listener and loader composition', () => {
compact.error = 'temporary failure'
const session = conversation(4)
await expect(preStep(ctx, agent(session, MODEL))).resolves.toBeUndefined()
await expect(postStep(ctx, agent(session, MODEL))).resolves.toBeUndefined()
expect(warnings).toContainEqual(expect.stringContaining('temporary failure'))
expect(session.events.some(event => event.type === 'compact/summary')).toBe(false)
})
@@ -754,21 +866,210 @@ describe('automatic listener and loader composition', () => {
it('propagates a named unknown-model configuration failure', async () => {
const ctx = createContext()
void new TestCompactService(ctx)
await expect(preStep(ctx, agent(conversation(4), 'missing'))).rejects.toMatchObject({
const session = conversation(4)
session.append('request/header', {
header: { config: { model: 'missing' } },
reason: 'resume',
})
await expect(postStep(ctx, agent(session, MODEL))).rejects.toMatchObject({
code: TOKEN_METER_MODEL_UNCONFIGURED,
model: 'missing',
})
})
it('auto:false installs no listener', async () => {
it('force-compacts below normal pressure for canonical overflow and retries only after replacement', async () => {
const ctx = createContext()
void new TestCompactService(ctx, {
models: { [MODEL]: { thresholdRatio: 1, retainTokens: 90 } },
})
const session = conversation(3)
const beforeGeneration = session.surface.replaceGeneration
const retainedSeq = session.surface.nodes.at(-1)!.seq
const threshold = 100
expect(ctx.tokenMeter.resolve(MODEL).measure(session).totalTokens).toBeLessThan(threshold)
const decision = await recover(ctx, agent(session, 'unconfigured-agent-fallback'), overflow())
expect(decision).toEqual({ action: 'retry' })
expect(session.surface.replaceGeneration).toBe(beforeGeneration + 1)
expect(session.events.some(event => event.type === 'compact/summary')).toBe(true)
expect(session.surface.nodes.some(node => node.seq === retainedSeq)).toBe(true)
})
it('preserves the newest whole tool-call/result pair during forced overflow compaction', async () => {
const ctx = createContext()
void new TestCompactService(ctx, {
models: { [MODEL]: { thresholdRatio: 1, retainTokens: 90 } },
})
const session = toolConversation()
const newestAssistant = session.surface.nodes.at(-2)!
const newestResult = session.surface.nodes.at(-1)!
expect(await recover(ctx, agent(session, MODEL), overflow())).toEqual({ action: 'retry' })
const currentAssistant = session.surface.nodes.find(node => node.seq === newestAssistant.seq)
const currentResult = session.surface.nodes.find(node => node.seq === newestResult.seq)
expect(currentAssistant).toBeDefined()
expect(currentResult).toBeDefined()
expect(toolPairingBalancedBefore(session, currentAssistant!)).toBe(true)
expect(toolPairingBalancedAfter(session, currentResult!)).toBe(true)
})
it('does not retry when a backend reports success without replacing the surface', async () => {
const ctx = createContext()
const compact = new TestCompactService(ctx)
const session = conversation(2)
const fakeResult: CompactionResult = {
startSeq: 1,
summarySeq: 2,
endSeq: 3,
summary: [{ type: 'text', text: 'fake' }],
shadowedRange: { start: 1, end: 2 },
shadowedSeqs: [1, 2],
shadowedTokenCount: 10,
}
vi.spyOn(compact, 'compactIfNeeded').mockResolvedValue(fakeResult)
expect(await recover(ctx, agent(session, MODEL), overflow())).toEqual({ action: 'fail' })
expect(session.surface.replaceGeneration).toBe(0)
})
it('delegates downstream exactly once when no replacement is available', async () => {
const ctx = createContext()
const compact = new TestCompactService(ctx)
vi.spyOn(compact, 'compactIfNeeded').mockResolvedValue(null)
const downstream = new Error('downstream recovery failed')
let calls = 0
await expect(recover(
ctx,
agent(conversation(2), MODEL),
overflow(),
0,
SIGNAL,
() => {
calls += 1
return Promise.reject(downstream)
},
)).rejects.toBe(downstream)
expect(calls).toBe(1)
})
it('preserves the original provider error when recovery throws', async () => {
const ctx = createContext()
const warnings: string[] = []
ctx.logger.warn = ((message: string) => void warnings.push(message)) as typeof ctx.logger.warn
const compact = new TestCompactService(ctx)
compact.error = new Error('summary unavailable')
const original = overflow('original provider overflow')
expect(await recover(ctx, agent(conversation(3), MODEL), original)).toEqual({ action: 'fail' })
expect(original).toMatchObject({
message: 'original provider overflow',
code: CONTEXT_WINDOW_EXCEEDED_CODE,
})
expect(warnings).toContainEqual(expect.stringContaining('preserving the original request error'))
})
it('delegates once when overflow recovery throws a non-Error value', async () => {
const ctx = createContext()
const warnings: string[] = []
ctx.logger.warn = ((message: string) => void warnings.push(message)) as typeof ctx.logger.warn
const compact = new TestCompactService(ctx)
compact.error = 'non-error recovery failure'
const session = conversation(3)
const generation = session.surface.replaceGeneration
const original = overflow('original provider failure')
let delegations = 0
const decision = await recover(ctx, agent(session, MODEL), original, 0, SIGNAL, () => {
delegations += 1
return Promise.resolve({ action: 'fail' })
})
expect(decision).toEqual({ action: 'fail' })
expect(delegations).toBe(1)
expect(session.surface.replaceGeneration).toBe(generation)
expect(original).toMatchObject({
message: 'original provider failure',
code: CONTEXT_WINDOW_EXCEEDED_CODE,
})
expect(warnings).toContainEqual(expect.stringContaining('non-error recovery failure'))
})
it('delegates once and preserves the original overflow for an unknown routed meter model', async () => {
const ctx = createContext()
void new TestCompactService(ctx)
const session = conversation(2)
session.append('request/header', {
header: { config: { model: 'unknown-routed-model' } },
reason: 'resume',
})
const original = overflow('original unknown-model overflow')
let delegations = 0
const decision = await recover(ctx, agent(session, MODEL), original, 0, SIGNAL, () => {
delegations += 1
return Promise.resolve({ action: 'fail' })
})
expect(decision).toEqual({ action: 'fail' })
expect(delegations).toBe(1)
expect(original).toMatchObject({
message: 'original unknown-model overflow',
code: CONTEXT_WINDOW_EXCEEDED_CODE,
})
})
it('honors retry caps, non-context failures, and cancellation', async () => {
const ctx = createContext()
const compact = new TestCompactService(ctx, { maxOverflowRetries: 1 })
const compactSpy = vi.spyOn(compact, 'compactIfNeeded')
const owner = agent(conversation(3), MODEL)
expect(await recover(ctx, owner, Object.assign(new Error('rate limit'), { code: 'RATE_LIMIT' })))
.toEqual({ action: 'fail' })
expect(await recover(ctx, owner, overflow(), 1)).toEqual({ action: 'fail' })
const controller = new AbortController()
controller.abort('cancelled')
expect(await recover(ctx, owner, overflow(), 0, controller.signal)).toEqual({ action: 'fail' })
expect(compactSpy).not.toHaveBeenCalled()
})
it('does not retry when cancellation lands during an awaited compaction', async () => {
const ctx = createContext()
const compact = new TestCompactService(ctx)
const controller = new AbortController()
compact.mutateDuringSummary = () => { controller.abort('cancelled during summary') }
const session = conversation(3)
const generation = session.surface.replaceGeneration
expect(await recover(ctx, agent(session, MODEL), overflow(), 0, controller.signal))
.toEqual({ action: 'fail' })
expect(session.surface.replaceGeneration).toBe(generation + 1)
})
it('maxOverflowRetries:0 disables recovery without disabling post-step pressure', async () => {
const ctx = createContext()
void new TestCompactService(ctx, {
maxOverflowRetries: 0,
models: { [MODEL]: { thresholdRatio: 0.5, retainTokens: 18 } },
})
const session = conversation(4)
await postStep(ctx, agent(session, MODEL))
const summaries = session.events.filter(event => event.type === 'compact/summary').length
expect(summaries).toBe(1)
expect(await recover(ctx, agent(session, MODEL), overflow())).toEqual({ action: 'fail' })
expect(session.events.filter(event => event.type === 'compact/summary')).toHaveLength(summaries)
})
it('auto:false installs neither automatic listener', async () => {
const ctx = createContext()
void new TestCompactService(ctx, {
auto: false,
models: { [MODEL]: { thresholdRatio: 0.5, retainTokens: 18 } },
})
const session = conversation(4)
await preStep(ctx, agent(session, MODEL))
await postStep(ctx, agent(session, MODEL))
expect(session.events.some(event => event.type === 'compact/start')).toBe(false)
expect(await recover(ctx, agent(session, MODEL), overflow())).toEqual({ action: 'fail' })
})
it('loads and disposes the real zero-config service stack', async () => {
@@ -797,8 +1098,9 @@ describe('automatic listener and loader composition', () => {
await fiber.dispose()
const session = conversation(4)
await preStep(ctx, agent(session, MODEL))
await postStep(ctx, agent(session, MODEL))
expect(session.events.some(event => event.type === 'compact/start')).toBe(false)
expect(await recover(ctx, agent(session, MODEL), overflow())).toEqual({ action: 'fail' })
})
})
@@ -1,7 +1,7 @@
import { describe, expect, it } from 'vitest'
import { Context } from 'cordis'
import { toolPairingBalancedAfter, toolPairingBalancedBefore } from '@deepseek-ai/dsh-compact'
import LlmService from '@deepseek-ai/dsh-llm'
import LlmService, { CONTEXT_WINDOW_EXCEEDED_CODE, LlmError } from '@deepseek-ai/dsh-llm'
import type { ContentBlock, GenerateOptions, StreamChunk } from '@deepseek-ai/dsh-llm'
import { CallId, LlmAdapter } from '@deepseek-ai/dsh-llm'
import SessionStore from '@deepseek-ai/dsh-session'
@@ -53,6 +53,45 @@ class StepwiseToolAdapter extends LlmAdapter {
}
}
/** First conversation request overflows, then the rebuilt retry succeeds. */
class OverflowRecoveryAdapter extends LlmAdapter {
readonly conversationRequests: GenerateOptions[] = []
readonly summaryRequests: GenerateOptions[] = []
constructor(private readonly delivery: 'thrown' | 'in-band') {
super()
}
override async * stream(options: GenerateOptions): AsyncIterable<StreamChunk> {
if (options.system?.includes('You are a compaction engine')) {
this.summaryRequests.push(options)
yield { type: 'block-start', index: 0, blockType: 'text' }
yield { type: 'block-end', index: 0, block: { type: 'text', text: 'RECOVERY CHECKPOINT' } }
yield { type: 'finish', reason: { kind: 'stop' } }
return
}
this.conversationRequests.push(options)
if (this.conversationRequests.length === 1) {
if (this.delivery === 'thrown') {
throw new LlmError('request too large for model context', CONTEXT_WINDOW_EXCEEDED_CODE, 400)
}
yield {
type: 'finish',
reason: {
kind: 'error',
message: 'request too large for model context',
code: CONTEXT_WINDOW_EXCEEDED_CODE,
},
}
return
}
yield { type: 'block-start', index: 0, blockType: 'text' }
yield { type: 'block-end', index: 0, block: { type: 'text', text: 'recovered' } }
yield { type: 'finish', reason: { kind: 'stop' } }
}
}
async function harness(toolSteps: number): Promise<{ ctx: Context; compact: ReproCompactService }> {
const ctx = new Context()
await ctx.plugin(LlmService)
@@ -98,6 +137,53 @@ function waitForIdle(ctx: Context, agent: ReactLoopAgent): Promise<void> {
}
describe('CBR-001: a real-loop checkpoint is a valid boundary on both sides', () => {
it('uses the model actually routed by agent/request for post-step pressure', async () => {
const { ctx } = await harness(8)
ctx.on('agent/request', async (_agent, _turn, _step, config) => ({ ...config, model: 'mock' }))
try {
const agent = ctx.agentLoop.create(AgentId('routed-pressure'), {
model: 'unconfigured-agent-fallback',
})
agent.send([{ type: 'text', text: 'do a routed multi-step task' }])
await waitForIdle(ctx, agent)
expect(agent.session.requestHeader()?.config.model).toBe('mock')
expect(agent.session.events.some(event => event.type === 'compact/summary')).toBe(true)
expect(agent.session.events.at(-1)).toMatchObject({
type: 'turn/end',
data: { reason: { kind: 'completed' } },
})
} finally {
await ctx.fiber.dispose()
}
})
it('runs automatic pressure after the current tool result and before step/end', async () => {
const { ctx } = await harness(4)
try {
const agent = ctx.agentLoop.create(AgentId('post-step-order'), { model: 'mock' })
agent.send([{ type: 'text', text: 'do tool work' }])
await waitForIdle(ctx, agent)
const events = [...agent.session.events]
const compactStart = events.find(event => event.type === 'compact/start')
expect(compactStart).toBeDefined()
const precedingResult = events.findLast(event =>
event.type === 'tool/result' && event.seq < compactStart!.seq,
)
if (precedingResult?.type !== 'tool/result') throw new Error('expected a durable tool result before compaction')
const stepEnd = events.find(event =>
event.type === 'step/end'
&& event.data.step === precedingResult.data.step
&& event.seq > compactStart!.seq,
)
expect(precedingResult.seq).toBeLessThan(compactStart!.seq)
expect(compactStart!.seq).toBeLessThan(stepEnd!.seq)
} finally {
await ctx.fiber.dispose()
}
})
it('the head checkpoint the loop lands is a balanced cut on both sides', async () => {
const { ctx } = await harness(8)
try {
@@ -130,3 +216,91 @@ describe('CBR-001: a real-loop checkpoint is a valid boundary on both sides', ()
}
})
})
describe('context-overflow recovery across the real loop and compact-basic', () => {
it.each(['thrown', 'in-band'] as const)(
'force-compacts a %s overflow between failed and retry steps',
async (delivery) => {
const ctx = new Context()
const adapter = new OverflowRecoveryAdapter(delivery)
await ctx.plugin(LlmService)
await ctx.plugin(SessionStore)
await ctx.plugin(Invariants)
await ctx.plugin(SystemPrompt)
await ctx.plugin(ToolRegistry)
await ctx.plugin(AgentRegistry)
await ctx.plugin(AgentLoop, { agents: [] })
await ctx.plugin(TokenMeterService, {
models: { mock: { contextWindow: 128, charsPerToken: 4 } },
})
ctx.llm.registerAdapter(['mock'], adapter)
ctx.on('agent/request', async (_agent, _turn, _step, config) => ({ ...config, model: 'mock' }))
await ctx.plugin(BasicCompactService, {
models: { mock: { thresholdRatio: 1, retainTokens: 100 } },
maxTokens: 64,
compactionRetries: 0,
maxOverflowRetries: 1,
})
try {
const agent = ctx.agentLoop.create(AgentId(`overflow-${delivery}`), {
model: 'unconfigured-agent-fallback',
})
for (let turn = 1; turn <= 2; turn += 1) {
const sentinel = turn === 1 ? 'OLD HISTORY SENTINEL' : 'RECENT HISTORY'
agent.session.append('turn/start', {
turn,
trigger: { kind: 'message', source: { kind: 'user' } },
})
agent.session.append('user/message', {
content: [{ type: 'text', text: `${sentinel} ${'old context '.repeat(200)}` }],
source: { kind: 'user' },
}, { surfaceOp: 'append' })
agent.session.append('step/start', { turn, step: 1 })
agent.session.append('assistant/message', {
turn,
step: 1,
content: [{ type: 'text', text: `historical response ${turn} ${'detail '.repeat(200)}` }],
}, { surfaceOp: 'append' })
agent.session.append('step/end', { turn, step: 1 })
agent.session.append('turn/end', { turn, reason: { kind: 'completed' } })
}
agent.send([{ type: 'text', text: 'continue from history' }])
await agent.whenIdle()
expect(adapter.conversationRequests).toHaveLength(2)
expect(adapter.summaryRequests).toHaveLength(1)
expect(JSON.stringify(adapter.conversationRequests[0]!.messages)).toContain('OLD HISTORY SENTINEL')
const retry = JSON.stringify(adapter.conversationRequests[1]!.messages)
expect(retry).toContain('RECOVERY CHECKPOINT')
expect(retry).not.toContain('OLD HISTORY SENTINEL')
const events = [...agent.session.events]
const failedEnd = events.find(event =>
event.type === 'step/end' && event.data.turn === 3 && event.data.step === 1,
)!
const retryStart = events.find(event =>
event.type === 'step/start' && event.data.turn === 3 && event.data.step === 2,
)!
const compaction = events.filter(event =>
event.type === 'compact/start'
|| event.type === 'compact/summary'
|| event.type === 'compact/end',
)
expect(compaction.map(event => event.type)).toEqual([
'compact/start',
'compact/summary',
'compact/end',
])
expect(compaction.every(event => event.seq > failedEnd.seq && event.seq < retryStart.seq)).toBe(true)
expect(events.at(-1)).toMatchObject({
type: 'turn/end',
data: { reason: { kind: 'completed' } },
})
} finally {
await ctx.fiber.dispose()
}
},
)
})
+3 -4
View File
@@ -18,7 +18,7 @@ Both methods are **abstract** — the backend owns trigger policy, retention, ev
| Member | Semantics |
|---|---|
| `compactIfNeeded(agent, fullSystemPrompt, sessionPrefix, signal)` | Estimate the surface-derived history size; if over the backend's threshold, compact an older range via `compactRegion`, keeping recent context intact. Returns the `CompactionResult`, or `null` if nothing needed compacting. All parameters required — the loop's `agent/pre-step` checkpoint supplies the agent, assembled `fullSystemPrompt`, composed `sessionPrefix` (request-only messages every request carries but the derived history omits — the pressure estimate must count them), and turn `signal`. A backend's summarization request is a direct `ctx.llm.stream()` call (not a loop step), so per-call interception happens at `llm/stream`. |
| `compactIfNeeded(agent, trigger, signal)` | Consider automatic compaction for `trigger: 'pressure' \| 'context-overflow'`. A pressure trigger may apply the backend's threshold and retained-tail policy; a confirmed overflow may force a useful balanced reduction. Returns the `CompactionResult`, or `null` when no safe range exists. A backend's summarization request is a direct `ctx.llm.stream()` call (not a loop step), so per-call interception happens at `llm/stream`. |
| `compactRegion(session, start, end, agent, signal?)` | Forcibly summarize surface nodes `[start, end]` (inclusive seqs) into a single replacement node. The agent must own the exact target (`session === agent.session`); a backend rejects mismatch before model resolution, lock acquisition, summarization, or log mutation. **Throws** if a compaction is already in progress, if `start`/`end` aren't surface nodes, or if `start` is positioned after `end` on the surface. The range is a SURFACE-POSITION span, not a numeric seq interval — after a prior replace lands a fresh high-seq summary node at the shadowed range's position, surface order no longer tracks seq order. |
`compactIfNeeded` takes a required `signal`; `compactRegion`'s is optional. A backend that summarizes via `ctx.llm.stream()` **must** forward it into the call's `GenerateOptions.signal`, so an abort or fiber dispose tears down the in-flight summarization instead of leaving an orphaned model call running past the cancellation. The turn that the `compact/*` events belong to is recoverable from the owned session's log (the currently-open turn), so the backend stamps it from the log rather than trusting a caller-supplied value.
@@ -72,6 +72,5 @@ Subclass `CompactService`, implement `compactIfNeeded` and `compactRegion`, and
## Known Limitations and Deferred Work
- **No model-facing consumer tier yet** — `@deepseek-ai/dsh-tool-compact` (the `/compact` tool) is deferred; compaction is reachable only via direct `ctx.compact` calls or a backend's auto listener.
- **Single-unit overflow is out of contract** — one retained unit (a closed step or a large pasted `user/message`) alone exceeding the budget cannot be compacted; the call may go out over-budget.
- **A session prefix that alone approaches the window is a configuration error no backend fixes** — compaction shrinks derived history, never the prefix.
- **Request context injected by downstream `agent/request` listeners sits outside pressure accounting** — `compactIfNeeded` counts prefix, derived history, and system prompt only.
- **Single-unit overflow is out of contract** — one indivisible unit (a closed tool pair or a large pasted `user/message`) alone exceeding the budget cannot be compacted.
- **An envelope that alone approaches the window is not surface-compaction work** — compaction shrinks derived history, never the system prompt, tools, or session prefix.
+10 -12
View File
@@ -8,7 +8,6 @@
*/
import { Context, Service } from 'cordis'
import type { Message } from '@deepseek-ai/dsh-llm'
import type { Session } from '@deepseek-ai/dsh-session'
import type { CompactionResult } from './types.ts'
@@ -16,6 +15,9 @@ export type { CompactionResult } from './types.ts'
export { renderContentBlocks, renderTranscript } from './render.ts'
export { toolPairingBalancedAfter, toolPairingBalancedBefore } from './tool-pairing.ts'
/** Why automatic policy is asking a backend to consider compaction. */
export type CompactionTrigger = 'pressure' | 'context-overflow'
/** Minimal agent context compaction needs without depending on the agent package. */
export interface CompactAgentContext {
session: Session
@@ -41,24 +43,20 @@ export abstract class CompactService extends Service {
}
/**
* Check token pressure and compact if the conversation is too large.
* Estimate the next request, including its session prefix, derived history,
* and system prompt. Above threshold, compact a head-anchored range ending at
* a balanced tool boundary and reconsolidate any prior automatic checkpoint.
* Return `null` when no compaction is needed or an open tail leaves no safe
* cutoff. A single oversized retained unit or prefix cannot be repaired here.
* Consider automatic compaction for one explicit trigger. Pressure policy
* uses the latest durable routed request, while context-overflow policy may
* force a useful balanced reduction even below the normal threshold. Return
* `null` when no safe range can be compacted. A single oversized retained
* unit or request envelope cannot be repaired through surface compaction.
*
* @param agent - agent context owning the session surface and model options.
* @param fullSystemPrompt - assembled system prompt, counted toward the estimate.
* @param sessionPrefix - the instance's composed session prefix, counted toward the
* estimate.
* @param trigger - normal pressure or provider-confirmed context overflow.
* @param signal - cancellation signal; model-backed implementations must forward it.
* @returns the compaction result, or `null` if no compaction was needed.
*/
abstract compactIfNeeded(
agent: CompactAgentContext,
fullSystemPrompt: string,
sessionPrefix: readonly Message[],
trigger: CompactionTrigger,
signal: AbortSignal,
): Promise<CompactionResult | null>
@@ -1,8 +1,7 @@
import { describe, expect, it } from 'vitest'
import { Context } from 'cordis'
import { CompactService } from '@deepseek-ai/dsh-compact'
import type { CompactionResult } from '@deepseek-ai/dsh-compact'
import type { Message } from '@deepseek-ai/dsh-llm'
import type { CompactionResult, CompactionTrigger } from '@deepseek-ai/dsh-compact'
import { Session, SessionId } from '@deepseek-ai/dsh-session'
import type { CompactAgentContext } from '@deepseek-ai/dsh-compact'
@@ -18,8 +17,7 @@ class StubCompactService extends CompactService {
override async compactIfNeeded(
_agent: CompactAgentContext,
_fullSystemPrompt: string,
_sessionPrefix: readonly Message[],
_trigger: CompactionTrigger,
signal: AbortSignal,
): Promise<CompactionResult | null> {
this.lastSignal = signal
@@ -80,7 +78,7 @@ describe('CompactService seam', () => {
const ctx = new Context()
const svc = new StubCompactService(ctx)
const session = new Session(SessionId('s'))
expect(await svc.compactIfNeeded(stubAgent(session), '', [], new AbortController().signal)).toBeNull()
expect(await svc.compactIfNeeded(stubAgent(session), 'pressure', new AbortController().signal)).toBeNull()
})
it('compact/* events merge into SessionEventMap and are log-only', async () => {
@@ -109,7 +107,7 @@ describe('CompactService seam', () => {
await svc.compactRegion(session, 0, 0, stubAgent(session, 'm'), controller.signal)
expect(svc.lastSignal).toBe(controller.signal)
await svc.compactIfNeeded(stubAgent(session), '', [], controller.signal)
await svc.compactIfNeeded(stubAgent(session), 'context-overflow', controller.signal)
expect(svc.lastSignal).toBe(controller.signal)
})
})
@@ -108,7 +108,7 @@ export const SERVICE_API: readonly ServiceApiEntry[] = [
key: 'compact',
summary: 'Abstract compaction service.',
methods: [
'abstract compactIfNeeded( agent: CompactAgentContext, fullSystemPrompt: string, sessionPrefix: readonly Message[], signal: AbortSignal, ): Promise<CompactionResult | null>',
'abstract compactIfNeeded( agent: CompactAgentContext, trigger: CompactionTrigger, signal: AbortSignal, ): Promise<CompactionResult | null>',
'abstract compactRegion( session: Session, start: number, end: number, agent: CompactAgentContext, signal?: AbortSignal, ): Promise<CompactionResult>',
],
},
@@ -289,8 +289,8 @@ export const EVENT_API: readonly EventApiEntry[] = [
{
name: 'agent/pre-step',
mode: 'serial',
signature: '\'agent/pre-step\'(this: Scoped<Agent>, agent: Agent, turn: number, step: number, fullSystemPrompt: string, sessionPrefix: readonly Message[], signal: AbortSignal): Promise<void> | void',
summary: 'Awaited serial checkpoint for session-surface mutation after prompt assembly and before `step/start`; appends land outside the pending step.',
signature: '\'agent/pre-step\'(this: Scoped<Agent>, agent: Agent, turn: number, step: number, signal: AbortSignal): Promise<void> | void',
summary: 'Awaited serial checkpoint before `step/start`; appends land outside the pending step and are included when the loop derives request history.',
},
{
name: 'agent/prompt-submit',
@@ -656,6 +656,10 @@ export const TYPE_API: readonly TypeApiEntry[] = [
name: 'CompactionResult',
declaration: 'export interface CompactionResult {\n startSeq: number;\n summarySeq: number;\n endSeq: number;\n summary: ContentBlock[];\n shadowedRange: {\n start: number;\n end: number;\n };\n shadowedSeqs: number[];\n shadowedTokenCount: number;\n}',
},
{
name: 'CompactionTrigger',
declaration: 'export type CompactionTrigger = \'pressure\' | \'context-overflow\';',
},
{
name: 'ConfinedArgv',
declaration: 'export interface ConfinedArgv {\n argv: string[];\n enforcement: SandboxEnforcement;\n denialSignatures: readonly string[];\n runnerFailureSignatures: readonly string[];\n}',
+1 -1
View File
@@ -58,7 +58,7 @@ Plugin failure ends the current turn, not the loop. Only final adapter dispatch/
Everything that goes beyond "call the model, run the tools, repeat" belongs to plugins listening on the event taxonomy:
- Hooks and policy: the relevant `agent/*` checkpoints plus the guarded `tools/pre-execute` → `tools/execute` → `tools/post-execute` → `tools/result` pipeline; exact signatures and modes live in the [generated event catalog](../../../docs/cordis-catalog/events.md)
- Compaction: `agent/pre-step`
- Compaction: pressure on `agent/post-step`; canonical context overflow on `agent/request-error`
- Sandbox, permission, plan mode: `tools/pre-execute` for extensible deny/ask, `tools.guard()` for monotonic owner policy, `tools/post-execute` for result decisions, and `tools/result` for final observation
- Sub-agents: implemented outside the loop as `ctx.subagents` providers; in-process providers use `ctx.agents.create()` and owned `AgentHandle` teardown, while child streaming/progress and background/poll collection remain deferred.
- Persistence: `session/event` + `session/flush`
+6 -6
View File
@@ -276,7 +276,7 @@ async function runTurn(
const abort = new AbortController()
handle.setAbort(abort)
// Assemble once before pre-step so pressure checks and the request share the same prompt.
// Assemble once before pre-step so listener work and the request share one prompt value.
const assembly = await ctx.systemPrompt.assemble(assembleContextFor(agent))
const fullSystemPrompt = renderPrompt(assembly)
@@ -287,9 +287,9 @@ async function runTurn(
break
}
// Compose the request-only prefix once per loop instance before pressure
// checks. It precedes all derived history and is recorded only in the
// request header, not as session history.
// Compose the request-only prefix once per loop instance before the first
// request boundary. It precedes all derived history and is recorded only
// in the request header, not as session history.
if (transmission.sessionPrefix === undefined) {
const emptyPrefix: Message[] = deepFreeze([])
const composed = await events.waterfall(
@@ -306,8 +306,8 @@ async function runTurn(
transmission.sessionPrefix = deepFreeze(structuredClone(composed))
}
// Await surface mutations outside the step; pressure checks receive the pending prefix.
await events.serial('agent/pre-step', turn, step, fullSystemPrompt, transmission.sessionPrefix, abort.signal)
// Await surface mutations outside the step before snapshotting history.
await events.serial('agent/pre-step', turn, step, abort.signal)
// Interruption landing during the pre-step seam: do not open an empty step.
if (handle.isCancelled() || handle.isDisposed()) {
@@ -118,7 +118,7 @@ describe('agent/prompt-submit', () => {
it('a prompt-submit rewrite + additionalContext is VISIBLE to the agent/pre-step seam (merged ordering)', async () => {
// Prompt rewrites and injected context land before `agent/pre-step`, so a
// compaction listener measures the current surface before the single derive.
// surface listener sees the current state before the single derive.
const adapter = new MockAdapter([textResponse('ok')])
const ctx = await harness(adapter)
const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' })
@@ -130,8 +130,7 @@ describe('agent/prompt-submit', () => {
additionalContext: { content: [{ type: 'text', text: 'injected ctx' }], source: { kind: 'plugin', plugin: 'test' } },
}))
// The pre-step seam (where compaction lives) derives the surface it would act
// on. Capture what it sees on the first step.
// Capture the surface visible at the generic pre-step seam on the first step.
let preStepDerived: string | undefined
ctx.on('agent/pre-step', (subject, _turn, step) => {
if (subject === agent && step === 1) preStepDerived = JSON.stringify(subject.session.deriveMessages())
@@ -375,7 +374,7 @@ describe('agent/session-prefix', () => {
expect(agent.session.deriveMessages()[0]).toEqual({ role: 'user', content: [{ type: 'text', text: 'go' }] })
})
it('composes before the first pre-step and hands the prefix to the seam (pressure gates see the real value)', async () => {
it('composes before the first pre-step and records the prefix on the request header', async () => {
const adapter = new MockAdapter([textResponse('ok')])
const ctx = await harness(adapter)
const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' })
@@ -386,20 +385,15 @@ describe('agent/session-prefix', () => {
order.push('compose')
return [reminder, ...await next()]
})
const seen: (readonly Message[])[] = []
ctx.on('agent/pre-step', (_agent, _turn, _step, _system, sessionPrefix) => {
ctx.on('agent/pre-step', () => {
order.push('pre-step')
seen.push(sessionPrefix)
})
send(agent, 'hi')
await waitForIdle(ctx, agent)
// Composition precedes the pre-step seam, and the seam receives THIS
// instance's composed prefix — a token-pressure gate (compaction) counts
// what the request will actually carry, never a stale logged prefix.
expect(order).toEqual(['compose', 'pre-step'])
expect(seen[0]).toEqual([reminder])
expect(agent.session.requestHeader()?.messagePrefix).toEqual([reminder])
})
it('the canonical prepend pattern composes contributions in registration order', async () => {
+9 -12
View File
@@ -486,9 +486,8 @@ describe('agent loop', () => {
it('agent/pre-step fires once per step before the step is opened', async () => {
// Two steps (a tool call, then a final text turn) → two model calls → two
// pre-step fires, each carrying the assembled full system prompt, BEFORE
// the step is opened and its request is derived (the request the adapter
// sees reflects any surface state at fire time).
// pre-step fires BEFORE the step is opened and its request is derived (the
// request the adapter sees reflects any surface state at fire time).
const adapter = new MockAdapter([
toolCallResponse('c1', 'echo', {}, 'calling echo'),
textResponse('done'),
@@ -500,21 +499,19 @@ describe('agent loop', () => {
}))
const agent = ctx.agentLoop.create(AgentId('a1'), { model: 'mock' })
const fires: { turn: number; step: number; fullSystemPrompt: string }[] = []
ctx.on('agent/pre-step', (subject, turn, step, fullSystemPrompt) => {
if (subject === agent) fires.push({ turn, step, fullSystemPrompt })
const fires: { turn: number; step: number; signal: AbortSignal }[] = []
ctx.on('agent/pre-step', (subject, turn, step, signal) => {
if (subject === agent) fires.push({ turn, step, signal })
})
send(agent, 'go')
await waitForIdle(ctx, agent)
// One fire per step, in order, each with the assembled system prompt
// (here just the loop's own harness-identity section — no persona set).
const HARNESS = 'You are an AI agent powered by the DeepSeek Harness SDK.'
expect(fires).toEqual([
{ turn: 1, step: 1, fullSystemPrompt: HARNESS },
{ turn: 1, step: 2, fullSystemPrompt: HARNESS },
expect(fires.map(({ turn, step }) => ({ turn, step }))).toEqual([
{ turn: 1, step: 1 },
{ turn: 1, step: 2 },
])
expect(fires.every(({ signal }) => signal instanceof AbortSignal)).toBe(true)
})
it('agent/pre-step fires BEFORE the step it precedes opens (events land outside the step)', async () => {
@@ -219,6 +219,48 @@ describe('agent post-step and request-error lifecycle', () => {
})
})
it('closes the successful step as disposed when disposal lands during post-step', async () => {
const adapter = new FailureScriptAdapter([
toolCallResponse('dispose-call', 'work', {}),
textResponse('must not continue'),
])
const ctx = await harness(adapter)
ctx.tools.register(defineTool({
name: 'work',
description: 'do work',
parameters: {},
async execute() { return [{ type: 'text', text: 'worked' }] },
}))
const agent = ctx.agentLoop.create(AgentId('dispose-post-step'), { model: 'mock' })
let entered!: () => void
const postStepEntered = new Promise<void>((resolve) => { entered = resolve })
ctx.on('agent/post-step', async (_agent, turn, step, signal) => {
expect({ turn, step }).toEqual({ turn: 1, step: 1 })
entered()
await new Promise<void>((resolve) => {
signal.addEventListener('abort', () => { resolve() }, { once: true })
})
})
send(agent)
await postStepEntered
await ctx.fiber.dispose()
expect(adapter.requests).toHaveLength(1)
const boundaries = agent.session.events.filter(event =>
event.type === 'step/start' || event.type === 'step/end',
)
expect(boundaries.map(event => event.type)).toEqual(['step/start', 'step/end'])
expect(boundaries.map(event => event.data)).toEqual([
{ turn: 1, step: 1 },
{ turn: 1, step: 1 },
])
expect(agent.session.events.at(-1)).toMatchObject({
type: 'turn/end',
data: { reason: { kind: 'disposed' } },
})
})
it.each([
['thrown', contextError()],
['in-band', [{ type: 'finish', reason: { kind: 'error', message: 'too large', code: CONTEXT_WINDOW_EXCEEDED_CODE } }] satisfies StreamChunk[]],
+6 -12
View File
@@ -189,23 +189,17 @@ declare module 'cordis' {
// ---- step/request extension seams (serial + waterfall) ----
/**
* Awaited serial checkpoint for session-surface mutation after prompt
* assembly and before `step/start`; appends land outside the pending step.
* The loop derives history once afterward, so compaction records and
* replacements are included without rewriting an assembled request. The
* prompt and prefix are the exact pressure inputs for that request, and
* Awaited serial checkpoint before `step/start`; appends land outside the
* pending step and are included when the loop derives request history.
* `signal` cancels listener work.
* Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent.
* @param agent - the agent opening the step.
* @param turn - the open turn number.
* @param step - the pending step number.
* @param fullSystemPrompt - the assembled prompt.
* @param sessionPrefix - the frozen request prefix.
* @param signal - the turn abort signal.
* @mode serial
*/
// TODO: Move prompt-pressure inputs behind a compaction-specific seam if no second consumer appears.
'agent/pre-step'(this: Scoped<Agent>, agent: Agent, turn: number, step: number, fullSystemPrompt: string, sessionPrefix: readonly Message[], signal: AbortSignal): Promise<void> | void
'agent/pre-step'(this: Scoped<Agent>, agent: Agent, turn: number, step: number, signal: AbortSignal): Promise<void> | void
/**
* Allow, rewrite, or block one drained prompt before it becomes a user
* message. Call `next()` for the unchanged default.
@@ -233,9 +227,9 @@ declare module 'cordis' {
* result is computed once per loop instance, logged on its anchoring request
* header, and reused so the provider prefix remains stable. Interrupted
* composition is discarded. Composition precedes the first `agent/pre-step`
* and request boundary, so listener appends join the current request and
* pressure accounting sees the composed prefix. Changing context belongs in
* history; contributors should prepend to `await next()` to preserve registration order.
* and request boundary, so listener appends join the current request.
* Changing context belongs in history; contributors should prepend to
* `await next()` to preserve registration order.
* Scope-filtered dispatch (`@deepseek-ai/dsh-scope`): agent-scoped listeners receive only that agent.
* @param agent - the agent whose session prefix is being composed.
* @param prefix - the frozen seed; return an extended replacement.
@@ -849,7 +849,7 @@ describe('scoped-dispatch invariants', () => {
['agent/status', [agent, 'idle']],
['agent/queued', [agent, [], { source: { kind: 'user' }, steering: false }]],
['agent/session-start', [agent, 'startup']],
['agent/pre-step', [agent, 1, 1, '', new AbortController().signal]],
['agent/pre-step', [agent, 1, 1, new AbortController().signal]],
['agent/prompt-submit', [agent, [], { kind: 'user' }, () => Promise.resolve({ kind: 'allow' })]],
['agent/request', [agent, 1, 1, { model: 'm' }, () => Promise.resolve({ model: 'm' })]],
['agent/session-prefix', [agent, [], new AbortController().signal, () => Promise.resolve([])]],
@@ -371,7 +371,7 @@ describe('approval policy (the approval/policy fold)', () => {
}
const preStep = (ctx: Context, agent: Agent): Promise<void> =>
ctx.serial('agent/pre-step', agent, 1, 1, '', [], new AbortController().signal)
ctx.serial('agent/pre-step', agent, 1, 1, new AbortController().signal)
/** Append a `request/header` snapshot whose system text is exactly `system`. */
function appendHeader(session: Session, system: string): void {