refactor(token-meter): make context occupancy durable projection state

Replace the transient `session/model-request` mux frame with ordinary durable
session state. Occupancy now rides two last-wins projection fields instead of a
non-replayable frame that needed removal tombstones and cross-stream fencing.

The frame was the only non-replayable class on the mux stream. Because host and
mux are independent SSE streams with no cross-stream order, a request emitted
before a removal could arrive after `host/session-removed`, and a legitimate
request for a new lifecycle reusing the same id could be fenced by a late
removal. Fixing that needed a lifecycle generation on every frame; the frame
itself was the problem.

Removed: the `session/model-request` frame and schema, the `agent/model-request`
core event, the ApiProxy measurement point, the client-side telemetry map and
removal tombstone, and the synthetic `cancelled` open error used to signal
reconnect through the error channel.

Added: `request/context`, a log-only session event recording the
registration-bound capacity of the route a request resolved to, appended beside
`request/header` from the lookup that already prepared the call and skipped when
the route is unchanged. Capacity stays out of `EpochHeader` because it is
adapter metadata about a route, not an input the request was built from, so it
must not join request reconstruction or header equality.

The `contextPressure` projection pairs the newest provider-reported prompt size
with the newest recorded capacity. The two are deliberately not one atomic
request observation: switching models can pair a fresh capacity with the prior
route's pressure until the next request reports usage. The figure is a
user-facing reference, and this matches how the TUI status line has always
computed occupancy.
This commit is contained in:
Hypatia May
2026-07-30 13:53:08 +08:00
parent fdccc58cef
commit 4819210142
62 changed files with 382 additions and 1362 deletions
+17 -14
View File
@@ -489,20 +489,6 @@ export class ReactLoopAgent implements Agent {
const assembler = new BlockAssembler()
const chunkSeqs: number[] = []
const stream = preparedCall?.stream(request) ?? this.loopCtx.llm.stream(request)
emitAgentEvent(
this.loopCtx,
this,
'agent/model-request',
turn,
step,
{
provider: request.provider,
model: request.model,
...preparedCall?.context === undefined
? {}
: { contextWindow: preparedCall.context.contextWindow },
},
)
try {
for await (const chunk of stream) {
signal.throwIfAborted()
@@ -640,6 +626,23 @@ export class ReactLoopAgent implements Agent {
session.append('request/header', { header, reason: 'change' })
}
// Capacity of the route this request resolved to, recorded from the same
// registration-bound lookup that prepared the call (no second resolve).
// Deduplicated against the last record: an unchanged route logs nothing.
const contextWindow = preparedCall?.context?.contextWindow
if (contextWindow !== undefined) {
const previous = session.requestContext()
if (previous?.provider !== config.provider
|| previous.model !== config.model
|| previous.contextWindow !== contextWindow) {
session.append('request/context', {
provider: config.provider,
model: config.model,
contextWindow,
})
}
}
const request = markAgentLoopRequest(deepFreeze({
...header.config,
messages: boundaryMessages,