feat(web): show durable token usage and context occupancy in the stats line
The chat stats line took its token totals from the loaded conversation nodes, so paging changed them and compaction erased the billing behind replaced content. It also had no way to show context occupancy: the numerator and capacity never reached the browser. Both now come from token-meter session projections read through the standard useProjection seat. Window nodes keep supplying turn and step counts plus LLM and tool wall times, which are correctly window-scoped facts about what is on screen; accounting no longer comes from there. `tokenUsage` supplies billing and cache hit. `contextPressure` supplies occupancy, pairing the newest provider-reported prompt size with the newest capacity recorded by `request/context`. Deployments without token-meter drop the token groups; a route whose adapter advertises no capacity drops the occupancy group rather than rendering a placeholder. Occupancy is deliberately approximate: the numerator and capacity are independent last-wins fields, not one atomic request observation, so switching models pairs a fresh capacity with the prior route's pressure until the next request reports usage. It is a user-facing reference figure that nothing in the harness makes decisions from, and it matches how the TUI status line has always computed occupancy. The Agent Note and token-meter README state this as a decision, including why the atomic alternative was implemented and rejected, so it is not re-litigated as a defect. Snapshot delta is one added `Context N% of 128K` segment across eight web goldens; the preceding commit absorbed master's pre-existing golden drift.
This commit is contained in:
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/llm/llm/README.md
|
||||
README.md: d343449d1530bf70a3a8c57f883894e29c42d18f
|
||||
README.zh.md: 4dc4a0ca06378116d05fdb4b9b048738930511fd
|
||||
README.md: 1b815cfba8d2ee1dd69e1f00b8a3911c7d9a788c
|
||||
README.zh.md: c08a46a7037dab28c27e1a593aa83fcca08144f3
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/llm/token-meter/README.md
|
||||
README.md: 578728ded9cf51a12abcd70d541404e995028f26
|
||||
README.zh.md: 9d54ddb4792e6c897af7d57a6f8ae98204caa11d
|
||||
README.md: b9bf1dfa253e424e5ec35cd3e7bf0f52af579077
|
||||
README.zh.md: c97fc0b87364dfe9ca46139f0ec82519e191b772
|
||||
|
||||
@@ -21,11 +21,23 @@ The fold tracks full request-header snapshots, step boundaries, surface appends
|
||||
|
||||
Usage accounting sums disjoint input, cache-read, cache-write, and output buckets; reasoning is not added again. Every successful call records an assistant anchor, including content-less calls. An explicit empty provenance list means a known empty provider stream, while absent legacy provenance conservatively treats the durable assistant output as provider output.
|
||||
|
||||
## Session projection
|
||||
## Session projections
|
||||
|
||||
When the composition provides `ctx.sessionProjections`, token-meter registers the `tokenUsage` unit through an optional child fiber. Its client-safe value is the complete durable log's `uncachedInputTokens`, `outputTokens`, `cacheReadTokens`, and `cacheWriteTokens`. Usage chunks are counted even when a request later fails; a final assistant-message usage for the same `(turn, step)` replaces that sample instead of double-counting it. Reasoning remains an output subdivision.
|
||||
When the composition provides `ctx.sessionProjections`, token-meter registers two units through an optional child fiber.
|
||||
|
||||
The unit uses the standard projection baseline, live frame, higher-seq-wins store, and JSON checkpoint paths. Unloading token-meter removes the key. A headless or TUI composition without the projection seam keeps the measurement service's existing behavior.
|
||||
`tokenUsage` carries the complete durable log's `uncachedInputTokens`, `outputTokens`, `cacheReadTokens`, and `cacheWriteTokens`. Usage chunks are counted even when a request later fails; a final assistant-message usage for the same `(turn, step)` replaces that sample instead of double-counting it. Reasoning remains an output subdivision. The single last-sample slot relies on a session-log ordering property: once a later step reports usage, a legal log never reports usage for an earlier step again.
|
||||
|
||||
`contextPressure` carries `pressureTokens` — the newest provider-reported prompt size, summing uncached input plus cache reads and writes — and the optional `contextWindow` from the newest `request/context` record. Output is excluded, so the numerator holds still while a turn streams and steps forward when the next request reports its usage.
|
||||
|
||||
Both units use the standard projection baseline, live frame, higher-seq-wins store, and JSON checkpoint paths. Unloading token-meter removes both keys. A headless or TUI composition without the projection seam keeps the measurement service's existing behavior.
|
||||
|
||||
### Context occupancy is an approximation, by design
|
||||
|
||||
`pressureTokens` and `contextWindow` are independent last-wins fields and are **not** one atomic observation of a single request. Switching models pairs the fresh capacity with the previous route's pressure until the next request reports usage, and `pressureTokens` describes the last request rather than the surface as it stands right now.
|
||||
|
||||
This is deliberate. An occupancy percentage is a user-facing reference figure, not a billing record or a gating input — nothing in the harness makes decisions from it, and compaction reads `measure()` instead. The TUI status line has always computed occupancy the same way, dividing a `measure()` total by a separately-resolved capacity for the selected model.
|
||||
|
||||
Making the pair atomic was tried and rejected: it required a transient non-replayable wire frame, which needed lifecycle fencing against cross-stream reordering and left occupancy blank after every reconnect. The [Agent Note](../../../.agents/notes/implemented/architecture/2026-07-29-projected-token-usage-and-request-context.md) records that comparison. Consumers that need an exact same-boundary figure should call `measure()` at their own request boundary rather than read this projection.
|
||||
|
||||
## Composition
|
||||
|
||||
|
||||
@@ -23,9 +23,21 @@ fold 跟踪完整请求标头快照、步骤边界、表层追加与替换、成
|
||||
|
||||
## 会话投影
|
||||
|
||||
当组合提供 `ctx.sessionProjections` 时,token-meter 会通过一个可选子 fiber 注册 `tokenUsage` 单元。其可安全传给客户端的值包含完整持久日志中的 `uncachedInputTokens`、`outputTokens`、`cacheReadTokens` 和 `cacheWriteTokens`。即使请求随后失败,用量分片仍会计入;同一 `(turn, step)` 的最终 assistant 消息用量会替换该样本,而不是重复计数。推理仍是输出的一个细分项。
|
||||
当组合提供 `ctx.sessionProjections` 时,token-meter 会通过一个可选子 fiber 注册两个单元。
|
||||
|
||||
该单元使用标准的投影基线、实时帧、seq 高者胜值仓和 JSON 检查点路径。卸载 token-meter 会移除该键。不带投影 seam 的 headless 或 TUI 组合会保留测量服务的既有行为。
|
||||
`tokenUsage` 携带完整持久日志中的 `uncachedInputTokens`、`outputTokens`、`cacheReadTokens` 和 `cacheWriteTokens`。即使请求随后失败,用量分片仍会计入;同一 `(turn, step)` 的最终 assistant 消息用量会替换该样本,而不是重复计数。推理仍是输出的一个细分项。只保留单个最新样本,依赖的是会话日志的一条顺序性质:一旦某个更晚的步骤报告了用量,合法日志就绝不会再为更早的步骤报告用量。
|
||||
|
||||
`contextPressure` 携带 `pressureTokens`(提供方报告的最新提示词规模,为未缓存输入加缓存读取与写入之和),以及来自最新一条 `request/context` 记录的可选 `contextWindow`。输出不计入其中,因此轮次流式输出期间分子保持不动,等到下一个请求报告用量时才前进。
|
||||
|
||||
两个单元都使用标准的投影基线、实时帧、seq 高者胜值仓和 JSON 检查点路径。卸载 token-meter 会移除这两个键。不带投影 seam 的 headless 或 TUI 组合会保留测量服务的既有行为。
|
||||
|
||||
### 上下文占用率是刻意为之的近似值
|
||||
|
||||
`pressureTokens` 与 `contextWindow` 是两个各自后者胜的独立字段,**不是**对单个请求的一次原子观测。切换模型时,新容量会与上一路由的压力配对,直到下一个请求报告用量为止;而 `pressureTokens` 描述的是最后一个请求,不是此刻的表层。
|
||||
|
||||
这是刻意的选择。占用率百分比是面向用户的参考数字,既不是计费记录,也不是门控输入:harness 中没有任何环节依据它做决策,压缩改为直接读取 `measure()`。TUI 状态行一直以同样的方式计算占用率,即用 `measure()` 总量除以为所选模型单独解析出的容量。
|
||||
|
||||
让这对值保持原子已经尝试过并被否决:它需要一个临时且不可回放的协议帧,进而需要针对跨流重排序的生命周期栅栏,还会让占用率在每次重连后变为空白。[Agent Note(agent 决策记录)](../../../.agents/notes/implemented/architecture/2026-07-29-projected-token-usage-and-request-context.md)记录了这项对比。需要同一边界精确数字的消费方应在自己的请求边界调用 `measure()`,而不是读取该投影。
|
||||
|
||||
## 组合
|
||||
|
||||
|
||||
@@ -6,7 +6,7 @@
|
||||
|
||||
import type { TokenUsage } from '@deepseek-ai/dsh-llm'
|
||||
|
||||
export type { TokenUsageProjection } from './projection.ts'
|
||||
export type { ContextPressureProjection, TokenUsageProjection } from './projection.ts'
|
||||
|
||||
/** Token-meter plugin configuration; the fixed estimator has no settings. */
|
||||
export type TokenMeterConfig = Record<string, never>
|
||||
|
||||
@@ -147,5 +147,5 @@ ProjectionDefinition<'contextPressure', ContextPressureProjection> = {
|
||||
: { ...state, pressureTokens }
|
||||
},
|
||||
view: state => state,
|
||||
stateVersion: 0,
|
||||
stateVersion: 1,
|
||||
}
|
||||
|
||||
@@ -6,7 +6,7 @@ import SessionStore from '@deepseek-ai/dsh-session'
|
||||
import type { Session } from '@deepseek-ai/dsh-session'
|
||||
import SessionProjectionRegistry from '@deepseek-ai/dsh-session-projection'
|
||||
import TokenMeterService from '@deepseek-ai/dsh-token-meter'
|
||||
import type { TokenUsageProjection } from '@deepseek-ai/dsh-token-meter/client'
|
||||
import type { ContextPressureProjection, TokenUsageProjection } from '@deepseek-ai/dsh-token-meter/client'
|
||||
|
||||
const ZERO: TokenUsageProjection = {
|
||||
uncachedInputTokens: 0,
|
||||
@@ -220,3 +220,93 @@ describe('tokenUsage session projection', () => {
|
||||
})
|
||||
})
|
||||
})
|
||||
|
||||
const pressure = (ctx: Context, session: Session): ContextPressureProjection => {
|
||||
const value = ctx.sessionProjections.snapshot(session).values.contextPressure
|
||||
if (value === undefined) throw new Error('contextPressure projection is not registered')
|
||||
return value
|
||||
}
|
||||
|
||||
function recordContext(session: Session, model: string, contextWindow: number): void {
|
||||
session.append('request/context', { provider: 'mock', model, contextWindow })
|
||||
}
|
||||
|
||||
describe('contextPressure session projection', () => {
|
||||
it('serves zero pressure and no capacity for an empty log', async () => {
|
||||
const { ctx, session } = await harness()
|
||||
expect(pressure(ctx, session)).toEqual({ pressureTokens: 0 })
|
||||
})
|
||||
|
||||
it('sums prompt-side buckets and excludes response output', async () => {
|
||||
const { ctx, session } = await harness()
|
||||
startStep(session, 1, 1)
|
||||
usageChunk(session, {
|
||||
inputTokens: 100,
|
||||
outputTokens: 4_000,
|
||||
cacheReadTokens: 20,
|
||||
cacheWriteTokens: 5,
|
||||
}, 1, 1)
|
||||
// Output is deliberately absent: occupancy describes the prompt that was
|
||||
// sent, so it holds still while the response streams.
|
||||
expect(pressure(ctx, session).pressureTokens).toBe(125)
|
||||
})
|
||||
|
||||
it('replaces pressure with the newest request rather than accumulating', async () => {
|
||||
const { ctx, session } = await harness()
|
||||
startStep(session, 1, 1)
|
||||
const first = usageChunk(session, { inputTokens: 100, outputTokens: 10 }, 1, 1)
|
||||
finalUsage(session, { inputTokens: 100, outputTokens: 10 }, 1, 1, [first])
|
||||
startStep(session, 2, 1)
|
||||
usageChunk(session, { inputTokens: 250, outputTokens: 10 }, 2, 1)
|
||||
expect(pressure(ctx, session).pressureTokens).toBe(250)
|
||||
})
|
||||
|
||||
it('carries the newest recorded capacity and replaces it on a model switch', async () => {
|
||||
const { ctx, session } = await harness()
|
||||
startStep(session, 1, 1)
|
||||
recordContext(session, 'small', 64_000)
|
||||
usageChunk(session, { inputTokens: 100, outputTokens: 10 }, 1, 1)
|
||||
expect(pressure(ctx, session)).toEqual({ pressureTokens: 100, contextWindow: 64_000 })
|
||||
recordContext(session, 'large', 256_000)
|
||||
expect(pressure(ctx, session)).toEqual({ pressureTokens: 100, contextWindow: 256_000 })
|
||||
})
|
||||
|
||||
it('pushes no change for unrelated events or a restated capacity', async () => {
|
||||
// The registry gates its change feed on Object.is, so a unit that rebuilt
|
||||
// state for an event it does not care about would push phantom updates.
|
||||
const { ctx, session } = await harness()
|
||||
startStep(session, 1, 1)
|
||||
recordContext(session, 'small', 64_000)
|
||||
usageChunk(session, { inputTokens: 100, outputTokens: 10 }, 1, 1)
|
||||
const changed: string[] = []
|
||||
ctx.sessionProjections.onChanged((_session, key) => { changed.push(key) })
|
||||
|
||||
session.append('todo/write', { todos: [] })
|
||||
expect(changed).not.toContain('contextPressure')
|
||||
// A repeated capacity record for the same window is also a no-op.
|
||||
recordContext(session, 'small', 64_000)
|
||||
expect(changed).not.toContain('contextPressure')
|
||||
// A real capacity change still reports.
|
||||
recordContext(session, 'large', 256_000)
|
||||
expect(changed).toContain('contextPressure')
|
||||
})
|
||||
|
||||
it('restores from a JSON checkpoint and unregisters with the token-meter fiber', async () => {
|
||||
const { ctx, session, meterFiber } = await harness()
|
||||
startStep(session, 1, 1)
|
||||
recordContext(session, 'small', 64_000)
|
||||
usageChunk(session, { inputTokens: 42, outputTokens: 2 }, 1, 1)
|
||||
const checkpoint = JSON.parse(JSON.stringify(
|
||||
ctx.sessionProjections.checkpoint(session),
|
||||
)) as ReturnType<typeof ctx.sessionProjections.checkpoint>
|
||||
|
||||
await meterFiber.dispose()
|
||||
expect(ctx.sessionProjections.snapshot(session).values).not.toHaveProperty('contextPressure')
|
||||
|
||||
await ctx.plugin(TokenMeterService)
|
||||
expect(ctx.sessionProjections.viewCheckpoint(checkpoint).contextPressure).toEqual({
|
||||
pressureTokens: 42,
|
||||
contextWindow: 64_000,
|
||||
})
|
||||
})
|
||||
})
|
||||
|
||||
Reference in New Issue
Block a user