Merge remote-tracking branch 'origin/master' into xtr/agent-loop-message-machine
# Conflicts: # docs/cordis-catalog/services.md # docs/core-data-structures/session.i18n.yaml # docs/event-producer-consumer.md # examples/acp-agent/tests/snapshots/cordis-inspect-jsdoc/session.jsonl
This commit is contained in:
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/telemetry/session-telemetry-otel/README.md
|
||||
README.md: 9b208e291e77bee50d9d4fd14808268dca75f2db
|
||||
README.zh.md: f36cfe74146b779c4ddb1101227cb45a06f0968c
|
||||
@@ -0,0 +1,41 @@
|
||||
# @deepseek-ai/dsh-session-telemetry-otel
|
||||
|
||||
English | [中文](README.zh.md)
|
||||
|
||||
The OpenTelemetry backend for [the telemetry seam](../session-telemetry/) — the only entry a deployment loads. It composes the OTel JS SDK as-is (`LoggerProvider` → `BatchLogRecordProcessor` → OTLP/HTTP log exporter) and maps each record the seam hands over onto `logger.emit()`, under two instrumentation scopes: ledger records on `@deepseek-ai/dsh-session-telemetry-otel`, operational records on `@deepseek-ai/dsh-session-telemetry-otel/ops`. Resource identity (`service.name`/`service.version`) comes from `dsh-llm`'s `APP_IDENTITY`, the same source the attribution headers use.
|
||||
|
||||
## Config
|
||||
|
||||
```yaml
|
||||
- id: telemetry-otel
|
||||
name: '@deepseek-ai/dsh-session-telemetry-otel'
|
||||
config:
|
||||
exporter: # passed verbatim to the SDK's OTLP/HTTP log exporter
|
||||
url: https://collector.example.com/v1/logs
|
||||
headers:
|
||||
authorization: !!js `Bearer ${process.env.OTLP_TOKEN}`
|
||||
processor: {} # optional; passed verbatim to BatchLogRecordProcessor
|
||||
```
|
||||
|
||||
`exporter.url` is the one field this package validates itself — required, no default, must parse as `http(s)` — so a missing endpoint fails at plugin load (as does a non-positive-integer `processor.maxExportBatchSize`, which the SDK accepts but then hangs on at shutdown). Everything else is the SDK's option shape, owned and documented by the SDK, and both blocks pass through whole: every `OTLPExporterNodeConfigBase` field (`headers`, `timeoutMillis`, `compression`, `keepAlive`, …) reaches the exporter, and batching, export cadence (`scheduledDelayMillis`), retry, queue bounds, and loss policy under sustained failure are the SDK's documented behavior, tuned through the `processor` passthrough. The backend deliberately implements no `flush()`: the batch processor is the only flusher in the process, which is what makes `shutdown()`'s drain complete. Removing this block from `cordis.yml` is the opt-out: no residual state, no `enabled` flag.
|
||||
|
||||
## What leaves the machine
|
||||
|
||||
Records carry the complete `event.data` as the seam's `telemetry/record` waterfall returns it — user and assistant message content, tool arguments and results (command output, file contents), the full system prompt and tool schemas (`request/header`), todo text, compaction summaries, hook `stderrSummary`, and the session `cwd` (a local path). The seam ships no redaction rules: with no `telemetry/record` listener mounted, that is the raw captured copy, so a deployment exporting beyond a trusted boundary mounts its own rules (see [the seam README](../session-telemetry/README.md#the-redact-waterfall)). Provider credentials never appear regardless: adapter API keys are constructor parameters, not session events, so they are structurally absent from the log and therefore from telemetry.
|
||||
|
||||
## Field mapping
|
||||
|
||||
Seam record → SDK log record: `time` → `timestamp`/`observedTimestamp`; `severity` → `severityNumber`/`severityText` (INFO 9 / WARN 13 / ERROR 17); `body` → the structured log body; `attributes` verbatim. Receivers dedupe on `(session.id, event.seq)`, alert on severity, and detect crashes by `shutdown`-record absence (a session with activity, no `shutdown` ops record, gone stale ended uncleanly). The marker means telemetry stopped observing the session cleanly — emitted at the session's own disposal, or at application teardown for sessions still running then; a marker followed by more of that session's events is a telemetry reload, not a session restart. Streams are not self-contained across lineage: a resumed session continues its own id's stream from where the previous process left off, and a forked session's stream starts at its inherited boundary — its prefix lives in the parent's stream, stitched via `session.parent_id` + `session.seed_length`. One consequence of continuing rather than replaying: a turn left open mid-stream and never closed marks the previous process dying inside it. The local log is repaired with synthetic closers at resume, but those repairs are never exported — the wire stream stays faithful to what the crashed process actually shipped, and a later clean `shutdown` marker attests only to the resumed process's own exit.
|
||||
|
||||
## Model Experience
|
||||
|
||||
None, as the backend only forwards the seam's redacted records into the OTel SDK pipeline; it never contributes to a model request.
|
||||
|
||||
#### KV Cache effect
|
||||
|
||||
None; this package neither assembles nor sends a provider request.
|
||||
|
||||
## Known Limitations and Deferred Work
|
||||
|
||||
- **Upstream experimental tree** — `@opentelemetry/sdk-logs` is still published from the upstream experimental tree; SDK API churn lands here and only here — the seam contract does not move.
|
||||
- **No live-collector coverage** — every test exports to a local mock collector; the keyless Loader-composition e2e (`tests/loader-composition.e2e.ts`) covers the wire shape on every run, and behavior against a real OTLP deployment (auth, TLS, throttling) is the SDK exporter's documented territory.
|
||||
@@ -0,0 +1,41 @@
|
||||
# @deepseek-ai/dsh-session-telemetry-otel
|
||||
|
||||
[English](README.md) | 中文
|
||||
|
||||
[遥测(telemetry)seam](../session-telemetry/) 的 OpenTelemetry 后端,也是部署方唯一要加载的条目。它原样组合 OTel JS SDK(`LoggerProvider` → `BatchLogRecordProcessor` → OTLP/HTTP 日志导出器),把 seam 交接过来的每条记录映射到 `logger.emit()`,并使用两个插桩作用域(instrumentation scope):ledger 记录挂在 `@deepseek-ai/dsh-session-telemetry-otel` 下,运维记录挂在 `@deepseek-ai/dsh-session-telemetry-otel/ops` 下。资源身份(`service.name`/`service.version`)来自 `dsh-llm` 的 `APP_IDENTITY`,与归因标头同源。
|
||||
|
||||
## 配置
|
||||
|
||||
```yaml
|
||||
- id: telemetry-otel
|
||||
name: '@deepseek-ai/dsh-session-telemetry-otel'
|
||||
config:
|
||||
exporter: # passed verbatim to the SDK's OTLP/HTTP log exporter
|
||||
url: https://collector.example.com/v1/logs
|
||||
headers:
|
||||
authorization: !!js `Bearer ${process.env.OTLP_TOKEN}`
|
||||
processor: {} # optional; passed verbatim to BatchLogRecordProcessor
|
||||
```
|
||||
|
||||
`exporter.url` 是本包唯一自行校验的字段:必填、无默认值、必须能解析为 `http(s)`,因此缺失端点会在插件加载时失败(`processor.maxExportBatchSize` 不是正整数时同样如此:SDK 会接受该值,随后却在关闭时因它挂起)。其余全部是 SDK 自己的选项形态,由 SDK 拥有并在 SDK 文档中说明,两个配置块都整体透传(passthrough):`OTLPExporterNodeConfigBase` 的每个字段(`headers`、`timeoutMillis`、`compression`、`keepAlive` 等)都会到达导出器;批处理、导出节奏(`scheduledDelayMillis`)、重试、队列上限,以及持续失败下的丢失策略,都是 SDK 的文档化行为,经 `processor` 透传调优。该后端刻意不实现 `flush()`:批处理器是进程内唯一执行 flush 的组件,`shutdown()` 的排空正因如此才是完整的。从 `cordis.yml` 中删除该配置块即为退出方式:无残留状态,也没有 `enabled` 开关。
|
||||
|
||||
## 哪些数据会离开本机
|
||||
|
||||
记录携带完整的 `event.data`,内容以 seam 的 `telemetry/record` waterfall(瀑布式事件)返回的结果为准:用户与 assistant 消息内容、工具参数与工具结果(命令输出、文件内容)、完整的系统提示词与工具 schema(`request/header`)、todo 文本、压缩(compaction)摘要、钩子的 `stderrSummary`,以及会话 `cwd`(一个本地路径)。seam 不带任何脱敏规则:未挂载 `telemetry/record` 监听器时,导出的就是捕获原样的副本,因此向可信边界之外导出的部署方要挂载自己的规则(见 [seam README](../session-telemetry/README.md#the-redact-waterfall))。无论如何,提供方凭据都不会出现:适配器的 API key 是构造函数参数而非会话事件,因此它们在结构上就不存在于日志中,也就不存在于遥测中。
|
||||
|
||||
## 字段映射
|
||||
|
||||
seam 记录 → SDK 日志记录:`time` → `timestamp`/`observedTimestamp`;`severity` → `severityNumber`/`severityText`(INFO 9 / WARN 13 / ERROR 17);`body` → 结构化日志 body;`attributes` 原样照搬。接收端基于 `(session.id, event.seq)` 去重、按严重级别告警,并通过 `shutdown` 记录的缺失检测崩溃(一个曾有活动、没有 `shutdown` 运维记录、且已然陈旧的会话,就是未干净结束的会话)。该标记的含义是遥测干净地停止了对该会话的观察:它在会话自身 dispose(资源释放)时发出,对于届时仍在运行的会话,则在应用拆卸时发出;标记之后又出现该会话的更多事件,说明发生的是遥测重载,而不是会话重启。跨谱系(lineage)的流并不自足:恢复的会话在其自身 id 的流上从上一个进程停止之处继续;fork 出的会话,其流从继承边界开始,前缀位于父会话的流中,由接收端基于 `session.parent_id` + `session.seed_length` 拼接。继续而非回放的一个后果:流中一个开启后再未关闭的轮次,标志着上一个进程死在了该轮次之内。恢复时本地日志会以合成的关闭事件修复,但这些修复绝不导出:导出的流忠实于崩溃进程实际发出的内容,其后干净的 `shutdown` 标记也只证明恢复后进程自身的退出。
|
||||
|
||||
## 模型体验
|
||||
|
||||
无。该后端只把 seam 脱敏后的记录转发进 OTel SDK 流水线;它绝不向模型请求贡献任何内容。
|
||||
|
||||
#### KV Cache 影响
|
||||
|
||||
无;本包既不组装也不发送提供方请求。
|
||||
|
||||
## 已知限制与延期工作
|
||||
|
||||
- **上游实验性源码树**:`@opentelemetry/sdk-logs` 仍从上游实验性(experimental)源码树发布;SDK API 的变动只会落在本包,也仅落在本包;seam 契约不动。
|
||||
- **无真实 collector 覆盖**:所有测试都导出到本地 mock collector;无密钥的 Loader 组合 e2e(`tests/loader-composition.e2e.ts`)在每次运行中都覆盖协议格式(wire format)形态,而面对真实 OTLP 部署的行为(认证、TLS、限流)属于 SDK 导出器文档的职责范围。
|
||||
@@ -0,0 +1,53 @@
|
||||
{
|
||||
"name": "@deepseek-ai/dsh-session-telemetry-otel",
|
||||
"description": "OpenTelemetry backend for the DeepSeek Harness telemetry seam: hands captured session records to the OTel JS SDK's log pipeline",
|
||||
"version": "0.0.1",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"main": "lib/index.js",
|
||||
"types": "lib/types/index.d.ts",
|
||||
"exports": {
|
||||
".": {
|
||||
"types": "./lib/types/index.d.ts",
|
||||
"default": "./lib/index.js"
|
||||
},
|
||||
"./invariant": {
|
||||
"types": "./lib/types/invariant.d.ts",
|
||||
"default": "./lib/invariant.js"
|
||||
},
|
||||
"./src/*": "./src/*",
|
||||
"./package.json": "./package.json"
|
||||
},
|
||||
"files": [
|
||||
"lib/index.js",
|
||||
"lib/invariant.js",
|
||||
"lib/types/**/*.d.ts",
|
||||
"lib/types/**/*.d.ts.map",
|
||||
"src"
|
||||
],
|
||||
"license": "BSD-3-Clause",
|
||||
"dependencies": {
|
||||
"@opentelemetry/api": "^1.9.1",
|
||||
"@opentelemetry/api-logs": "^0.220.0",
|
||||
"@opentelemetry/exporter-logs-otlp-http": "^0.220.0",
|
||||
"@opentelemetry/otlp-exporter-base": "^0.220.0",
|
||||
"@opentelemetry/resources": "^2.9.0",
|
||||
"@opentelemetry/sdk-logs": "^0.220.0",
|
||||
"schemastery": "^3.18.0"
|
||||
},
|
||||
"peerDependencies": {
|
||||
"@deepseek-ai/dsh-invariants": "^0.0.1",
|
||||
"@deepseek-ai/dsh-llm": "^0.0.1",
|
||||
"@deepseek-ai/dsh-session": "^0.0.1",
|
||||
"@deepseek-ai/dsh-session-telemetry": "^0.0.1",
|
||||
"cordis": "^4.0.0-rc.7"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@cordisjs/plugin-loader": "workspace:^",
|
||||
"@deepseek-ai/dsh-invariants": "workspace:^",
|
||||
"@deepseek-ai/dsh-llm": "workspace:^",
|
||||
"@deepseek-ai/dsh-session": "workspace:^",
|
||||
"@deepseek-ai/dsh-session-telemetry": "workspace:^",
|
||||
"cordis": "^4.0.0-rc.7"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,181 @@
|
||||
/**
|
||||
* OpenTelemetry backend for the DeepSeek Harness telemetry seam.
|
||||
*
|
||||
* Composes the OTel JS SDK as-is — a `LoggerProvider` with a
|
||||
* `BatchLogRecordProcessor` and an OTLP/HTTP log exporter — and maps each
|
||||
* record handed over by the seam onto `logger.emit()`. Per the seam's
|
||||
* boundary axiom, everything downstream of that call (batching, retry,
|
||||
* queueing, loss policy) is the SDK's documented behavior, configured
|
||||
* verbatim through the `exporter`/`processor` passthroughs; this package
|
||||
* adds no knobs of its own on top of them.
|
||||
*
|
||||
* @module @deepseek-ai/dsh-session-telemetry-otel
|
||||
*/
|
||||
|
||||
import { createRequire } from 'node:module'
|
||||
import z from 'schemastery'
|
||||
import type { Context } from 'cordis'
|
||||
import { Telemetry, TelemetryCoordinator, type TelemetryRecord, type TelemetrySeverity } from '@deepseek-ai/dsh-session-telemetry'
|
||||
import { APP_IDENTITY } from '@deepseek-ai/dsh-llm'
|
||||
import {
|
||||
BatchLogRecordProcessor,
|
||||
LoggerProvider,
|
||||
type BatchLogRecordProcessorOptions,
|
||||
} from '@opentelemetry/sdk-logs'
|
||||
import { OTLPLogExporter } from '@opentelemetry/exporter-logs-otlp-http'
|
||||
import type { OTLPExporterNodeConfigBase } from '@opentelemetry/otlp-exporter-base'
|
||||
import { SeverityNumber, type AnyValue, type Logger } from '@opentelemetry/api-logs'
|
||||
import { resourceFromAttributes } from '@opentelemetry/resources'
|
||||
|
||||
// The package's own manifest is the single source of the instrumentation-scope
|
||||
// version (same pattern as dsh-llm's attribution identity).
|
||||
const { version } = createRequire(import.meta.url)('../package.json') as { version: string }
|
||||
|
||||
/**
|
||||
* Plugin configuration: two verbatim SDK option shapes plus nothing else.
|
||||
* `exporter.url` is the one field this package validates itself — required,
|
||||
* no default, must parse as an `http(s)` URL — because a missing endpoint
|
||||
* must fail at plugin load, not at first export.
|
||||
*/
|
||||
export interface Config {
|
||||
/**
|
||||
* Passed verbatim to the SDK's OTLP/HTTP log exporter — the complete
|
||||
* `OTLPExporterNodeConfigBase` shape (`headers`, `timeoutMillis`,
|
||||
* `compression`, `keepAlive`, …), owned and documented by the SDK. `url`
|
||||
* is the one field this package requires and validates itself.
|
||||
*/
|
||||
exporter?: OTLPExporterNodeConfigBase & {
|
||||
/** Full logs endpoint (e.g. `https://collector.example.com/v1/logs`). Required; validated at plugin load. */
|
||||
url?: string
|
||||
}
|
||||
/**
|
||||
* Passed verbatim to `BatchLogRecordProcessor` (minus the exporter slot,
|
||||
* which this plugin fills); the SDK owns and documents these knobs.
|
||||
*/
|
||||
processor?: Omit<BatchLogRecordProcessorOptions, 'exporter'>
|
||||
}
|
||||
|
||||
/**
|
||||
* Schemastery validator for {@link Config}; cordis runs it before the plugin
|
||||
* starts. Shape-level only — the load-bearing `exporter.url` check lives in
|
||||
* the constructor so its error message names the field. Both slots are opaque
|
||||
* passthroughs: the SDK owns their shapes and validates its own options;
|
||||
* re-declaring them field-by-field here would violate the boundary axiom
|
||||
* (and silently drop every field not re-declared).
|
||||
*/
|
||||
export const Config: z<Config> = z.object({
|
||||
exporter: z.any(),
|
||||
processor: z.any(),
|
||||
})
|
||||
|
||||
/** Severity mapping from the seam's three-level vocabulary to OTel severity numbers. */
|
||||
const SEVERITY: Record<TelemetrySeverity, { severityNumber: SeverityNumber; severityText: string }> = {
|
||||
info: { severityNumber: SeverityNumber.INFO, severityText: 'INFO' },
|
||||
warn: { severityNumber: SeverityNumber.WARN, severityText: 'WARN' },
|
||||
error: { severityNumber: SeverityNumber.ERROR, severityText: 'ERROR' },
|
||||
}
|
||||
|
||||
/**
|
||||
* The backend plugin — the only entry a deployment loads. Constructing it
|
||||
* wires the SDK pipeline, registers the `telemetry` service (duplicate load
|
||||
* throws, cordis' standard duplicate-service behavior), and composes the
|
||||
* seam's {@link TelemetryCoordinator}, which installs the capture side onto
|
||||
* this fiber.
|
||||
*/
|
||||
export class TelemetryOtel extends Telemetry {
|
||||
static inject = ['sessions']
|
||||
static Config = Config
|
||||
|
||||
private readonly provider: LoggerProvider
|
||||
private readonly ledger: Logger
|
||||
private readonly ops: Logger
|
||||
|
||||
constructor(ctx: Context, config: Config) {
|
||||
super(ctx)
|
||||
const url = config.exporter?.url
|
||||
if (url === undefined || url.length === 0) {
|
||||
throw new Error('session-telemetry-otel: exporter.url is required (the full OTLP logs endpoint)')
|
||||
}
|
||||
let parsed: URL
|
||||
try {
|
||||
parsed = new URL(url)
|
||||
} catch {
|
||||
// Re-thrown as a config error: the only way here is a malformed url string.
|
||||
throw new Error(`session-telemetry-otel: exporter.url is not a valid URL: ${JSON.stringify(url)}`)
|
||||
}
|
||||
if (parsed.protocol !== 'http:' && parsed.protocol !== 'https:') {
|
||||
throw new Error(`session-telemetry-otel: exporter.url must be http(s), got ${parsed.protocol}`)
|
||||
}
|
||||
// The one processor field checked beyond the SDK's own validation: the
|
||||
// SDK accepts a non-positive batch size, but its shutdown drain then
|
||||
// splices empty batches without consuming the queue — dispose would hang
|
||||
// forever with records queued. Misconfiguration fails at load instead.
|
||||
const batchSize = config.processor?.maxExportBatchSize
|
||||
if (batchSize !== undefined && (!Number.isInteger(batchSize) || batchSize < 1)) {
|
||||
throw new Error(`session-telemetry-otel: processor.maxExportBatchSize must be a positive integer, got ${String(batchSize)}`)
|
||||
}
|
||||
this.provider = new LoggerProvider({
|
||||
resource: resourceFromAttributes({
|
||||
'service.name': APP_IDENTITY.product,
|
||||
'service.version': APP_IDENTITY.version,
|
||||
}),
|
||||
processors: [
|
||||
new BatchLogRecordProcessor({
|
||||
...config.processor,
|
||||
// The complete validated exporter object, verbatim: every SDK
|
||||
// option (`timeoutMillis`, `compression`, `keepAlive`, …) reaches
|
||||
// the exporter — rebuilding selected fields here would silently
|
||||
// ignore the rest. App identity travels in the Resource
|
||||
// (service.name/version); the transport-level user-agent is the
|
||||
// SDK's own, per the axiom.
|
||||
exporter: new OTLPLogExporter(config.exporter),
|
||||
}),
|
||||
],
|
||||
})
|
||||
this.ledger = this.provider.getLogger('@deepseek-ai/dsh-session-telemetry-otel', version)
|
||||
this.ops = this.provider.getLogger('@deepseek-ai/dsh-session-telemetry-otel/ops', version)
|
||||
new TelemetryCoordinator(ctx, this)
|
||||
}
|
||||
|
||||
/**
|
||||
* Map one seam record onto the SDK logger for its channel — a synchronous
|
||||
* enqueue into the batch processor's queue.
|
||||
* @param record - the logical record handed over by the coordinator.
|
||||
*/
|
||||
emit(record: TelemetryRecord): void {
|
||||
const logger = record.channel === 'ops' ? this.ops : this.ledger
|
||||
logger.emit({
|
||||
timestamp: record.time,
|
||||
observedTimestamp: record.time,
|
||||
...SEVERITY[record.severity],
|
||||
// JSON-serializable by the seam's contract (validated at Session.append),
|
||||
// which is exactly the AnyValue subset.
|
||||
body: record.body as AnyValue,
|
||||
attributes: record.attributes,
|
||||
})
|
||||
}
|
||||
|
||||
// The seam's optional flush() hint is deliberately NOT implemented. The
|
||||
// batch processor exports on its own cadence (`processor.scheduledDelayMillis`,
|
||||
// the SDK's documented knob), and this backend is the SDK pipeline's only
|
||||
// caller — forwarding the hint to `forceFlush()` was the sole source of
|
||||
// concurrent flushes, whose undocumented interactions with shutdown's
|
||||
// internal drain (concurrent-flush guard, provider-level flush timeout)
|
||||
// silently dropped tail records. Removal history and the revival trigger:
|
||||
// the revival Agent Note.
|
||||
|
||||
/**
|
||||
* Delegate disposal to the SDK's shutdown contract: drain the queue and
|
||||
* quiesce. With no concurrent `forceFlush()` in the process (see above),
|
||||
* shutdown's internal drain is complete — everything emitted before this
|
||||
* call, including the coordinator's dispose-time `shutdown` markers, is
|
||||
* exported before the exporter closes. Awaited (and error-contained) by
|
||||
* the coordinator's disposer.
|
||||
* @returns resolves when the SDK pipeline has quiesced.
|
||||
*/
|
||||
shutdown(): Promise<void> {
|
||||
return this.provider.shutdown()
|
||||
}
|
||||
}
|
||||
|
||||
export default TelemetryOtel
|
||||
@@ -0,0 +1,32 @@
|
||||
/**
|
||||
* Package-owned invariant companion for `@deepseek-ai/dsh-session-telemetry-otel`.
|
||||
* @module @deepseek-ai/dsh-session-telemetry-otel/invariant
|
||||
*/
|
||||
|
||||
/* jscpd:ignore-start */
|
||||
import type { Context } from 'cordis'
|
||||
import type { InvariantInstaller } from '@deepseek-ai/dsh-invariants'
|
||||
|
||||
const PACKAGE_NAME = '@deepseek-ai/dsh-session-telemetry-otel'
|
||||
|
||||
/** Cordis companion plugin name. */
|
||||
export const name = 'session-telemetry-otel-invariant'
|
||||
/** Service required before the companion can reserve package ownership. */
|
||||
export const inject = ['invariants']
|
||||
|
||||
/**
|
||||
* No runtime invariant: the backend forwards seam records into the OTel SDK's
|
||||
* in-process pipeline and appends nothing to any session; its only observable
|
||||
* effects (batching, export) happen inside the SDK past the seam's boundary
|
||||
* axiom, out of reach of an independent companion.
|
||||
*/
|
||||
const install: InvariantInstaller = () => {}
|
||||
|
||||
/**
|
||||
* Register this package's invariant companion.
|
||||
* @param ctx - Cordis context carrying the invariant service.
|
||||
* @returns the installed registration's disposer after setup succeeds.
|
||||
*/
|
||||
export const apply = (ctx: Context): Promise<() => void> =>
|
||||
Promise.resolve(ctx.invariants.register(PACKAGE_NAME, install))
|
||||
/* jscpd:ignore-end */
|
||||
@@ -0,0 +1,99 @@
|
||||
/**
|
||||
* REAL-composition tier: boot the examples-owned telemetry Loader fixture as
|
||||
* a subprocess (per testing policy, through the same app/boot path a
|
||||
* deployment uses), run one mocked-model turn with a real bash round trip,
|
||||
* and assert against what the mock OTLP collector actually received on the
|
||||
* wire: ledger mirroring, the deployment-mounted redact rule applied to the
|
||||
* exported copy, ops markers, and the untouched canonical log.
|
||||
*/
|
||||
|
||||
import { readFile, readdir } from 'node:fs/promises'
|
||||
import { join } from 'node:path'
|
||||
import { fileURLToPath } from 'node:url'
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { LOADER_SMOKE_TEST_TIMEOUT_MS, runLoaderSmoke } from '@deepseek-ai/dsh-loader-smoke'
|
||||
|
||||
const driver = fileURLToPath(new URL(
|
||||
'../../../../examples/headless-agent/tests/fixtures/telemetry-otel-driver.ts',
|
||||
import.meta.url,
|
||||
))
|
||||
const configPath = fileURLToPath(new URL(
|
||||
'../../../../examples/headless-agent/tests/fixtures/telemetry-otel.cordis.yml',
|
||||
import.meta.url,
|
||||
))
|
||||
const repoTsconfig = fileURLToPath(new URL('../../../../tsconfig.json', import.meta.url))
|
||||
|
||||
const FIXTURE_SECRET = 'sk-e2efixture1234567890'
|
||||
const FIXTURE_PLACEHOLDER = '[E2E-REDACTED]'
|
||||
|
||||
interface OtlpLogRecord {
|
||||
attributes?: { key: string; value: Record<string, unknown> }[]
|
||||
body?: unknown
|
||||
}
|
||||
|
||||
interface OtlpCapture {
|
||||
resourceLogs: {
|
||||
scopeLogs: {
|
||||
scope: { name: string }
|
||||
logRecords: OtlpLogRecord[]
|
||||
}[]
|
||||
}[]
|
||||
}
|
||||
|
||||
async function jsonlFiles(dir: string): Promise<string[]> {
|
||||
const entries = await readdir(dir, { withFileTypes: true })
|
||||
const paths = await Promise.all(entries.map(async (entry) => {
|
||||
const path = join(dir, entry.name)
|
||||
if (entry.isDirectory()) return jsonlFiles(path)
|
||||
return entry.isFile() && entry.name.endsWith('.jsonl') ? [path] : []
|
||||
}))
|
||||
return paths.flat()
|
||||
}
|
||||
|
||||
describe('session-telemetry-otel through a real headless cordis.yml', () => {
|
||||
it('exports redacted ledger records to the collector while the canonical log keeps the secret', async () => {
|
||||
let captures: OtlpCapture[] = []
|
||||
let logContent = ''
|
||||
const { stderr } = await runLoaderSmoke({
|
||||
label: 'session-telemetry-otel loader smoke',
|
||||
tempDirPrefix: 'telemetry-otel-e2e-',
|
||||
binScript: driver,
|
||||
libBinScript: driver,
|
||||
configPath,
|
||||
tsconfigPath: repoTsconfig,
|
||||
inspect: async (cwd) => {
|
||||
captures = JSON.parse(await readFile(join(cwd, 'otlp-captures.json'), 'utf8')) as OtlpCapture[]
|
||||
const logs = await jsonlFiles(join(cwd, '.sessions'))
|
||||
expect(logs).toHaveLength(1)
|
||||
logContent = await readFile(logs[0] as string, 'utf8')
|
||||
},
|
||||
})
|
||||
expect(stderr).not.toContain('UNHANDLED')
|
||||
|
||||
const records = captures.flatMap(capture => capture.resourceLogs.flatMap(resource =>
|
||||
resource.scopeLogs.flatMap(scoped => scoped.logRecords.map(record => ({ scope: scoped.scope.name, record })))))
|
||||
expect(records.length).toBeGreaterThan(0)
|
||||
|
||||
const eventTypes = records.flatMap(({ record }) =>
|
||||
record.attributes?.flatMap(attribute =>
|
||||
attribute.key === 'event.type' && typeof attribute.value['stringValue'] === 'string'
|
||||
? [attribute.value['stringValue']]
|
||||
: []) ?? [])
|
||||
for (const expected of ['turn/start', 'user/message', 'tool/call', 'tool/result', 'assistant/message', 'turn/end']) {
|
||||
expect(eventTypes, expected).toContain(expected)
|
||||
}
|
||||
expect(records.some(({ scope }) => scope.endsWith('/ops'))).toBe(true)
|
||||
|
||||
// The deployment-mounted rule on the wire: the fixture credential never
|
||||
// leaves the process, its surrounding prose does, and the placeholder
|
||||
// marks the spot — the seam itself ships no rules.
|
||||
const wire = JSON.stringify(captures)
|
||||
expect(wire).not.toContain(FIXTURE_SECRET)
|
||||
expect(wire).toContain(FIXTURE_PLACEHOLDER)
|
||||
expect(wire).toContain('prove telemetry with key')
|
||||
|
||||
// The canonical session log is never rewritten.
|
||||
expect(logContent).toContain(FIXTURE_SECRET)
|
||||
expect(logContent).not.toContain(FIXTURE_PLACEHOLDER)
|
||||
}, LOADER_SMOKE_TEST_TIMEOUT_MS)
|
||||
})
|
||||
@@ -0,0 +1,238 @@
|
||||
/**
|
||||
* OTel backend unit tier: wire assertions against a scripted `node:http`
|
||||
* mock collector through the SDK's REAL pipeline (BatchLogRecordProcessor →
|
||||
* OTLP/HTTP JSON), config fail-loud cases, and the real-Loader-path guard
|
||||
* for the default-exported Service class.
|
||||
*/
|
||||
|
||||
import { afterEach, describe, expect, it } from 'vitest'
|
||||
import { createServer, type Server } from 'node:http'
|
||||
import { once } from 'node:events'
|
||||
import { gunzipSync } from 'node:zlib'
|
||||
import { Context } from 'cordis'
|
||||
import Loader from '@cordisjs/plugin-loader'
|
||||
import SessionStore, { SessionId } from '@deepseek-ai/dsh-session'
|
||||
import TelemetryOtel, { Config } from '../src/index.ts'
|
||||
|
||||
interface Capture {
|
||||
headers: import('node:http').IncomingHttpHeaders
|
||||
body: OtlpLogsRequest
|
||||
}
|
||||
|
||||
/** Just the slice of ExportLogsServiceRequest JSON these assertions touch. */
|
||||
interface OtlpLogsRequest {
|
||||
resourceLogs: {
|
||||
resource: { attributes: { key: string; value: { stringValue?: string } }[] }
|
||||
scopeLogs: {
|
||||
scope: { name: string }
|
||||
logRecords: {
|
||||
timeUnixNano: string
|
||||
severityNumber: number
|
||||
severityText: string
|
||||
attributes?: { key: string; value: Record<string, unknown> }[]
|
||||
}[]
|
||||
}[]
|
||||
}[]
|
||||
}
|
||||
|
||||
const servers: Server[] = []
|
||||
|
||||
afterEach(async () => {
|
||||
for (const server of servers.splice(0)) {
|
||||
server.close()
|
||||
server.closeAllConnections()
|
||||
}
|
||||
})
|
||||
|
||||
async function mockCollector(
|
||||
beforeRespond?: (requestIndex: number) => Promise<void> | void,
|
||||
): Promise<{ url: string; captures: Capture[] }> {
|
||||
const captures: Capture[] = []
|
||||
let requestIndex = 0
|
||||
const server = createServer((request, response) => {
|
||||
const chunks: Buffer[] = []
|
||||
request.on('data', chunk => chunks.push(chunk as Buffer))
|
||||
request.on('end', () => {
|
||||
const index = requestIndex++
|
||||
void (async () => {
|
||||
await beforeRespond?.(index)
|
||||
const raw = Buffer.concat(chunks)
|
||||
const body = request.headers['content-encoding'] === 'gzip' ? gunzipSync(raw) : raw
|
||||
captures.push({
|
||||
headers: request.headers,
|
||||
body: JSON.parse(body.toString()) as OtlpLogsRequest,
|
||||
})
|
||||
response.writeHead(200, { 'content-type': 'application/json' }).end('{}')
|
||||
})()
|
||||
})
|
||||
})
|
||||
servers.push(server)
|
||||
server.listen(0, '127.0.0.1')
|
||||
await once(server, 'listening')
|
||||
const address = server.address()
|
||||
if (address === null || typeof address === 'string') throw new Error('no port')
|
||||
return { url: `http://127.0.0.1:${address.port}/v1/logs`, captures }
|
||||
}
|
||||
|
||||
async function boot(url: string) {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const fiber = await ctx.plugin(TelemetryOtel, {
|
||||
exporter: { url, headers: { authorization: 'Bearer test-token' } },
|
||||
})
|
||||
return { ctx, fiber }
|
||||
}
|
||||
|
||||
function allRecords(captures: Capture[]) {
|
||||
return captures.flatMap(c => c.body.resourceLogs.flatMap(r => r.scopeLogs.flatMap(s =>
|
||||
s.logRecords.map(record => ({ scope: s.scope.name, record })))))
|
||||
}
|
||||
|
||||
describe('TelemetryOtel wire', () => {
|
||||
it('ships session records and the ops shutdown marker through the real SDK pipeline', async () => {
|
||||
const { url, captures } = await mockCollector()
|
||||
const { ctx, fiber } = await boot(url)
|
||||
const session = ctx.sessions.create(SessionId('wire'), { meta: { cwd: '/tmp/w' } })
|
||||
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'error', step: 1, message: 'boom' } })
|
||||
await fiber.dispose()
|
||||
|
||||
expect(captures.length).toBeGreaterThan(0)
|
||||
const first = captures[0]!
|
||||
const authorization: string | undefined = first.headers.authorization
|
||||
expect(authorization).toBe('Bearer test-token')
|
||||
|
||||
const resource = first.body.resourceLogs[0]!.resource.attributes
|
||||
expect(resource).toContainEqual({ key: 'service.name', value: { stringValue: 'deepseek-harness' } })
|
||||
|
||||
const records = allRecords(captures)
|
||||
const ledger = records.filter(r => r.scope === '@deepseek-ai/dsh-session-telemetry-otel')
|
||||
const ops = records.filter(r => r.scope === '@deepseek-ai/dsh-session-telemetry-otel/ops')
|
||||
|
||||
const start = ledger.find(r => r.record.attributes?.some(a => a.key === 'event.type' && a.value.stringValue === 'turn/start'))
|
||||
expect(start).toBeDefined()
|
||||
expect(start?.record.severityNumber).toBe(9)
|
||||
expect(BigInt(start!.record.timeUnixNano)).toBe(BigInt(session.events[0]!.time) * 1_000_000n)
|
||||
expect(start?.record.attributes).toContainEqual({ key: 'session.cwd', value: { stringValue: '/tmp/w' } })
|
||||
|
||||
const end = ledger.find(r => r.record.attributes?.some(a => a.key === 'event.type' && a.value.stringValue === 'turn/end'))
|
||||
expect(end?.record.severityNumber).toBe(17)
|
||||
expect(end?.record.severityText).toBe('ERROR')
|
||||
|
||||
expect(ops).toHaveLength(1)
|
||||
expect(ops[0]!.record.attributes).toContainEqual({ key: 'telemetry.op', value: { stringValue: 'shutdown' } })
|
||||
})
|
||||
|
||||
it('drains records enqueued after a timer export began: dispose during an in-flight batch', async () => {
|
||||
// The backend implements NO flush() — the batch processor exports on its
|
||||
// own cadence, and shutdown's internal drain is complete exactly because
|
||||
// nothing in the process calls forceFlush() concurrently (the SDK's
|
||||
// concurrent-flush guard skips draining otherwise). Pin that: hold the
|
||||
// collector's response to the timer-triggered export open across
|
||||
// disposal, and the dispose-time shutdown marker (enqueued after that
|
||||
// batch's snapshot) must still arrive.
|
||||
const gate = Promise.withResolvers<boolean>()
|
||||
const arrived = Promise.withResolvers<boolean>()
|
||||
const { url, captures } = await mockCollector(async (index) => {
|
||||
if (index === 0) {
|
||||
arrived.resolve(true)
|
||||
await gate.promise
|
||||
}
|
||||
})
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const fiber = await ctx.plugin(TelemetryOtel, {
|
||||
exporter: { url },
|
||||
processor: { scheduledDelayMillis: 10 },
|
||||
})
|
||||
const session = ctx.sessions.create(SessionId('drain'), { meta: {} })
|
||||
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
await arrived.promise
|
||||
|
||||
const disposal = fiber.dispose()
|
||||
// Let disposal reach the backend's shutdown while the export is held open.
|
||||
await new Promise(resolve => setTimeout(resolve, 50))
|
||||
gate.resolve(true)
|
||||
await disposal
|
||||
|
||||
const ops = allRecords(captures).filter(r => r.scope === '@deepseek-ai/dsh-session-telemetry-otel/ops')
|
||||
expect(ops).toHaveLength(1)
|
||||
expect(ops[0]!.record.attributes).toContainEqual({ key: 'telemetry.op', value: { stringValue: 'shutdown' } })
|
||||
})
|
||||
|
||||
it('passes exporter options beyond url and headers through to the SDK exporter', async () => {
|
||||
const { url, captures } = await mockCollector()
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
// `compression` is a documented SDK exporter option; the advertised
|
||||
// verbatim passthrough must hand it (and every other field) to the
|
||||
// exporter rather than silently rebuilding url/headers only.
|
||||
const fiber = await ctx.plugin(TelemetryOtel, {
|
||||
exporter: { url, compression: 'gzip' },
|
||||
} as Config)
|
||||
const session = ctx.sessions.create(SessionId('gzip'), { meta: {} })
|
||||
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
await fiber.dispose()
|
||||
|
||||
expect(captures.length).toBeGreaterThan(0)
|
||||
expect(captures[0]!.headers['content-encoding']).toBe('gzip')
|
||||
const types = allRecords(captures).flatMap(({ record }) =>
|
||||
record.attributes?.flatMap(a => a.key === 'event.type' ? [a.value.stringValue] : []) ?? [])
|
||||
expect(types).toContain('turn/start')
|
||||
})
|
||||
|
||||
it('maps warn severity from record policy and leaves the seam flush hint unimplemented', async () => {
|
||||
const { url, captures } = await mockCollector()
|
||||
const { ctx, fiber } = await boot(url)
|
||||
ctx.on('telemetry/record', (_record, next) => ({ ...next(), severity: 'warn' }))
|
||||
const session = ctx.sessions.create(SessionId('warn'), { meta: {} })
|
||||
session.append('turn/start', { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
// No flush(): the coordinator's optional-call forwarding no-ops, and the
|
||||
// batch processor owns export cadence end to end (see the backend note).
|
||||
expect('flush' in ctx.telemetry && ctx.telemetry.flush !== undefined).toBe(false)
|
||||
await fiber.dispose()
|
||||
const start = allRecords(captures).find(r =>
|
||||
r.record.attributes?.some(a => a.key === 'event.type' && a.value.stringValue === 'turn/start'))
|
||||
expect(start?.record.severityNumber).toBe(13)
|
||||
})
|
||||
})
|
||||
|
||||
describe('TelemetryOtel config fails loud', () => {
|
||||
it.each([
|
||||
[{}, /exporter\.url is required/],
|
||||
[{ exporter: { url: '' } }, /exporter\.url is required/],
|
||||
[{ exporter: { url: 'not a url' } }, /not a valid URL/],
|
||||
[{ exporter: { url: 'ftp://collector' } }, /must be http\(s\)/],
|
||||
// The SDK accepts a non-positive batch size but its shutdown drain then
|
||||
// splices empty batches forever — dispose would hang, so reject at load.
|
||||
[{ exporter: { url: 'http://c/v1/logs' }, processor: { maxExportBatchSize: 0 } }, /maxExportBatchSize/],
|
||||
[{ exporter: { url: 'http://c/v1/logs' }, processor: { maxExportBatchSize: 0.5 } }, /maxExportBatchSize/],
|
||||
])('rejects %j at plugin load', async (config, message) => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
await expect(ctx.plugin(TelemetryOtel, config as Config)).rejects.toThrow(message)
|
||||
})
|
||||
})
|
||||
|
||||
describe('dsh-session-telemetry-otel real-load-path guard', () => {
|
||||
it('keeps the Service class with inject/Config through unwrapExports', async () => {
|
||||
const module = await import('../src/index.ts')
|
||||
const loader = Object.create(Loader.prototype) as Loader
|
||||
const unwrapped = loader.unwrapExports(module) as typeof TelemetryOtel
|
||||
expect(unwrapped).toBe(TelemetryOtel)
|
||||
expect(unwrapped.inject).toEqual(['sessions'])
|
||||
expect(typeof unwrapped.Config).toBe('function')
|
||||
})
|
||||
|
||||
it('boots through the unwrapped class and registers ctx.telemetry', async () => {
|
||||
const { url } = await mockCollector()
|
||||
const module = await import('../src/index.ts')
|
||||
const loader = Object.create(Loader.prototype) as Loader
|
||||
const unwrapped = loader.unwrapExports(module) as Parameters<Context['plugin']>[0]
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const fiber = await ctx.plugin(unwrapped, { exporter: { url } })
|
||||
expect(ctx.telemetry).toBeInstanceOf(TelemetryOtel)
|
||||
await fiber.dispose()
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,33 @@
|
||||
{
|
||||
"extends": "../../../tsconfig.base.json",
|
||||
"compilerOptions": {
|
||||
"rootDir": "src",
|
||||
"outDir": "lib/types"
|
||||
},
|
||||
"include": [
|
||||
"src"
|
||||
],
|
||||
"references": [
|
||||
{
|
||||
"path": "../../../vendor/cosmokit"
|
||||
},
|
||||
{
|
||||
"path": "../../../vendor/cordis"
|
||||
},
|
||||
{
|
||||
"path": "../../../vendor/schemastery"
|
||||
},
|
||||
{
|
||||
"path": "../../core/session"
|
||||
},
|
||||
{
|
||||
"path": "../../llm/llm"
|
||||
},
|
||||
{
|
||||
"path": "../session-telemetry"
|
||||
},
|
||||
{
|
||||
"path": "../../support/invariants"
|
||||
}
|
||||
]
|
||||
}
|
||||
Reference in New Issue
Block a user