Pine/deepseek-harness
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases 1 Wiki Activity

feat(telemetry): require explicit opt-in

Browse Source
This commit is contained in:
Turtle
2026-08-10 17:45:50 +08:00
parent 504e96a049
commit e6eb44ed2a
53 changed files with 265 additions and 370 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.agents/notes/implemented/feature/2026-07-31-web-telemetry-default-mount.md
+11 -10
View File
@@ -10,23 +10,24 @@ The telemetry seam and OTel backend ([revival Note](2026-07-23-session-telemetry
## Decision
The shared `dsh` base (`apps/cli/config/base.cordis.yml`) mounts the `telemetry-otel` row by default with a baked-in production endpoint, so Web and headless report; the raw-config command also mounts it before applying its required deployment overlay. This is the **internal-testing deployment stance** — reporting is on when an endpoint exists, and users opt out through the environment. Web and headless use the [bounded, escalating process-shutdown controller](../bug-fix/2026-08-03-cli-signal-shutdown-escalation.md) on SIGINT/SIGTERM, giving the backend's three-second shutdown deadline time to drain before the five-second launcher bound.
The shared dsh base bundle (`packages/bundle/base/cordis.patch.yml`) mounts the `telemetry-otel` row with a baked-in production endpoint, so every profile has one consistent telemetry capability. The [default-off decision](2026-08-10-telemetry-default-off.md) keeps that row in `DISABLED` mode unless a deployment explicitly selects `FULL` or `FEEDBACK_ONLY`; the endpoint alone does not authorize reporting. Web and headless use the [bounded, escalating process-shutdown controller](../bug-fix/2026-08-03-cli-signal-shutdown-escalation.md) on SIGINT/SIGTERM, giving an enabled backend's three-second shutdown deadline time to drain before the five-second launcher bound.
| Ruling | Value | Rationale |
|---|---|---|
| Mount surface | base.cordis.yml (raw config + Web + headless) | One deployment stance for every tree that loads the shared base; the raw overlay decides whether that deployment creates sessions |
| Mount surface | `packages/bundle/base/cordis.patch.yml` | One capability row for every profile that loads the shared base |
| Sharing mode | `DSH_TELEMETRY_MODE`, default `DISABLED`; explicit `FULL` or `FEEDBACK_ONLY` opts in | A fresh profile makes no telemetry network request, while internal deployments retain both upload policies |
| Endpoint | `DSH_TELEMETRY_OTLP_URL`, default `https://harness-telemetry.deepseeksvc.com/v1/logs` | Internal collector; the env override serves local/dev runs |
| Opt-out switch | any non-empty `DSH_TELEMETRY_DISABLED` (including `0`/`false`) disables | A privacy switch prefers off-by-mistake over on-by-mistake; a row can only be disabled at AppCLIEntry's patch layer (config has no disable semantic, and the switch must precede the load-time `exporter.url` validation) |
| Cadence | `processor.scheduledDelayMillis: 10000` (10s/batch) | Streaming while the session runs, never exit-time-only; a crash loses at most the last unexported interval |
| Hard opt-out | any non-empty `DSH_TELEMETRY_DISABLED` (including `0`/`false`) disables the row | The launcher patch takes effect before load-time transport validation and overrides every configured mode |
| Cadence | `processor.scheduledDelayMillis: 10000` (10s/batch) in uploading modes | Streaming while the session runs, never exit-time-only; a crash loses at most the last unexported interval |
| Exit-drain bound | `exporter.timeoutMillis: 1000` + `maxExportBatchSize: 2048` (== maxQueueSize) + `exportTimeoutMillis: 1500` + `shutdownTimeoutMillis: 3000` | Ordinary unreachable-collector failure releases in ~1s: timeoutMillis is the per-attempt socket timeout and retry deadline, while one queue-sized batch avoids sequential drain multiplication. The DSH-owned 3s outer bound covers the SDK's preceding unbounded `forceFlush()` wait when the transport Promise never obtains a socket. |
| Compression | `compression: gzip` | Event bodies carry full content; cross-datacenter bandwidth |
| CI isolation | top-level `env: DSH_TELEMETRY_DISABLED: '1'` in all 8 GitHub workflows | Every CI channel that boots the web composition (e2e/snapshot/built smokes) must not stream test sessions to the production endpoint |
| CI isolation | top-level `env: DSH_TELEMETRY_DISABLED: '1'` in GitHub workflows | Defense in depth keeps test sessions local even when a job explicitly selects an uploading mode |
The keyless integration test `apps/cli/tests/telemetry-web.e2e.ts` pins the deployment-level behavior: an in-test OTLP collector plus a mock LLM server, a real `dsh web` boot, asserting ledger coverage, seq monotonicity, the first-of-step chunk projection, and the ops `shutdown` marker arriving through the SIGINT drain.
The base bundle test pins the shipped `DISABLED` mode expression, the backend suite pins that omitted mode constructs no transport, and the real Loader composition suite explicitly selects each uploading mode when it verifies OTLP delivery.
## Alternatives considered
**No default mount; deployments add the row themselves (continuing the SDK stance).** Rejected for this stage: this repo's web/headless composition IS the internal deployment, and default-on reporting is that deployment's product requirement; the SDK stance survives in the Service Definition packages (unmounted = nothing leaves).
**No default mount; deployments add the row themselves.** Rejected because the mounted `DISABLED` mode retains a local feedback warning and gives all profiles one patch target without authorizing any upload.
**A config field instead of an env patch for the switch.** Infeasible: cordis rows have no config-level disable semantic, and `exporter.url` validation fails loud at plugin construction, so the switch must take effect before the Loader — AppCLIEntry's patch layer is the only seat.
@@ -34,6 +35,6 @@ The keyless integration test `apps/cli/tests/telemetry-web.e2e.ts` pins the depl
## Consequences
- A developer running `dsh web` without a local collector POSTs to the production endpoint every 10s (silent failure when unreachable; no OTel diag logger is registered); local development sets `DSH_TELEMETRY_DISABLED=1` or points `DSH_TELEMETRY_OTLP_URL` locally.
- **No redaction rule is mounted yet**: exports are the raw captured copy (full user/assistant message text, tool arguments and results, the system prompt, the local `session.cwd` path). Crossing a trust boundary requires `telemetry/record` rules first — the redaction rule, the remaining identity Resource attributes (hostname / surface; the anonymous user id shipped via the [anonymous-user-id Note](2026-07-31-telemetry-anonymous-user-id.md)), and the usage-metrics track are the explicit follow-ups of this decision.
- Test rigs reusing this tree (e.g. `apps/web/tests/scaffold.ts`) must explicitly disable the row, or fixture sessions stream to whatever collector the environment happens to name.
- A developer running `dsh web` without telemetry configuration makes no telemetry network request. An internal deployment sets `DSH_TELEMETRY_MODE` and may point `DSH_TELEMETRY_OTLP_URL` at another collector.
- **No redaction rule is mounted**: explicitly enabled exports are the raw captured copy (full user/assistant message text, tool arguments and results, the system prompt, the local `session.cwd` path). Crossing a trust boundary requires `telemetry/record` rules first — the redaction rule, remaining identity Resource attributes, and usage metrics remain separate deployment work. The anonymous user id ships through the [anonymous-user-id Note](2026-07-31-telemetry-anonymous-user-id.md).
- Test rigs remain local by default; explicit uploading-mode tests provide their own collector and mode.