Merge remote-tracking branch 'origin/master' into web-e2e-lane
# Conflicts: # docs/config-catalog.md # docs/testing.i18n.yaml # docs/testing.md # docs/testing.zh.md # packages/host/runtime/src/start.ts # packages/support/acp-snapshot/README.md
This commit is contained in:
Tianyi Cui
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-06-19-acp-snapshot-tests.md: 43900632c4e5e3904c2d0e3b75f2a3fe3b3a50cc
|
||||
2026-06-19-acp-snapshot-tests.zh.md: a4613b19001afe87c1f6bdb61196fabbfa947a4c
|
||||
2026-06-19-acp-snapshot-tests.md: b4cda8f32fe7a84a977bcbdbe5db0671cb9a7083
|
||||
2026-06-19-acp-snapshot-tests.zh.md: 5337c3852b524af4e8c556e93ec80084b30a6d0b
|
||||
|
||||
@@ -6,7 +6,7 @@ English | [中文](2026-06-19-acp-snapshot-tests.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
Unit tests do not exercise the complete ACP subprocess transcript, while real-API tests are nondeterministic and key-gated. Editor-facing `session/update` output can therefore regress despite green unit coverage, as the [default-export postmortem](../../../../docs/postmortem/0001-acp-default-export-drops-inject.md) demonstrated.
|
||||
Unit tests do not exercise the complete assembled-agent subprocess or its ACP automation wire, while real-API tests are nondeterministic and key-gated. Loader wiring, backend behavior, and protocol output can therefore regress despite green unit coverage, as the [default-export postmortem](../../../../docs/postmortem/0001-acp-default-export-drops-inject.md) demonstrated.
|
||||
|
||||
The blocker for a full-transcript test is the model: the agent's output is driven by a non-deterministic LLM, and a key-gated test that hits the real API on every run is neither deterministic nor CI-runnable. We want the fidelity of a real run with the determinism of a fixture.
|
||||
|
||||
@@ -52,10 +52,10 @@ Replay uses a `cordis.snapshot.yml` overlay that replaces the real adapter with
|
||||
|
||||
A snapshot run asserts **two** normalized surfaces, because the harness's external surfaces are distinct:
|
||||
|
||||
1. The **stdout transcript** — the framed `session/update` JSON-RPC the editor sees. Catches regressions in the ACP bridge's event→update translation (`streamSessionEventUpdate`). Compared against a committed `stdout.expected.jsonl`.
|
||||
1. The **stdout transcript** — the framed ACP JSON-RPC responses and committed-message updates an automation client receives. It catches regressions in the transport contract and is compared against a committed `stdout.expected.jsonl`.
|
||||
2. The **re-persisted session JSONL**, normalized and compared with `session.jsonl`. The same fixture is both replay source and expected log. Prompt text is scrubbed; one scenario per header class pins readable prompt and tool content as described in the [header-pinning Agent Note](2026-07-06-pin-request-header-content-in-one-scenario.md). Override scenarios derive model behavior solely from their sidecar.
|
||||
|
||||
The surfaces are complementary: stdout covers bridge projection, while JSONL covers loop, tool, and boundary structure that the projection omits.
|
||||
The surfaces are complementary: stdout covers the minimal automation wire, while JSONL covers loop, tool, and boundary structure that the wire intentionally omits.
|
||||
|
||||
Normalization replaces session, cwd, protocol-id, timestamp, path, and process volatility while preserving deterministic sequence numbers. Scenarios constrain real bash use to stable commands. The stdout expected output remains wire-shaped JSONL and every raw line must parse as JSON. Vitest updates only the stdout expected output; normalized session equality never overwrites the replay fixture.
|
||||
|
||||
@@ -79,6 +79,6 @@ Tool determinism comes from a generated cwd, scrubbed environment, fresh non-log
|
||||
|
||||
## Consequences
|
||||
|
||||
The new tier adds reviewed per-scenario input, session, stdout, optional override, and optional workspace fixtures. Workspace seeds are copied into the generated cwd for both record and replay. In return the tier provides deterministic keyless transcript coverage through the real Loader and tool composition. The subprocess, input, workspace, normalization, and replay harness can support examples beyond ACP.
|
||||
The tier adds reviewed per-scenario input, session, stdout, optional override, and optional workspace fixtures. Workspace seeds are copied into the generated cwd for both record and replay. In return the tier provides deterministic keyless coverage through the real Loader and tool composition. Most retained scenarios exercise the assembled backend rather than ACP; the [automation-only ACP decision](../simplification/2026-07-23-acp-automation-only-protocol.md#snapshot-boundary) keeps that corpus here and defers any move to a transport-neutral headless suite as an independent testing change (the suite-level FIXME marks it).
|
||||
|
||||
This Agent Note relates to but does not supersede the [proposed determinism Agent Note](../../proposed/testing/2026-06-11-deterministic-and-stress-testing.md): that proposal's "universal replay fixture" re-derives session *message history* after every test (an internal-consistency invariant), whereas snapshot tests pin the *external protocol output*. They are complementary — one guards the event-sourcing invariant, the other guards the editor-facing contract.
|
||||
This Agent Note relates to but does not supersede the [proposed determinism Agent Note](../../proposed/testing/2026-06-11-deterministic-and-stress-testing.md): that proposal's "universal replay fixture" re-derives session *message history* after every test (an internal-consistency invariant), whereas these snapshots pin assembled behavior plus the external automation output. They are complementary until the backend corpus moves off ACP.
|
||||
|
||||
@@ -6,9 +6,9 @@ Status: implemented
|
||||
|
||||
## 问题
|
||||
|
||||
单元测试不会覆盖完整的 ACP（Agent Client Protocol）子进程 transcript（文本记录），而真实 API 测试不具确定性且受密钥门控。因此，即使单元覆盖率为绿色，面向编辑器的 `session/update` 输出仍可能回归，[默认导出事后分析](../../../../docs/postmortem/0001-acp-default-export-drops-inject.md)已经证明了这一点。
|
||||
单元测试不会覆盖组装后的完整 agent（智能体）子进程及其 ACP（Agent Client Protocol）自动化线协议，而真实 API 测试不具确定性且受密钥门控。因此，即使单元覆盖率为绿色，Loader 接线、后端行为和协议输出仍可能回归，[默认导出事后分析](../../../../docs/postmortem/0001-acp-default-export-drops-inject.md)已经证明了这一点。
|
||||
|
||||
全 transcript 测试的阻塞因素在于模型：agent（智能体）的输出由非确定性的 LLM（大语言模型）驱动，而每次运行都命中真实 API 的密钥门控测试既不确定也无法在 CI 中运行。我们需要真实运行的保真度与 fixture（测试前置数据）的确定性兼得。
|
||||
全 transcript（文本记录）测试的阻塞因素在于模型：agent 的输出由非确定性的 LLM（大语言模型）驱动，而每次运行都命中真实 API 的密钥门控测试既不确定也无法在 CI 中运行。我们需要真实运行的保真度与 fixture（测试前置数据）的确定性兼得。
|
||||
|
||||
本 Agent Note（agent 决策记录）记下了新增第三层测试——**快照测试**——的决策，以及让它具备确定性、在 CI 中无需密钥、且维护成本低廉的设计选择。
|
||||
|
||||
@@ -52,10 +52,10 @@ Status: implemented
|
||||
|
||||
快照运行断言**两个**归一化后的表面，因为 harness 的外部表面是不同的：
|
||||
|
||||
1. **stdout transcript**——编辑器看到的、经过 framing 的 `session/update` JSON-RPC。用于捕获 ACP bridge 中事件→更新转换（`streamSessionEventUpdate`）的回归。与已提交的 `stdout.expected.jsonl` 比较。
|
||||
1. **stdout transcript**——自动化客户端收到的、经过 framing 的 ACP JSON-RPC 响应与已提交的消息更新。它捕获传输契约的回归，与已提交的 `stdout.expected.jsonl` 比较。
|
||||
2. **重新持久化的会话 JSONL**，经过规范化后与 `session.jsonl` 比较。同一 fixture 同时作为重放来源和预期日志。提示词文本会被清理；按照[请求头固定 Agent Note](2026-07-06-pin-request-header-content-in-one-scenario.md)所述，每种请求头类别由一个场景固定可读提示词与工具内容。Override 场景仅从其 sidecar 派生模型行为。
|
||||
|
||||
两个表面互补：stdout 覆盖 bridge 投影，JSONL 覆盖投影所省略的 loop、工具和 boundary 结构。
|
||||
两个表面互补：stdout 覆盖精简的自动化线协议，JSONL 覆盖线协议有意省略的 loop、工具和 boundary 结构。
|
||||
|
||||
规范化会替换会话、cwd、协议 id、时间戳、路径和进程易变值，同时保留确定性序号。场景把真实 bash 使用限制在稳定命令上。stdout 预期输出仍是线协议形状的 JSONL，每个原始行都必须可解析为 JSON。Vitest 只更新 stdout 预期输出；规范化会话相等性检查从不覆盖重放 fixture。
|
||||
|
||||
@@ -79,6 +79,6 @@ Status: implemented
|
||||
|
||||
## 后果
|
||||
|
||||
新测试层为每个场景增加经过评审的输入、会话、stdout、可选 override 和可选 workspace fixture。记录与重放都会把 workspace seed 复制到生成的 cwd。作为回报，该层通过真实 Loader 和工具组合提供确定性的无密钥 transcript 覆盖。子进程、输入、workspace、规范化和重放 harness 也可以支持 ACP 之外的示例。
|
||||
该测试层为每个场景增加经过评审的输入、会话、stdout、可选 override 和可选 workspace fixture。记录与重放都会把 workspace seed 复制到生成的 cwd。作为回报，该层通过真实 Loader 和工具组合提供确定性的无密钥覆盖。保留下来的大多数场景测试的是组装后的后端而非 ACP；[仅面向自动化的 ACP 决策](../simplification/2026-07-23-acp-automation-only-protocol.md#snapshot-boundary)将该语料保留在此处，并把向传输无关 headless 套件的任何迁移推迟为一项独立的测试变更（套件级 FIXME 标记了这一点）。
|
||||
|
||||
本 Agent Note 与[拟议的确定性 Agent Note](../../proposed/testing/2026-06-11-deterministic-and-stress-testing.md)相关，但不取代它：该提案的“通用重放 fixture”在每次测试后重新派生会话*消息历史*（内部一致性不变量），而快照测试固定*外部协议输出*。两者相互补充——一个守护事件溯源不变量，另一个守护面向编辑器的契约。
|
||||
本 Agent Note 与[拟议的确定性 Agent Note](../../proposed/testing/2026-06-11-deterministic-and-stress-testing.md)相关，但不取代它：该提案的“通用重放 fixture”在每次测试后重新派生会话*消息历史*（内部一致性不变量），而这些快照固定组装后的行为与外部自动化输出。在后端语料迁出 ACP 之前，两者相互补充。
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-06-19-real-api-e2e-ci.md: a9289980ada8ab6390b5daa488f07ec258db1bd5
|
||||
2026-06-19-real-api-e2e-ci.zh.md: dcb5759d26e7680d392a4855665b2eb88551ba79
|
||||
2026-06-19-real-api-e2e-ci.md: 935664fd01df4844ee19be7b4f2f297ebf5bd29b
|
||||
2026-06-19-real-api-e2e-ci.zh.md: 9c10614f5a7e6b38f6850b29fab87d0e09806c5f
|
||||
|
||||
@@ -6,7 +6,7 @@ English | [中文](2026-06-19-real-api-e2e-ci.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
The harness leans hard on real-API tests by policy: [docs/testing.md](../../../../docs/testing.md) argues that a no-key suite proves the plumbing but not the product, and the [ACP inject postmortem](../../../../docs/postmortem/0001-acp-default-export-drops-inject.md) is the standing proof — 178 keyless tests stayed green while a real editor session crashed instantly. The real-API e2e suite (`pnpm run test:e2e`, the `*.e2e.ts` files) exists precisely to close that gap: it drives the agent against the live DeepSeek API — real model calls, real bash tools, multi-turn, resume, ACP-over-stdio.
|
||||
The harness leans hard on real-API tests by policy: [docs/testing.md](../../../../docs/testing.md) argues that a no-key suite proves the plumbing but not the product, and the [ACP inject postmortem](../../../../docs/postmortem/0001-acp-default-export-drops-inject.md) is the standing proof — 178 keyless tests stayed green while a real ACP client session crashed instantly. The real-API e2e suite (`pnpm run test:e2e`, the `*.e2e.ts` files) exists precisely to close that gap: it drives the agent against the live DeepSeek API — real model calls, real bash tools, multi-turn, resume, ACP-over-stdio.
|
||||
|
||||
The default gate ([.github/workflows/ci.yml](../../../../.github/workflows/ci.yml)) is deliberately keyless: it carries no secret and runs for forks. `test:e2e` self-skips without a key (`describe.skipIf(!process.env.DEEPSEEK_API_KEY)`), so adding it there would report green without exercising the real suite. A separate secret-bearing workflow is required to make real-API coverage a merge signal.
|
||||
|
||||
|
||||
@@ -6,7 +6,7 @@ Status: implemented
|
||||
|
||||
## 问题
|
||||
|
||||
根据策略，harness 高度依赖真实 API 测试：[docs/testing.md](../../../../docs/testing.md) 指出，无密钥套件证明的是管线，而非产品；[ACP（Agent Client Protocol）inject 事后分析](../../../../docs/postmortem/0001-acp-default-export-drops-inject.md)则是常设证据——178 项无密钥测试保持绿色时，真实编辑器会话却立即崩溃。真实 API e2e 套件（`pnpm run test:e2e`，即 `*.e2e.ts` 文件）的存在正是为了弥合这一缺口：它针对实时 DeepSeek API 驱动 agent（智能体）——真实模型调用、真实 bash 工具、多轮次、恢复、ACP-over-stdio。
|
||||
根据策略，harness 高度依赖真实 API 测试：[docs/testing.md](../../../../docs/testing.md) 指出，无密钥套件证明的是管线，而非产品；[ACP（Agent Client Protocol）inject 事后分析](../../../../docs/postmortem/0001-acp-default-export-drops-inject.md)则是常设证据——178 项无密钥测试保持绿色时，真实 ACP 客户端会话却立即崩溃。真实 API e2e 套件（`pnpm run test:e2e`，即 `*.e2e.ts` 文件）的存在正是为了弥合这一缺口：它针对实时 DeepSeek API 驱动 agent（智能体）——真实模型调用、真实 bash 工具、多轮次、恢复、ACP-over-stdio。
|
||||
|
||||
默认门禁（[.github/workflows/ci.yml](../../../../.github/workflows/ci.yml)）刻意无密钥：不携带 secret，可供 fork 运行。`test:e2e` 在无密钥时自动跳过（`describe.skipIf(!process.env.DEEPSEEK_API_KEY)`），因此将其加入该工作流只会报绿而不会真正执行真实套件。要让真实 API 覆盖率成为合并信号，需要一个独立的、携带 secret 的工作流。
|
||||
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-06-22-subagent-snapshot-replay.md: 4aad8b6ddd3e4f8b8ad5565e10b263953d81ea31
|
||||
2026-06-22-subagent-snapshot-replay.zh.md: 96890bc98a64357d9785b8081c8ba123bb3c716e
|
||||
2026-06-22-subagent-snapshot-replay.md: 6e5e94308ed145b83160146fd9e9ef023f2dde5d
|
||||
2026-06-22-subagent-snapshot-replay.zh.md: 82bb7d0735c7dbf918941d00ee4c59498cc59085
|
||||
|
||||
@@ -6,7 +6,7 @@ English | [中文](2026-06-22-subagent-snapshot-replay.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
The snapshot tier (`pnpm run test:snapshot`) boots the real `acp-agent` subprocess, replays a recorded session through [`dsh-llm-replay`](../../../../packages/support/llm-replay), and diffs the normalized stdout transcript + re-persisted session log against committed expected outputs. It is the only tier that exercises the full editor-facing transcript end to end.
|
||||
The snapshot tier (`pnpm run test:snapshot`) boots the real `acp-agent` subprocess, replays a recorded session through [`dsh-llm-replay`](../../../../packages/support/llm-replay), and diffs the normalized automation wire + re-persisted session log against committed expected outputs. Most scenarios exercise assembled backend behavior through that real process boundary.
|
||||
|
||||
It was built for ONE session per process, and that assumption is wired into two places:
|
||||
|
||||
|
||||
@@ -6,12 +6,12 @@ Status: implemented
|
||||
|
||||
## 问题
|
||||
|
||||
快照层（`pnpm run test:snapshot`）会启动真实 `acp-agent` 子进程，通过 [`dsh-llm-replay`](../../../../packages/support/llm-replay) 重放已记录会话，并将规范化 stdout transcript（文本记录）+ 重新持久化的会话日志与已提交预期输出进行 diff。它是唯一端到端覆盖完整面向编辑器 transcript 的测试层。
|
||||
快照层（`pnpm run test:snapshot`）会启动真实 `acp-agent` 子进程，通过 [`dsh-llm-replay`](../../../../packages/support/llm-replay) 重放已记录会话，并将规范化后的自动化线协议 + 重新持久化的会话日志与已提交预期输出进行 diff。大多数场景通过这条真实进程边界测试组装后的后端行为。
|
||||
|
||||
该层最初为每个进程只有一个会话而构建，这一假设硬编码在两处：
|
||||
|
||||
- **`dsh-llm-replay` 没有做任何键控。** 它用一个全局游标，将第 N 次 `llm/stream` 调用对应到单一录制序列的第 N 条。当父 agent（智能体）和一个进程内 subagent 在同一个上下文上同时流式输出时，调用交错，单一游标会把子 agent 的脚本发给父 agent（反之亦然）。
|
||||
- **harness 只收集一份日志。** `findSessionLog` 遍历 sessions 根目录，返回找到的第一个 `.jsonl`。subagent 作为第二个 `Session` 运行，在同一个 cwd bucket 下有自己的日志，因此子 agent 的 transcript 被静默丢弃。
|
||||
- **harness 只收集一份日志。** `findSessionLog` 遍历 sessions 根目录，返回找到的第一个 `.jsonl`。subagent 作为第二个 `Session` 运行，在同一个 cwd bucket 下有自己的日志，因此子 agent 的 transcript（文本记录）被静默丢弃。
|
||||
|
||||
这就是 [subagent seam Agent Note（agent 决策记录）](../feature/2026-06-21-subagent-capability-seam.md)中通过 `TODO(subagent-snapshots)` 推迟的工作：进程内后端（PR2）落地时已有单元 + e2e 覆盖，但在这套基础设施落地前，完整 transcript 快照层无法表达嵌套 agent 形状。本 Agent Note 就是该堆叠式后续工作。
|
||||
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-07-04-hook-snapshot-matrix.md: ceb91a70e1f9582cf2a7cb4cec0ba8aaf5699b8e
|
||||
2026-07-04-hook-snapshot-matrix.zh.md: 250995dd1aae566aa5b9d40a14ed281b14fbfe0b
|
||||
2026-07-04-hook-snapshot-matrix.md: 98f9db27fd8afaaa99c9985dff4d148ef4eb926b
|
||||
2026-07-04-hook-snapshot-matrix.zh.md: 40b9ee84ad7232ac806096375e8da42bd02bfddf
|
||||
|
||||
@@ -8,7 +8,7 @@ English | [中文](2026-07-04-hook-snapshot-matrix.zh.md)
|
||||
|
||||
The hook bridges — [`dsh-hooks-claude`](../../../../packages/hooks/hooks-claude) (7 Claude Code hook points) and [`dsh-hooks-codex`](../../../../packages/hooks/hooks-codex) (5 Codex points) — map external hook commands onto the harness interception seams. They carry deep unit and coverage-spec coverage (every decision arm, every payload dialect, driven against a mocked seam) plus one key-gated e2e (`hooks.e2e.ts`, a live `PreToolUse` block). But the full-transcript snapshot tier — the one net that boots the real `acp-agent` subprocess, replays a recorded session keyless, and diffs the normalized ACP stdout + re-persisted log against committed expected outputs — covered exactly ONE hook: a Claude `UserPromptSubmit` block (`hook-cc-promptsubmit-block`).
|
||||
|
||||
That is the tier a mocked unit test structurally cannot be: it exercises the REAL bridge translating a REAL hook process's outcome into the REAL seam decision, then the REAL loop's reaction, rendered exactly as an editor sees it. A bridge-translation or loop-structure regression that left every unit green would still escape it for every hook point but one — and for the Codex bridge, the ACP example did not even LOAD it, so no Codex hook could fire end-to-end at all.
|
||||
That is the tier a mocked unit test structurally cannot be: it exercises the REAL bridge translating a REAL hook process's outcome into the REAL seam decision, then the REAL loop's reaction through the automation wire and persisted log. A bridge-translation or loop-structure regression that left every unit green would still escape it for every hook point but one — and for the Codex bridge, the ACP example did not even LOAD it, so no Codex hook could fire end-to-end at all.
|
||||
|
||||
## Decision
|
||||
|
||||
|
||||
@@ -8,7 +8,7 @@ Status: implemented
|
||||
|
||||
钩子 bridge——[`dsh-hooks-claude`](../../../../packages/hooks/hooks-claude)（7 个 Claude Code 钩子点）和 [`dsh-hooks-codex`](../../../../packages/hooks/hooks-codex)（5 个 Codex 点）——把外部钩子命令映射到 harness 拦截 seam。它们有深入的单元与覆盖率规格覆盖（每个决策分支、每种 payload dialect，针对 mock seam 驱动），外加一个受密钥门控的 e2e（`hooks.e2e.ts`，实时 `PreToolUse` 阻止）。但完整 transcript（文本记录）快照层——会启动真实 `acp-agent` 子进程、无需密钥重放已记录会话，并将规范化 ACP（Agent Client Protocol）stdout + 重新持久化日志与已提交预期输出进行 diff 的那张网——只覆盖了一个钩子：Claude `UserPromptSubmit` 阻止（`hook-cc-promptsubmit-block`）。
|
||||
|
||||
这正是 mock 单元测试在结构上无法替代的层级：它验证的是真实 bridge 将真实钩子进程的结果翻译到真实 seam 决策，再到真实 agent loop（智能体循环）的反应，渲染结果与编辑器看到的完全一致。一个 bridge 翻译或 loop 结构的回归，即使让所有单元测试保持绿色，也会在除那一个钩子点之外的所有点上逃逸；而对于 Codex bridge，ACP 示例甚至没有加载它，因此没有任何 Codex 钩子能端到端触发。
|
||||
这正是 mock 单元测试在结构上无法替代的层级：它验证的是真实 bridge 将真实钩子进程的结果翻译到真实 seam 决策，再经由自动化线协议和持久化日志检验真实 agent loop（智能体循环）的反应。一个 bridge 翻译或 loop 结构的回归，即使让所有单元测试保持绿色，也会在除那一个钩子点之外的所有点上逃逸；而对于 Codex bridge，ACP 示例甚至没有加载它，因此没有任何 Codex 钩子能端到端触发。
|
||||
|
||||
## 决策
|
||||
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write
|
||||
2026-07-24-web-gui-browser-e2e-lane.md: 9bab4d1eb9ec24acae0057143379629c87194d1f
|
||||
2026-07-24-web-gui-browser-e2e-lane.zh.md: c61aee7ae5c070d1c82dda0ab78d8db44c16b9bf
|
||||
2026-07-24-web-gui-browser-e2e-lane.md: b6e62f59e12c64dd5386eaaabe15e863ef52e291
|
||||
2026-07-24-web-gui-browser-e2e-lane.zh.md: 9f806c7030336bad4f7a9ca7695a03982d8a7878
|
||||
|
||||
@@ -10,15 +10,15 @@ The web GUI ships as a real assembled chain — chromium page → client plugin
|
||||
|
||||
## Decision
|
||||
|
||||
`pnpm run test:web` carries a keyless, deterministic browser e2e lane under `apps/web/tests/`: recorded session-log fixtures replayed through `@deepseek-ai/dsh-llm-replay` against the real in-process web assembly, asserting a normalized conversation aria golden plus in-process world state. No new package; the product deltas are the `BootHostOptions.llm` seam and two additive `dsh-llm-replay` surfaces.
|
||||
`pnpm run test:web` carries a keyless, deterministic browser e2e lane under `apps/web/tests/`: recorded session-log fixtures replayed through `@deepseek-ai/dsh-llm-replay` against the real in-process web composition, asserting a normalized conversation aria golden plus in-process world state. No new package; the product deltas are two additive `dsh-llm-replay` surfaces (`paceMs`, `ReplayHandle`).
|
||||
|
||||
### Scaffold: `apps/web/tests/scaffold.ts`
|
||||
|
||||
A plain shared-fixture module (the [testing-policy sanctioned shape](../../../../docs/testing.md)), not a package: the gate-worthy logic — replay derivation, session parsing, log scrubbing, persistence — lives in the gated packages `dsh-llm-replay`, `dsh-acp-snapshot`, and `dsh-session-persistence-jsonl`; what remains is boot wiring and browser glue, and chromium-driving code cannot hold per-file 100% coverage on the browserless coverage runners.
|
||||
|
||||
`launchWebScaffold()` boots the real web assembly in-process from the exported production functions — `startHost({ boot: { …, llm: false } })`, `installLlmReplay(host.ctx, { file, providers, paceMs })`, `mountWebPlugins(host.ctx, roster, anchor)`, `createHostWebPluginRegistry`, `startWebServer({ port: 0, … })`. This is the web analog of the TUI suite mounting the production bundle in-process ([TUI snapshots](2026-07-18-tui-terminal-state-snapshots.md)): the real entry boundary (`dsh web` bin arg-parsing, dist resolution) stays held by the keyless CLI smokes in `smoke-real.e2e.ts`, and the web surface has no `cordis.yml` to bypass — assembly is written in the app per the [GUI layering decision](../architecture/2026-07-19-gui-layering-and-rpc-protocol.md), a ruling this lane's design review explicitly reaffirmed (Loader-izing `dsh web` was declined; it would be its own proposal). Two deliberate assembly divergences from the `dsh web` shell, noted in the scaffold header: `workspaceContext: false` (recorded fixtures must not embed this repo's AGENTS.md) and `sessionTitleLlm` left at bootHost's disabled default (its fire-and-forget title call would share the session's replay cursor nondeterministically).
|
||||
`launchWebScaffold()` boots the real web composition: the shipped `apps/cli/cordis.yml` through the vendored Loader's include boot — the same tree and mechanism `AppCLIEntry` drives for `dsh web` (the config-tree boot landed upstream on 2026-07-25, superseding this lane's earlier in-process `startHost` assembly and resolving the original Loader-ization question in favor of Loader-izing). Divergences ride include patches over the SAME shipped tree, the ACP `cordis.snapshot.yml` pattern expressed in-process: temp `persistenceRoot`, `workspace-context` disabled (recorded fixtures must not embed this repo's AGENTS.md), `session-title-llm` disabled (its fire-and-forget title call would race the loop for the session's replay cursor), the webserver row pinned to port 0 with the built dist, and in keyless modes `llm-deepseek` disabled. A patch id that stops matching a row fails the boot sweep loudly instead of drifting. The boot runs `chdir`'d to the temp workspace so the api-gateway's `process.cwd()` session default, tool cwds, and fixtures agree; the `dsh web` bin's own glue (argv, profile json, AppCLIEntry) stays held by the keyless CLI smokes in `smoke-real.e2e.ts`.
|
||||
|
||||
The `llm: false` seam is the reviewed resolution of the keyless-boot question: `'deepseek' | false` on `BootHostOptions`, matching the `workspaceContext: Config | false` shape, with `RunningHost.ctx` JSDoc naming "filling a deliberately-open capability seam" as its third sanctioned use. Replay runs in providers-catalog mode with a published `contextWindow` (the TUI `PROVIDERS` shape), never catch-all: with no adapter registered, catch-all would leave `resolveModelContext` unroutable and `compact-basic`'s post-step pressure check would warn every step instead of being provably inert.
|
||||
Keyless model displacement is the disabled adapter row plus `installLlmReplay` filling the open seam on the settled root ctx in providers-catalog mode — never catch-all: with the adapter row disabled no adapter exists, so catch-all would leave `resolveModelContext` unroutable and `compact-basic`'s post-step pressure check would warn every step instead of being provably inert (the published 128k `contextWindow` keeps it inert for small fixtures). The direct install rather than an inserted replay plugin row is deliberate: it returns the `ReplayHandle` the teardown consumption check needs. A scenario with no fixture leaves the seam empty, so a stray stream fails loud with NO_ADAPTER. (The first round's `BootHostOptions.llm: 'deepseek' | false` seam was superseded by the config-tree boot and removed with `bootHost`'s web role.)
|
||||
|
||||
`seedSession()` seeds cold sessions through the real persistence API — a throwaway `Context` mounting `SessionStore` + `SessionPersistenceJsonl` against the host's root, `create()` + `append()`, one `utimes` backdate for deterministic sidebar order (the `semantic-checkpoint.snapshot.ts` precedent) — never raw file writes, so the seeder knows nothing of bucket hashing, filename encoding, or compression, and the host's zstd default needs no boot knob. Seeds are validated at seed time (parseable, ending in `turn/end` — an open final turn would be mutated by resume's crash repair).
|
||||
|
||||
@@ -61,7 +61,7 @@ Surveyed AI-chat/agent web UIs and mocking layers (LibreChat, vercel/ai-chatbot
|
||||
|
||||
**Growing the `?fixture` client.** Rejected: tier separation — `FixtureApiClient` exists to test the client shell without a server; everything below the client API seam stays untested by construction.
|
||||
|
||||
**Placeholder `DEEPSEEK_API_KEY` + replay interception instead of the `llm: false` seam.** Rejected despite zero product change and two in-tree precedents: it satisfies `llm-deepseek`'s fail-loud key check with a lie and leaves a dead adapter mounted-but-intercepted; the seam matches an existing option shape and fails loud at the earliest resolvable point.
|
||||
**Placeholder `DEEPSEEK_API_KEY` + replay interception instead of disabling the adapter row.** Rejected despite zero composition change and two in-tree precedents: it satisfies `llm-deepseek`'s fail-loud key check with a lie and leaves a dead adapter mounted-but-intercepted; the disabled row (the ACP overlay's move) is honest keylessness and fails loud at the earliest resolvable point.
|
||||
|
||||
**A `packages/support/web-snapshot` package with a `defineWebSnapshotSuite` factory.** Rejected: chromium-driving source cannot honestly hold per-file 100% coverage on browserless coverage runners, and at two scenarios a factory generalizes from one consumer while the genuinely shared logic is already exported from gated packages. Re-entry trigger: a second web-shaped consumer or ≥6 scenarios with demonstrably drifting inline branches; the package boundary would then be drawn browser-free.
|
||||
|
||||
@@ -77,7 +77,7 @@ Surveyed AI-chat/agent web UIs and mocking layers (LibreChat, vercel/ai-chatbot
|
||||
|
||||
## Testing
|
||||
|
||||
The lane itself: `pnpm run test:web` runs both scenarios keylessly alongside the existing smoke pair; `DSH_SNAPSHOT=record pnpm exec vitest run --config vitest.web.config.ts apps/web/tests/<spec>` re-records a scenario's fixture against the live model; `DSH_SNAPSHOT=refresh` rewrites both aria goldens keylessly. The `llm: false` seam is pinned by `packages/host/runtime/tests/host-runtime.spec.ts` (keyless boot, NO_ADAPTER at first stream, embedder fill through ctx); `paceMs` validation, pacing floor, abort-during-pace, and both `assertConsumed` failure shapes are pinned in `packages/support/llm-replay/tests/llm-replay.spec.ts`.
|
||||
The lane itself: `pnpm run test:web` runs both scenarios keylessly alongside the existing smoke pair; `DSH_SNAPSHOT=record pnpm exec vitest run --config vitest.web.config.ts apps/web/tests/<spec>` re-records a scenario's fixture against the live model; `DSH_SNAPSHOT=refresh` rewrites both aria goldens keylessly. `paceMs` validation, pacing floor, abort-during-pace, and both `assertConsumed` failure shapes are pinned in `packages/support/llm-replay/tests/llm-replay.spec.ts`.
|
||||
|
||||
## Deferred
|
||||
|
||||
|
||||
@@ -10,15 +10,15 @@ Web GUI 以一条真实组装链交付——chromium 页面 → client 插件 bu
|
||||
|
||||
## 决策
|
||||
|
||||
`pnpm run test:web` 携带 `apps/web/tests/` 下的无密钥、确定性浏览器 e2e 车道：录制的会话日志 fixture 经 `@deepseek-ai/dsh-llm-replay` 对真实进程内 web 组装回放，断言规范化后的会话区 aria 预期输出加进程内世界状态。不新增包（package）；产品侧增量只有 `BootHostOptions.llm` seam 和 `dsh-llm-replay` 的两处增量接口。
|
||||
`pnpm run test:web` 携带 `apps/web/tests/` 下的无密钥、确定性浏览器 e2e 车道：录制的会话日志 fixture 经 `@deepseek-ai/dsh-llm-replay` 对真实进程内 web 组合回放，断言规范化后的会话区 aria 预期输出加进程内世界状态。不新增包（package）；产品侧增量只有 `dsh-llm-replay` 的两处增量接口（`paceMs`、`ReplayHandle`）。
|
||||
|
||||
### Scaffold：`apps/web/tests/scaffold.ts`
|
||||
|
||||
一个普通的共享 fixture 模块（[测试政策认可的形态](../../../../docs/testing.md)），不是包：值得门禁把守的逻辑——回放推导、会话解析、日志脱敏、持久化——都在已受门禁的包 `dsh-llm-replay`、`dsh-acp-snapshot`、`dsh-session-persistence-jsonl` 中；剩下的只是启动接线和浏览器胶水，而驱动 chromium 的源码在无浏览器的覆盖率 runner 上无法诚实保持逐文件 100% 覆盖率。
|
||||
|
||||
`launchWebScaffold()` 用导出的生产函数在进程内启动真实 web 组装——`startHost({ boot: { …, llm: false } })`、`installLlmReplay(host.ctx, { file, providers, paceMs })`、`mountWebPlugins(host.ctx, roster, anchor)`、`createHostWebPluginRegistry`、`startWebServer({ port: 0, … })`。这是 TUI 套件进程内挂载生产 bundle 的 web 对应物（[TUI 快照](2026-07-18-tui-terminal-state-snapshots.md)）：真实入口边界（`dsh web` bin 的参数解析、dist 解析）仍由 `smoke-real.e2e.ts` 中的无密钥 CLI 冒烟把守，且 web 表面没有可绕过的 `cordis.yml`——按[GUI 分层决策](../architecture/2026-07-19-gui-layering-and-rpc-protocol.md)，组装写在应用里；本车道的设计评审明确重申了这一裁定（Loader 化 `dsh web` 被否决；那需要自己的提案）。与 `dsh web` shell 的两处刻意组装差异已注明在 scaffold 头部：`workspaceContext: false`（录制的 fixture 不得嵌入本仓库的 AGENTS.md），以及 `sessionTitleLlm` 保持 bootHost 的禁用默认值（其发后不管的标题调用会与循环自身的调用不确定地共享会话的回放游标）。
|
||||
`launchWebScaffold()` 启动真实 web 组合：经 vendored Loader 的 include boot 加载交付的 `apps/cli/cordis.yml`——与 `AppCLIEntry` 为 `dsh web` 驱动的是同一棵树、同一套机制（配置树 boot 于 2026-07-25 在上游落地，取代了本车道第一轮的进程内 `startHost` 组装，也把当初的 Loader 化问题裁定为「Loader 化」）。差异全部经 include patch 骑在同一棵交付树上，即 ACP `cordis.snapshot.yml` 模式的进程内表达：临时 `persistenceRoot`；禁用 `workspace-context`（录制的 fixture 不得嵌入本仓库的 AGENTS.md）；禁用 `session-title-llm`（其发后不管的标题调用会与循环争抢会话的回放游标）；webserver 行钉到端口 0 加已构建 dist；无密钥模式下禁用 `llm-deepseek`。patch 的 id 一旦不再匹配任何行，boot 扫描会大声失败而不是漂移。boot 在临时工作区 `chdir` 下运行，使 api-gateway 的 `process.cwd()` 会话默认值、工具 cwd 与 fixture 一致；`dsh web` bin 自身的胶水（argv、profile json、AppCLIEntry）仍由 `smoke-real.e2e.ts` 中的无密钥 CLI 冒烟把守。
|
||||
|
||||
`llm: false` seam 是无密钥启动问题经评审后的定论：`BootHostOptions` 上的 `'deepseek' | false`，与 `workspaceContext: Config | false` 形态一致，且 `RunningHost.ctx` 的 JSDoc 把「填充刻意开放的能力 seam」列为其第三种认可用法。回放必须以提供方目录（providers-catalog）模式运行并发布 `contextWindow`（TUI 的 `PROVIDERS` 形态），绝不用 catch-all：没有注册适配器时，catch-all 会让 `resolveModelContext` 无路由可走，`compact-basic` 的步后压力检查将步步告警，而不是被可证明地闲置。
|
||||
无密钥的模型替换 = 禁用适配器行的 patch 加 `installLlmReplay` 在停稳的根 ctx 上以提供方目录（providers-catalog）模式填充开放的 seam——绝不用 catch-all：适配器行被禁用后不存在任何适配器，catch-all 会让 `resolveModelContext` 无路由可走，`compact-basic` 的步后压力检查将步步告警，而不是被可证明地闲置（发布的 128k `contextWindow` 使该路径对小 fixture 保持闲置）。选择直接安装而非插入回放插件行是刻意的：直接安装返回收尾消费检查所需的 `ReplayHandle`。没有 fixture 的场景让 seam 保持空置，任何离群的流式调用都会以 NO_ADAPTER 大声失败。（第一轮的 `BootHostOptions.llm: 'deepseek' | false` seam 已随配置树 boot 取代 `bootHost` 的 web 角色而移除。）
|
||||
|
||||
`seedSession()` 通过真实持久化 API 播种冷会话——一次性 `Context` 挂载 `SessionStore` + `SessionPersistenceJsonl` 指向 host 的根目录，`create()` + `append()`，一次 `utimes` 回拨保证侧栏顺序确定（`semantic-checkpoint.snapshot.ts` 先例）——绝不裸写文件，因此播种器对桶哈希、文件名编码、压缩一无所知，host 的 zstd 默认值也无需任何启动开关。种子在播种时即校验（可解析、以 `turn/end` 结尾——未闭合的最终轮次会被恢复（resume）的崩溃修复改写）。
|
||||
|
||||
@@ -61,7 +61,7 @@ Web GUI 以一条真实组装链交付——chromium 页面 → client 插件 bu
|
||||
|
||||
**扩展 `?fixture` 客户端。** 已否决：分层纪律——`FixtureApiClient` 的存在意义就是脱离服务器测试客户端 shell；client API seam 以下按构造即失测。
|
||||
|
||||
**用占位 `DEEPSEEK_API_KEY` + 回放拦截替代 `llm: false` seam。** 尽管零产品改动且树内有两处先例仍被否决：它用谎言满足 `llm-deepseek` 的快速失败密钥检查，还留下一个挂载却被拦截的死适配器；seam 方案与既有选项形态一致，并在最早可解析点快速失败。
|
||||
**用占位 `DEEPSEEK_API_KEY` + 回放拦截替代禁用适配器行。** 尽管零组合改动且树内有两处先例仍被否决：它用谎言满足 `llm-deepseek` 的快速失败密钥检查，还留下一个挂载却被拦截的死适配器；禁用行（ACP overlay 的同款做法）是诚实的无密钥，并在最早可解析点快速失败。
|
||||
|
||||
**`packages/support/web-snapshot` 包 + `defineWebSnapshotSuite` 工厂。** 已否决：驱动 chromium 的源码在无浏览器的覆盖率 runner 上无法诚实保持逐文件 100%，且两个场景就上工厂是从单一消费方过度泛化，真正共享的逻辑已从受门禁的包中导出。重启条件：出现第二个 web 形态消费方，或 ≥6 个场景的内联分支被证实各自漂移；届时包边界将画在无浏览器一侧。
|
||||
|
||||
@@ -77,7 +77,7 @@ Web GUI 以一条真实组装链交付——chromium 页面 → client 插件 bu
|
||||
|
||||
## Testing
|
||||
|
||||
车道自身：`pnpm run test:web` 与既有冒烟对一起无密钥运行两个场景；`DSH_SNAPSHOT=record pnpm exec vitest run --config vitest.web.config.ts apps/web/tests/<spec>` 对真实模型重录某场景的 fixture；`DSH_SNAPSHOT=refresh` 无密钥重写两份 aria 预期输出。`llm: false` seam 由 `packages/host/runtime/tests/host-runtime.spec.ts` 钉住（无密钥启动、首次流式调用 NO_ADAPTER、嵌入方经 ctx 填充）；`paceMs` 校验、节奏下限、节奏中中止、`assertConsumed` 的两种失败形态钉在 `packages/support/llm-replay/tests/llm-replay.spec.ts`。
|
||||
车道自身：`pnpm run test:web` 与既有冒烟对一起无密钥运行两个场景；`DSH_SNAPSHOT=record pnpm exec vitest run --config vitest.web.config.ts apps/web/tests/<spec>` 对真实模型重录某场景的 fixture；`DSH_SNAPSHOT=refresh` 无密钥重写两份 aria 预期输出。`paceMs` 校验、节奏下限、节奏中中止、`assertConsumed` 的两种失败形态钉在 `packages/support/llm-replay/tests/llm-replay.spec.ts`。
|
||||
|
||||
## 暂缓
|
||||
|
||||
|
||||
Reference in New Issue
Block a user