ci: gate web browser snapshots

2026-07-30 10:23:05 +08:00
parent 2a53806275
commit 4b4f7752c1
18 changed files with 151 additions and 37 deletions
@@ -46,7 +46,7 @@ The lane covers three behavior families. Live-turn scenarios pin ordinary tool e

 ### CI stance

-The lane ships gate-exempt inside `pnpm run test:web`, exactly as that config's header records. Adding chromium to CI would reverse the "no browser infrastructure in CI" premise in the [GUI testing note](../process/2026-07-20-gui-testing-system.md) and therefore requires its own Agent Note cross-linked from there, staged as a non-required job first with measured promotion criteria (consecutive green runs, wall time, zero-retry flake budget, runner browser-cache strategy). `TODO(ci-browser)` marks the seam. Scenarios are POSIX-oriented (the lane is not in the Windows matrix).
+The lane is a required compare-only gate for Linux pull requests under the [browser snapshot CI decision](2026-07-30-web-browser-snapshot-ci-gate.md). The static job publishes `apps/web/dist` with the package build artifacts; the consumer job installs the lockfile-selected Chromium, restores its OS-and-lockfile-keyed cache, and runs the lane with `DSH_SNAPSHOT=replay`. CI never records or refreshes goldens. The default-branch Linux serial jobs run the same comparison and seed the hosted browser cache. Scenarios remain POSIX-oriented and stay outside the Windows and macOS matrices.

 ## Prior art

@@ -76,12 +76,11 @@ Surveyed AI-chat/agent web UIs and mocking layers (LibreChat, vercel/ai-chatbot

 ## Testing

-`pnpm run test:web` runs the lane keylessly. `DSH_SNAPSHOT=record pnpm exec vitest run --config vitest.web.config.ts apps/web/tests/<spec>` records a prompting scenario against the live model, and `DSH_SNAPSHOT=refresh` rewrites aria goldens keylessly. `dsh-llm-replay` unit coverage pins pacing, cancellation, consumption diagnostics, sidecar validation, indexed replacement, and the single append position.
+`pnpm run test:web` builds and runs the lane keylessly; `test:web:built` runs it against existing build artifacts. `DSH_SNAPSHOT=record pnpm exec vitest run --config vitest.web.config.ts apps/web/tests/<spec>` records a prompting scenario against the live model, and `DSH_SNAPSHOT=refresh pnpm run test:web` rewrites aria goldens keylessly. CI explicitly selects replay mode. `dsh-llm-replay` unit coverage pins pacing, cancellation, consumption diagnostics, sidecar validation, indexed replacement, and the single append position.

 ## Deferred

 - **Web header-class pin**: web fixtures tokenize `{{system}}`/`{{tools}}` everywhere and no scenario pins the web composition's prompt/tool schemas (`TODO(web-header-pin)` — the scaffold `recordFixture` JSDoc marks it). Following the TUI scrub-everywhere precedent; revisit when the web assembly's header diverges from the repl composition it mirrors.
- **CI browser provisioning**: reversal of the no-browser-in-CI ruling, staged criteria above (`TODO(ci-browser)`).
 - **Follow-up-prompt-after-resume scenario**: the history/live stitch path over the real wire; add as its own scenario when that code changes or regresses.
 - **Web error surface**: the client consumes no `agent/error` frames and a pre-chunk failure freezes no partial, so a non-retryable provider failure renders no error copy — the user sees the send simply stop. The AUTH scenario pins the current contract (no crash, composer recovers, turn logged `error`) and `FIXME(web-error-surface)` marks where visible error text gets asserted once the UI grows an error rendering.
 - **Composer steering gesture**: the input locks while running (stop-or-wait), so the steering scenario steers over the wire from the page; `TODO(web-steer-composer)` upgrades the drive step to a real composer gesture when the product grows one.
@@ -89,4 +88,4 @@ Surveyed AI-chat/agent web UIs and mocking layers (LibreChat, vercel/ai-chatbot

 ## Consequences

-The web surface gains its record-once/replay-forever tier: the real chromium → SSE → apiproxy → loop → tools → persistence chain runs keylessly in ~10-30s, deterministic across repeat runs, with fixtures owned and re-recordable by the lane itself. Costs accepted: every intentional conversation-UI change ends with a keyless `DSH_SNAPSHOT=refresh` (golden churn is reviewed diff, anchors keep semantic green); the aria format is Playwright-owned — the one committed snapshot format the repo does not control — so playwright version bumps must be deliberate bump-and-refresh commits (the dependency floats `^1.49.0` in `apps/web/package.json`; pin exactly if churn bites); replay's first-call-order binding constrains scenarios to one prompting session each, with the consumption assertion as the tripwire; `compact-basic` shares the session's replay cursor and stays inert only under the published 128k catalog window; and the lane guards regressions only where it runs (locally, `test:web`) until the CI reversal is separately decided.
+The web surface gains its record-once/replay-forever tier: the real chromium → SSE → apiproxy → loop → tools → persistence chain runs keylessly in ~10-30s, deterministic across repeat runs, with fixtures owned and re-recordable by the lane itself. Costs accepted: every intentional conversation-UI change ends with a keyless `DSH_SNAPSHOT=refresh` (golden churn is reviewed diff, anchors keep semantic green); the aria format is Playwright-owned — the one committed snapshot format the repo does not control — so playwright version bumps must be deliberate bump-and-refresh commits (the dependency floats `^1.49.0` in `apps/web/package.json`; pin exactly if churn bites); replay's first-call-order binding constrains scenarios to one prompting session each, with the consumption assertion as the tripwire; `compact-basic` shares the session's replay cursor and stays inert only under the published 128k catalog window; and required Linux CI pays for Chromium provisioning and the serial browser run so the PR that changes the assembled UI owns its expected-output diff.