Pine/deepseek-harness
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases 1 Wiki Activity

test(hooks): snapshot the CC + Codex hook matrix end-to-end

Browse Source 
Load both hook bridges in the ACP example (dsh-hooks-claude → ./hooks.json,
dsh-hooks-codex → ./codex-hooks.json) so the full-transcript snapshot tier can
exercise each dialect against the real app. An absent config file is a silent
no-op, so a scenario carries only the file it needs and the other bridge
vanishes — verified byte-identical against every pre-existing snapshot.

Add a scenario per hook point × its headline Decision outcome, both dialects:
UserPromptSubmit block (authored, keyless) + context-fold, PreToolUse deny/ask,
PostToolUse block/context, Stop force-continue. The mid-turn scenarios are
recorded against the real API with the hook active, so the model's reaction to
a denied/blocked/force-continued turn is part of the replayed transcript.

SessionStart and SubagentStart are deliberately excluded (detached best-effort
inject races the log position — a recorded golden fails 10/10 on its own
replay), as is SubagentStop (observe-only, zero transcript footprint — a golden
could never be proven to fail). Both stay on the bridges' unit coverage. See
docs/rfc/implemented/testing/2026-07-04-hook-snapshot-matrix.md.
This commit is contained in:
Tianyi Cui
2026-07-04 00:40:35 +08:00
parent 907c8d3871
commit 74f61c89e3
58 changed files with 3860 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files
examples/AGENTS.md
+1 -1
View File
@@ -21,6 +21,6 @@ A keyless smoke that spawns the example from a temp cwd must set `TSX_TSCONFIG_P
|---|---|---|
| `echo-agent` | `tests/echo.e2e.ts` — boots the real `cordis.yml`, drives the echo tool round-trip and the direct canned reply | **N/A — keyless by nature** (the `mock-echo` model has no real provider) |
| `coding-agent` | `tests/keyless-smoke.e2e.ts` — boots the full real tree (dummy key, no prompt → no model call), asserts banner + clean exit | `tests/{full-loop,coding-task,resume,compaction,todo-write}.e2e.ts` — real model + real bash + real todo_write, world-verified |
| `acp-agent` | `pnpm run test:snapshot` — boots the real ACP subprocess and replays a recorded session keyless (incl. `hook-prompt-block`, where a `UserPromptSubmit` hook blocks the prompt); `tests/acp.e2e.ts` also asserts stdout purity without a key | `tests/acp.e2e.ts` — real ACP prompt, verifies a file the agent wrote; `tests/hooks.e2e.ts` — a real `PreToolUse` hook blocks bash, verifies the file is NOT written |
| `acp-agent` | `pnpm run test:snapshot` — boots the real ACP subprocess and replays a recorded session keyless (incl. the hook matrix: a scenario per hook point × outcome for BOTH the Claude and Codex bridges — block, deny, ask, context-fold, force-continue); `tests/acp.e2e.ts` also asserts stdout purity without a key | `tests/acp.e2e.ts` — real ACP prompt, verifies a file the agent wrote; `tests/hooks.e2e.ts` — a real `PreToolUse` hook blocks bash, verifies the file is NOT written |
See [the root AGENTS.md](../AGENTS.md) for repo-wide conventions and [docs/architecture.md](../docs/architecture.md) for the design.