Residuals from the Codex re-review:
1. A system delta's insert was flattened to one token, so deltas differing
only in inserted-line count compared equal. Now one {{system}} token per
inserted line — position AND extent survive, content does not.
2. The live uniformity guard folded only request/header snapshots, so a
mid-run header CHANGE (request/header-delta) could diverge from the pin
invisibly. Non-pinning runs now assert zero header-delta events: a
scenario that legitimately changes its header mid-run exists to show
that change, so it must pin (fail-loud until it does).
Codex review findings on the pinned-header change:
1. scrubRequestHeaders flattened a request/header-delta's whole
system/tools payload to one token, so two meaningfully different deltas
compared equal. Now the structural facts survive — keepStart/keepEnd
line positions, added/removed/changed tool NAMES — and only the bulk
(inserted prompt lines, schema bodies) is tokenized.
2. The one-pin design rested on an unasserted premise (all sessions
compose the same header). Every non-pinning scenario now asserts, live,
that each request/header its run produces equals the pinned fixture's
header (both sides normalized against their own volatile values), so a
session-dependent header fails loud until it gets its own pin.
Verified the guard bites: perturbing the pinned fixture's prompt fails
a non-pinned scenario with the intended message.
3. RFC de-slopped per docs/AGENTS.md: no PR reference, no SHOULD
spec-speak; Decision/Verification/Consequences updated for 1 and 2.
Every session.jsonl fixture embedded the full composed system prompt and
complete tool-schema list in its request/header event (~8 KB on one line,
identical across the suite), so any prompt or tool-schema edit forced a
re-record or hand-edit of every fixture — see the dynamic-workflows PR for
the churn pattern this removes.
Now exactly one scenario (text-turn, flagged pinsHeader) commits and
compares that content verbatim; every other fixture stores and compares it
as {{system}}/{{tools}} tokens via the new pure scrubRequestHeaders
normalizer (applied to both compare sides and to record-mode writes, so a
re-record cannot reintroduce the content). request/header-delta payloads
are scrubbed the same way; config/reason stay verbatim — a model swap
SHOULD churn every fixture, a prompt edit should not. Replay is unaffected:
script derivation reads only assistant/chunk events.
Fixture meta-guards enforce the split: non-pinning fixtures must be fixed
points of the scrub, the pinning fixture must not be, and exactly one
scenario pins. Committed fixtures migrated through the same function.
Docs: pinned-header RFC (implemented/testing), base snapshot RFC + testing
policy + llm-replay module doc/README updated.
Tianyi Cui