Tianyi Cui
7c400e9c02
Collapse docs/adr/ and docs/rfc/ into a single docs/rfc/ with proposed/, implemented/, and rejected/ subfolders. Every file is renamed to yyyy-mm-dd-topic-title.md, where the date is when the topic was first proposed (from git history). ADRs and RFCs that covered exactly the same topic are merged (property-based testing, session persistence); the umbrella RFC 005 stays split across its three implemented decisions, and RFC 006's deferred part-3 (API extractor reports) splits into its own proposed RFC. All cross-references become machine-checkable relative links instead of bare "ADR NNNN" / "RFC NNN" prose. Add a verify-md-links doc-sync gate (scripts/verify-md-links.ts) that checks every relative Markdown cross-link resolves, wired into doc-sync alongside verify-md-wrap. This makes the reorganization self-verifying: the same change that rewrote ~forty inter-doc links adds the check that proves none dangle. Document the cross-link convention in a new docs/AGENTS.md and record the gate as an implemented RFC. doc-sync, typecheck, lint, and the full test suite (667) all pass.
29 lines
1.6 KiB
Markdown
29 lines
1.6 KiB
Markdown
# RFC: Mutation testing as the coverage counterweight
|
|
|
|
Status: proposed
|
|
|
|
<!-- XXX: legacy ADR/RFC body format, not yet normalized to a unified RFC template. -->
|
|
|
|
## Problem
|
|
|
|
The per-file 100% coverage gate ([the quality-gates decision](../implemented/2026-06-11-quality-gates.md)) proves every line *executes* under test — not that any assertion would notice if the line were wrong. Under agent-written tests, coverage pressure can produce execution-without-assertion. Mutation testing measures what coverage cannot: whether the suite *kills* deliberately injected bugs.
|
|
|
|
## Proposal
|
|
|
|
Stryker (`@stryker-mutator/vitest-runner`) over `packages/*/src`:
|
|
|
|
- **PR-scoped incremental runs** (changed files only) as a CI job — fast enough to gate merges once tuned.
|
|
- **Nightly full runs** with a tracked mutation score; start by recording, then set the threshold at the observed baseline and ratchet upward (same policy as coverage: thresholds only ever tighten).
|
|
- Surviving mutants are work items: an agent picks a survivor, writes the killing test, repeats — a well-shaped autonomous loop.
|
|
- Equivalent mutants (provably behavior-preserving) get annotated exclusions with reasons, mirroring the `/* v8 ignore */` policy.
|
|
|
|
## Plan
|
|
|
|
1. Add Stryker config scoped to one package (llm — smallest, most algorithmic) and measure runtime.
|
|
2. Expand to all packages; record baseline scores in the config.
|
|
3. Wire the nightly job; add the incremental PR job once runtime is acceptable.
|
|
|
|
## Risks
|
|
|
|
Runtime: mutation testing is expensive; per-file 100% coverage helps (every mutant is at least reached). If PR-scoped runs stay too slow, keep them nightly-only and rely on the score ratchet.
|