Pine/deepseek-harness
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases 1 Wiki Activity

docs(i18n): record the v1-vs-v2 briefing A/B in the Agent Note

Browse Source 
Head-to-head replay of the same ten historical examples, both arms in
one time window with identical prompts and pairwise blind judging:
prose quality and cost at parity (stylistic margins only); the shipped
briefing wins two objective outcomes — code-fence-only examples land
byte-identical to the human-reviewed updates with zero model tokens,
and the flagged first-occurrence move reproduces the human-reviewed
gloss relocation the section-only form leaves as a contract violation.
Chinese counterpart brought along via the briefed path and the pair
re-recorded.
This commit is contained in:
Tianyi Cui
2026-07-27 03:35:50 +08:00
parent 53283003e9
commit 3fe60837c1
3 changed files with 6 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.agents/notes/implemented/process/2026-07-26-briefed-minimal-translation-updates.md
+2
View File
@@ -26,6 +26,8 @@ The decision followed a controlled replay of ten real pair updates from this rep
- On the briefing, a small model performed at parity with the large one, so the update path no longer assumes a frontier translator.
- Batching three pairs into one subagent showed no reliable saving over three briefed runs and couples unrelated failures; it was rejected.
A second head-to-head replay on the same ten examples compared this note's shipped briefing against its earlier section-only form (no unit tier, no computed mechanical path, counterpart-only context, no first-occurrence tracking). Prose quality and cost were at parity — pairwise blind verdicts split with only stylistic margins — and the shipped form won on two objective outcomes: the two code-fence-only examples were completed byte-identical to the human-reviewed historical updates in under a second with no model tokens, and on the example whose edit moved a term's document-wide first occurrence, the shipped briefing's flagged move reproduced the human-reviewed gloss relocation while the section-only form left a 首次出现 violation for review to catch.
## Alternatives considered
- **Keep the workflow, just scope the gate** — the gate scan was the smaller cost; the corpus loads and archaeology dominated. Scoping alone would have left the ~3x overhead in place.