Commit Graph

18 Commits

Author SHA1 Message Date
kingwl d85d2a4be3 Merge remote-tracking branch 'origin/master' into jsonl-packed-chunk-rows
Conflicts: the four generated catalog docs (regenerated over merged sources),
session index.ts exports (keep chunk-rows exports + master's SessionSurface
re-export), stdio/acp demo config schema and persistence wiring (thread
packChunks through master's DEFAULT_PERSISTENCE_ROOT/UI shape), stdio README
config table, and the jsonl spec import line. The packed-chunk fixture also
gains the provenance field master made required on assistant/message.
2026-07-20 15:24:58 +08:00
Tianyi Cui 0893aaa7cb test: rename golden snapshot files to expected 2026-07-19 17:45:49 +08:00
Tianyi Cui ce393ccb93 Merge master into codex/simp-session-log-representation 2026-07-17 21:51:36 +08:00
Dudu-0223 a3f62478e1 Merge origin/master into codex/truncated-design 2026-07-17 18:21:54 +08:00
kingwl 7ef21239ca feat(session): opt-in packed chunk rows in the JSONL log
Providers stream token-sized deltas, so a session log stores hundreds of
near-identical assistant/chunk lines whose JSON envelopes dwarf their
payloads (~56x measured on a real DeepSeek session, 73% of file bytes).

Add a lossless storage codec to dsh-session: packChunkRuns() folds each
run of >=3 consecutive same-block delta chunks into one storage row --
text-chunks / reasoning-chunks / tool-call-chunks, bare slash-less tags
like the header line's 'session' so rows cannot be confused with session
events -- and decodeStorageRecord() expands rows back to the exact
original events (seq0/time0 + dt gap array reconstruct every member's
seq/time; tool-call rows carry the run-constant id/name). The encoder
whitelists exact shapes and stores anything unrecognized verbatim; the
decoder validates row-tagged values and fails loud on malformation.

The JSONL backend gains a packChunks config (default false). Writing
packs only when enabled -- default-off output stays byte-identical to
the previous layout, so snapshot goldens are untouched. Reading is
layout-blind: scanLog always decodes rows and now checks seq contiguity
with a cursor instead of the line index, so packed, unpacked, and mixed
files all load identically. Fixture readers (llm-replay parseSessionLog,
acp-snapshot normalizeSessionLog) share the codec; the normalizer zeroes
a row's time0/dt exactly like an event's time. The two demo bundles
plumb packChunks from cordis.yml to the backend.

Measured on a real coding session: 105 KB -> 42 KB (-60%), 475 lines ->
74, with reasoning/tool-call heavy sessions saving the most. Covered by
example + fast-check round-trip codec tests, backend packed/mixed/torn-
tail specs, and an end-to-end demo run loading a packed log through a
default-config backend.
2026-07-15 21:26:36 +08:00
Tianyi Cui 281eed3e57 Merge remote-tracking branch 'origin/master' into codex/simp-session-log-representation
# Conflicts:
#	docs/cordis-catalog/services.md
#	docs/rfc/INDEX.md
#	docs/rfc/implemented/testing/2026-07-06-pin-request-header-content-in-one-scenario.md
#	docs/rfc/proposed/simplification/2026-07-12-simplify-session-log-representation.md
#	packages/support/acp-snapshot/README.md
#	packages/support/acp-snapshot/src/normalize.ts
#	packages/support/acp-snapshot/src/suite.ts
#	packages/support/acp-snapshot/tests/fixtures/suite/pin-turn/session.jsonl
#	packages/support/acp-snapshot/tests/suite.spec.ts
2026-07-15 15:53:56 +08:00
Tianyi Cui 8ffcef7db9 test(snapshot): split pinned schemas into sidecars 2026-07-14 23:11:21 +08:00
Tianyi Cui 78276c52a7 Merge branch 'codex/simp-session-dead-surface' into codex/simp-session-log-representation
# Conflicts:
#	docs/cordis-catalog/events.md
#	docs/cordis-catalog/services.md
#	docs/core-data-structures/core.md
#	docs/persistence-catalog.md
#	docs/rfc/implemented/architecture/2026-07-05-reconstructable-requests.md
#	docs/rfc/implemented/feature/2026-06-18-compaction-capability-seam.md
#	docs/rfc/implemented/feature/2026-07-06-explicit-tool-order.md
#	docs/rfc/implemented/feature/2026-07-06-sandbox.md
#	docs/rfc/implemented/feature/2026-07-07-session-prefix.md
#	packages/compact/compact-basic/src/index.ts
#	packages/compact/compact-basic/tests/compact-basic.spec.ts
#	packages/core/agent-loop/src/loop.ts
#	packages/core/agent-loop/src/request-log.ts
#	packages/core/agent-loop/tests/request-reconstruction.spec.ts
#	packages/core/agent/src/types.ts
#	packages/core/session/README.md
#	packages/core/session/src/request-header.ts
#	packages/core/session/src/surface.ts
#	packages/core/session/src/tool-pairing.ts
#	packages/core/session/src/types.ts
#	packages/core/session/tests/surface.spec.ts
#	packages/core/session/tests/tool-pairing.spec.ts
#	packages/llm/llm/src/call-config.ts
#	packages/support/acp-snapshot/src/suite.ts
#	packages/support/invariants/README.md
#	packages/ui/user-approval/README.md
2026-07-14 18:34:34 +08:00
Tianyi Cui 1123e946c0 refactor: simplify session log representation 2026-07-13 23:56:10 +08:00
Tianyi Cui 148046b9c8 docs: rebalance prose cleanup and add trimming skill 2026-07-13 23:27:00 +08:00
Dudu-0223 5ac03dde3f fix(review): generalize spill storage locators 2026-07-13 11:07:27 +08:00
Dudu-0223 43535aab42 Merge remote-tracking branch 'origin/master' into codex/truncated-design
# Conflicts:
#	docs/capability-seams.md
#	docs/config-catalog.md
#	docs/cordis-catalog/services.md
#	docs/core-data-structures/core.md
#	docs/event-producer-consumer.md
#	docs/module-graph.md
#	docs/rfc/INDEX.md
#	docs/tool-catalog.md
#	examples/acp-agent/README.md
#	packages/README.md
#	packages/bash/bash/README.md
#	packages/core/tools/tests/gen-tool-catalog.spec.ts
#	packages/support/acp-snapshot/src/harness.ts
#	pnpm-lock.yaml
#	scripts/gen-doc-graphs.ts
#	scripts/gen-tool-catalog.ts
#	scripts/type-equiv.manifest.json
2026-07-13 09:49:46 +08:00
Tianyi Cui 75838e10b5 docs: trim generated prose 2026-07-12 03:36:43 +08:00
Tianyi Cui 9339622d3b test(acp): snapshot system prompts as Markdown 2026-07-11 22:24:20 +08:00
Dudu-0223 8b86c3febc fix: stabilize spill snapshot path budget 2026-07-10 13:15:15 +08:00
Dudu-0223 31be42e92d test: add ACP spill snapshot coverage 2026-07-10 10:07:32 +08:00
Yichen Jiang 77333c0a19 fix(agent): correct session-prefix composition-order docs; scrub messagePrefix in snapshots
Two ds-review-bot findings:

The seam JSDoc claimed extending 'await next()' composes in
registration order — false for the append form: the waterfall unwinds
innermost-first, so appending places later-registered contributions
first. The canonical contribution is now documented as the PREPEND
'[mine, ...await next()]' (registration order on the wire), with the
append form's reverse-order behavior stated explicitly; the ordering
test now uses the canonical pattern in both listeners.

scrubRequestHeaders tokenized only system/tools, so a fixture recording
a composed session prefix would carry its raw text (workspace-specific
churn/leak). The scrubber now maps each header/delta messagePrefix
entry to a {{messagePrefix}} token — count stays a structural fact,
absence stays absent, the empty-array transition stays visible — with
normalize.spec coverage for the header, delta, absence, and odd-shape
paths.
2026-07-08 16:16:08 +08:00
kingwl 556f847064 feat(acp-snapshot): extract the ACP snapshot suite into a support package
The snapshot tier's machinery leaves examples/acp-agent/tests for
packages/support/acp-snapshot (@deepseek-ai/dsh-acp-snapshot), where the
coverage gate measures it and a second example can consume it instead of
forking it: harness.ts (runScenario, parameterized by an AgentUnderTest
{binScript, configPath, tsconfigPath} instead of module constants),
normalize.ts (moved verbatim), and suite.ts (defineAcpSnapshotSuite — the
per-scenario golden/log compares, record write-back, per-suite header pin
with its uniformity guard, and the fixture guard block, lifted from
acp.snapshot.ts). The example file collapses to its scenario table plus
one factory call; env reading (DSH_SNAPSHOT) stays at that edge.

The exactly-one-pin meta-test generalizes from the hardcoded text-turn
name to "exactly one per suite" — which scenario pins is the scenario
table's reviewable choice (per-suite pinning per the proposal RFC).

Extraction parity: pnpm run test:snapshot is 36 passed + fs-policy-reject
failing BEFORE AND AFTER (BSD-sed environment failure, reproduced at the
base commit in a clean worktree — the recorded golden's sed -i syntax is
GNU-only), with zero byte changes under examples/acp-agent/tests/snapshots/.
Coverage for the new src files lands in the next commit.
2026-07-08 02:38:25 +08:00