Commit Graph

3194 Commits

Author SHA1 Message Date
imccyu 1529be6fd4 feat(storage): json backend — one human-readable file per unit
Atomic whole-file replacement (same-dir temp + fsync + rename + parent
fsync); the in-memory unit state is authoritative and the file is always
the current net state, pretty-printed. Missing files open as empty units
and materialize on first write; foreign or unparsable files reject with
malformed-medium, stored-version drift with version-mismatch.
2026-07-25 11:08:04 +08:00
imccyu e90b0d51df feat(storage): storage hub with named backend registry and data-form mounts
ctx.storage is a pure registration hub: multiple named backends stay
mounted side by side, data forms (domain first) mount via the
merge-extensible StorageForms map. src/backend.ts is the normative
KV-facet contract; tests/contract.ts is the shared conformance suite
every backend runs. Backends expose data-shape facets (kv now, an
append-log facet reserved for the future session-backend migration).
2026-07-25 11:08:04 +08:00
imccyu 2e27b8aeef chore(web): gate repairs for the config-tree boot round
Lint (bridge JSDoc params, service-class export shape, async invariant
listener form), regenerated doc catalogs/graphs with role classifications
for httpServer and clientModuleHost, catalog type-link exemptions for the
route/graph contracts, knip alignment (apps/cli composes via cordis.yml so
its yml-named deps are runtime edges knip cannot see; webserver's deleted
test dir), the zh side of the loading-model note brought along with its
pairing records, and coverage exclusions for the new web-transport halves
under the GUI test-lane TODO (real-composition harnesses land with that
lane).
2026-07-25 10:38:54 +08:00
imccyu 80cd8f54b5 refactor(web): collapse the shell boot into the AppWebEntry class
The four free functions in boot.tsx become one kernel class holding what
must exist before cordis: the parsed BootManifest, the ClientModuleSystem
instance, and the loading-page handles. Context/Loader setup runs in
parallel with the immediately-tier prefetch, but entry creation awaits the
prefetch: materialization is tree.import's synchronous require, so
cross-package require edges (i18n -> runtime/client) need every
immediately-tier factory registered first — unbarriered creation raced
10-25% of boots. The kernel adopts the modules entry (writes the
__DSH_MODULES__ slot pre-cordis, creates the entry first, skips its graph
row), and provide('modules') now lives in the adoption apply. apps/web
drops its host-package edges (composition is apps/cli's job).
2026-07-25 10:23:40 +08:00
imccyu 8d4aa73abe feat(web): grow real node halves in connection and hmr
connection binds the web transport: it injects httpServer + apiProxy and
registers toFetchHandler(ctx.apiProxy) under the /api prefix (the node:http
to fetch bridge moves in from the webserver, keeping the res-close disconnect
detection and drain/close backpressure waits). hmr owns dev reload: a
stat-poll watch per graph row driven by clientModuleHost.onGraphChanged,
rebuilt(id) on content change, and the /plugins/events SSE route (GET/HEAD
guarded); frame types are single-sourced in events.ts shared by both halves.
2026-07-25 10:23:40 +08:00
imccyu c12277b4bb feat(web): make dsh-client-modules dual-face with an incremental host scan
The node half is ClientModuleHostService (ctx.clientModuleHost): it composes
the __DSH_BOOT__ graph by scanning loader entries for dshClient packages,
serves /plugins/<id>/client.js, taps the index render, and exposes
rebuilt/onRebuilt/onGraphChanged. Scanning is incremental per package — no
full-rescan path exists: internal/plugin marks the fiber's entry name dirty,
a flush reconciles each name against live entries, package metadata
(including negative verdicts) caches forever, and re-hashing is reachable
only through rebuilt(id). The browser half moves wholesale to the standard
./client export (ClientModuleSystem, parseBootManifest with the dual-view
BootManifest, and the adoption plugin face that reads the
window.__DSH_MODULES__ slot and provides ctx.modules).
2026-07-25 10:23:40 +08:00
imccyu f499872cec refactor(web): rewrite webserver as a plain route-registration plugin
HttpServerService provides ctx.httpServer: register(route) -> disposer
(duplicate patterns throw), tapIndex transforms in registration order, and
the bound port; matching is exact > longest prefix > static dist fallback
(403/405/SPA semantics preserved). The server listens on activation, answers
per-request failures with 400 + a log line instead of exiting the process,
and knows no harness concepts — the boot graph, bundle routes, SSE channel,
and /api prefix all moved to their owning plugins.
2026-07-25 10:23:39 +08:00
imccyu 3cad7f6957 refactor(web): upgrade apiproxy to the api-gateway service plugin
createApiProxy moves from dsh-host-runtime into dsh-host-apiproxy (the
dependency direction already pointed this way); the package now
default-exports ApiProxyService (config {provider, model}, provides
ctx.apiProxy) while staying transport-agnostic — it registers no routes.
Runtime keeps bootHost/startHost for the headless path with its import
re-anchored, and drops the mountWebPlugins roster mounting helper.
2026-07-25 10:22:43 +08:00
Yichen Jiang 83cccd7ffc feat(llm): add scriptable mock fault server 2026-07-25 08:20:51 +08:00
kingwl d398eda432 fix(telemetry): join overlapping flush hints; contain adoption replay per event
Second review round, both pinned red-first:

- Overlapping turn-boundary flush hints now JOIN the outstanding flush
  promise (Promise.all) instead of displacing it: the SDK's
  concurrent-flush guard resolves an overlapping forceFlush()
  immediately, so retaining only the latest promise let shutdown()
  proceed while the first export was still in flight — the same silent
  drop the single-flush fix closed.
- Adoption replay contains failures per event, matching the firehose:
  one rejected record is withheld fail-closed while the rest of the
  historical log still hands off. Wrapping the whole loop let a single
  failure silently skip the remainder on an already-adopted session.
2026-07-25 03:48:32 +08:00
kingwl e6a8ff2621 fix(telemetry): review fixes — flush/shutdown ordering, session retirement, whole-exporter passthrough
Three review findings, each pinned by a red test first:

- The OTel backend retains the latest turn-boundary flush promise and
  shutdown() awaits it before provider.shutdown(): the SDK's
  concurrent-flush guard makes the shutdown-internal flush return early
  while one is in flight, silently dropping everything enqueued after
  the flush snapshot (including the coordinator's dispose-time shutdown
  markers).
- The coordinator retires sessions from the adopted set on
  session/disposed (mirroring the persistence coordinator): a long-lived
  backend no longer retains closed sessions and their event logs, and
  final unload no longer stamps shutdown markers for dead sessions.
- The exporter config passes through whole to OTLPLogExporter and its
  type/JSDoc now advertise the full OTLPExporterNodeConfigBase shape:
  rebuilding url/headers only silently dropped documented SDK options
  (timeoutMillis, compression, keepAlive, ...).
2026-07-25 03:20:29 +08:00
Tianyi Cui 2beaa18f42 Merge remote-tracking branch 'origin/master' into worktree/acp-automation-protocol
# Conflicts:
#	packages/support/acp-snapshot/README.md
2026-07-25 02:07:28 +08:00
Tianyi Cui 9c4bc3e516 Merge remote-tracking branch 'origin/master' into web-e2e-lane 2026-07-25 02:00:05 +08:00
Tianyi Cui 05ee162723 Merge remote-tracking branch 'origin/master' into fix-webplugins-watch-flake
# Conflicts:
#	packages/host/webserver/src/web-plugins.ts
#	packages/host/webserver/tests/web-plugins.spec.ts
2026-07-25 01:58:59 +08:00
kingwl b5523b0b48 refactor(telemetry): drop dead live-collector smoke and the compact/end severity probe
tests/otel.e2e.ts self-skipped on $DSH_OTLP_E2E_ENDPOINT, which nothing in
the repo sets — it never ran; the mock-collector wire spec and the keyless
Loader-composition e2e already cover the pipeline both ways.

The severityOf compact/end probe parsed another package's merged event
shape by string comparison — an untyped cross-package contract that breaks
silently — and its only consumer was the test's own stand-in declaration.
Unknown event types now uniformly fall through as info; outcome semantics
stay with the owning package.
2026-07-25 01:55:35 +08:00
kingwl 70febffe1a refactor(telemetry): ship the redact waterfall without built-in rules
The seam keeps the telemetry/redact scrubbing interface but ships no rules
of its own: the innermost next() passes records through unchanged, and
deployments mount their rules as waterfall listeners. As an SDK we cannot
know which patterns are secrets in a given deployment; a shipped list
invites false confidence while catching only known shapes, and false
positives would corrupt exported bodies. Mechanism stays with the seam,
policy moves to the deployment; both READMEs and the Agent Note state the
raw-export default plainly.

The loader-composition e2e now mounts a deployment-style rule fixture and
pins the same wire behavior: secret absent, placeholder present, canonical
log untouched.
2026-07-25 01:55:35 +08:00
kingwl cf2e184112 feat(telemetry): session-telemetry seam with mandatory redaction + OTel backend
Revive the reviewed session-telemetry packages from the closed
session-telemetry-otlp-rfc branch (PR #222/#231) on current master, renamed
to @deepseek-ai/dsh-session-telemetry{,-otel} (the SDK component-telemetry
package holds the dsh-telemetry name).

Delta over the branch version: every record now passes a telemetry/redact
waterfall between projection and emit() — the innermost next() applies a
non-configurable conservative credential-shape rule set, listeners stack
stricter rules, a throwing rule withholds the record fail-closed, and the
canonical log is never rewritten. This answers the export-side concern that
closed PR #222; the boundary axiom (our aspect ends at emit(); delivery is
the reporting SDK's) is unchanged, and the runtime-telemetry RFC's outbox /
readCommitted lane is recorded as deferred in the Agent Note.

Covered by seam/redact/OTel-wire unit tiers (100% per-file) and a keyless
Loader-composition e2e that boots the examples fixture against a mock OTLP
collector and pins redaction on the wire plus the untouched canonical log.
2026-07-25 01:55:09 +08:00
Tianyi Cui 2096050824 fix(webserver): registry-owned stat poll replaces fs.watchFile baseline race
The dev bundle watch missed rebuilds that landed while the registry was
constructing: fs.watchFile captures its comparison baseline with an
ASYNCHRONOUS first stat, so a write racing that window is absorbed into
the baseline and never reported. Standalone repro missed 24/400 same-tick
rewrites; the CI flake in web-plugins.spec.ts ('watch mode: a bundle
content change re-hashes the row...') was exactly this — the spec writes
immediately after createHostWebPluginRegistry returns.

The watch now polls from one registry-owned setInterval against a stat
baseline the scan itself captures synchronously, stat-before-read: a
write landing between stat and read leaves the hash newer than the
baseline (next tick re-hashes to the same rev, no spurious notify); a
write landing after the read leaves the baseline older (next tick
detects and notifies). No blind window. The poll iterates the live
table, so rescans retarget the watch for free and dispose clears one
timer. Stress: real-registry same-tick rewrite 0/600 missed (was 1/300);
spec watch tests 0/50.

New regression test pins the same-tick-as-construction write. Its
rewrite deliberately differs in size from the seed: a same-millisecond
same-size rewrite is invisible to any mtime+size poll (coarse fs
timestamps) — a stat-polling limit, not this regression. Loading-model
Agent Note updated in both languages (pair re-recorded).
2026-07-25 01:22:46 +08:00
Tianyi Cui 1d066e0f74 Fix remaining PTY and bundle watch races
Keep inherited child prompt markers bounded by the normal silence fallback. Stage web-plugin rescans atomically and retain missing watch state until a successful rebuild.
2026-07-25 00:33:33 +08:00
Tianyi Cui f9a638b8a6 Stabilize master CI across platforms 2026-07-25 00:10:37 +08:00
Tianyi Cui 440384fdb4 Merge remote-tracking branch 'origin/master' into worktree/acp-automation-protocol
# Conflicts:
#	.agents/notes/implemented/architecture/2026-06-14-session-persistence.md
#	.agents/notes/implemented/architecture/2026-06-20-package-hierarchy.md
#	.agents/notes/implemented/architecture/2026-07-02-tool-render-intent-union.md
#	.agents/notes/implemented/feature/2026-06-14-acp-agent-client-protocol.md
#	.agents/notes/implemented/feature/2026-06-14-acp-multi-session.md
#	.agents/notes/implemented/feature/2026-06-18-acp-terminal-and-tool-rendering.md
#	.agents/notes/implemented/feature/2026-07-19-model-facing-goal-tools.i18n.yaml
#	.agents/notes/implemented/feature/2026-07-19-model-facing-goal-tools.md
#	.agents/notes/implemented/feature/2026-07-19-model-facing-goal-tools.zh.md
#	.agents/notes/implemented/simplification/2026-07-04-trim-acp-bridge-unreachable-surface.md
#	docs/architecture.i18n.yaml
#	docs/cookbook/extension-cookbook.i18n.yaml
#	docs/cookbook/extension-cookbook.md
#	docs/cookbook/extension-cookbook.zh.md
#	docs/core-data-structures/approval.md
#	docs/core-data-structures/user-interaction.md
#	docs/event-producer-consumer.md
#	docs/persistence-catalog.md
#	docs/testing.md
#	docs/tool-catalog.md
#	examples/acp-agent/tests/fixtures/live-mode-switching-2026-07-07.session.jsonl
#	examples/acp-agent/tests/snapshots/cordis-inspect-jsdoc/stdout.expected.jsonl
#	examples/acp-agent/tests/snapshots/permission-switching/session.jsonl
#	packages/goal/tool-goal/README.md
#	packages/ui/acp/README.md
#	packages/ui/acp/acp-feature-support.md
#	packages/ui/acp/src/index.ts
#	packages/ui/acp/tests/bridge.spec.ts
#	packages/ui/acp/tests/dispose.spec.ts
#	packages/ui/acp/tests/edges.spec.ts
#	packages/ui/acp/tests/turns.spec.ts
2026-07-24 23:43:10 +08:00
Tianyi Cui 8a7472d11d Merge remote-tracking branch 'origin/master' into web-e2e-lane
# Conflicts:
#	docs/testing.md
2026-07-24 23:11:00 +08:00
Tianyi Cui 90057b25f2 docs(acp): state both arms of the spec's version-negotiation rule 2026-07-24 22:19:10 +08:00
Tianyi Cui 6d3c25f494 refactor(web-e2e): rename harness -> scaffold; add interaction coverage
The shared test module was named harness.ts inside a repo whose product
IS a harness — hopelessly ambiguous. Renamed to scaffold.ts with
launchWebScaffold/WebScaffold; tsconfig plane-split entries, the seam
JSDoc/README mentions, and both Agent Note languages updated.

Both scenarios gain a Playwright interaction step over the settled
transcript (after the golden capture, so committed aria surfaces stay
untouched): replay-round-trip clicks the reasoning fold open/closed over
wire-delivered state; seeded-history expands a read tool row rebuilt from
the cold log and asserts the recorded result text appears (read rows are
expand-in-place — rowExpands routes the click to the inline fold, not the
details column). test:web 30 passed | 1 skipped.
2026-07-24 22:18:19 +08:00
Tianyi Cui 5569f3f8ac docs: repair rewritten rationale and stale claims from the ACP reduction
The automation-only rewrite edited many implemented Agent Notes; several
edits replaced still-live or historical rationale instead of reframing:

- llm-model-catalog: restore the prompt/request consistency section and
  selection-ownership alternatives — installAgentLlmTarget and the TUI
  /model selector still ship that design; only the ACP wire is gone.
- plan-specific-collaboration-state, acp-multi-session, todo-write,
  ask-user-question: link the superseding automation-only note instead
  of silently rewriting the original decision or motivation; drop a
  paragraph duplicating the Web-provider facts stated two paragraphs up.
- sandbox: stop claiming unit coverage for turn-enclosed config writes
  (that mechanism left with the bridge) and retitle the commit-boundary
  paragraph accordingly.
- Fix the missing blank line before '## Consequences' in the
  plugin-command-registration pair, the JSON-RPC/Web render-intent
  consumer misattribution (the second consumer is the host/client
  runtime), stale bash_output/bash_kill names, and 'optional goals' in
  architecture.md.
- examples/acp-agent/README.md: point at the package contract instead
  of restating it; packages/ui/permission and plan-mode READMEs record
  the consumer-less preset service and the exit_plan_mode coverage gap
  under Known Limitations.
- 2026-06-19-acp-snapshot-tests: the new note defers the corpus
  migration rather than committing to it; say so.

Re-record the touched bilingual pairs.
2026-07-24 22:12:23 +08:00
Tianyi Cui 74d14f68bb fix(acp): accept baseline resource links and release a failed prompt slot
ACP v1 requires every agent to accept text AND resource_link prompt
content; the automation rewrite dropped the resource_link half of that
baseline. Restore the old bracketed-reference flattening in the codec,
reject only beyond-baseline blocks, and update the package contract and
Agent Note.

Also release the per-session prompt slot when agent.send() throws
synchronously (an agent disposed outside the bridge would otherwise
wedge the session into permanent 'already in flight' rejections),
drop the tautological version-negotiation branch, prove the scenario
env layer reaches the snapshot subprocess, pin bridge-side fail-closed
permission errors, and correct two overpromising test names.
2026-07-24 22:11:49 +08:00
Turtle 1ffdacb2c4 fix(jsonl): handle filesystem path aliases 2026-07-24 21:50:11 +08:00
Hypatia May 224e00b2bd fix: quiesce cancelled session reconciliation 2026-07-24 21:27:07 +08:00
Turtle c14f488b00 fix(persistence): use normalized project directory names 2026-07-24 21:15:53 +08:00
Tianyi Cui 46dfd503e0 Merge origin/master into codex/sqlite-metadata-contracts 2026-07-24 21:07:23 +08:00
Hypatia May cbc6d81fc3 Merge remote-tracking branch 'origin/master' into session-query-tool
# Conflicts:
#	docs/architecture.i18n.yaml
2026-07-24 20:59:18 +08:00
Hypatia May ba6fc1cfa3 fix: harden session query authorization 2026-07-24 20:50:53 +08:00
Tianyi Cui 9e691d54c0 fix(goal): normalize strict-schema fillers 2026-07-24 20:45:23 +08:00
Tianyi Cui 145ba169f6 Merge commit 'refs/codex-unblock/20260724/pr591-base-current' into worktree/unblock-pr-591-ci-20260724 2026-07-24 20:32:02 +08:00
Tianyi Cui d08050ab69 chore(web-e2e): gate fixes — catalog, budgets, knip
Regenerate config-catalog for the llm-replay paceMs row; condense the
testing.md web-lane entry to pointer form and raise its ceiling 1020->1060
(the two-sentence tier entry for a genuinely new surface does not fit the
old ceiling after relocation-first trims); internalize two harness helpers
knip flagged (rawSessionLog/normalizeAria are module-internal).
2026-07-24 20:30:33 +08:00
Hypatia May 04c8a17de5 fix: cancel exact session observations 2026-07-24 20:08:38 +08:00
Tianyi Cui 2cd37e2765 docs(testing): web e2e lane docs + Agent Note to implemented
testing.md gains the web browser snapshot tier entry (divergent
DSH_SNAPSHOT=... test:web commands) and names apps/web/tests/snapshots/
as the web surface's snapshot home. The GUI testing note's tier map and
lane map gain the e2e scenarios (both languages, pair re-recorded) and
drop the stale verify-session-real references (those scripts left with
the missions/ tree). packages/client/AGENTS.md check ladder covers the
wire-carriage trigger and refresh/record commands. acp-snapshot README
stops claiming the whole package is ACP-specific — its normalizers are
transport-neutral with three consumers now. vitest.web.config.ts header
carries TODO(ci-browser) with the staged-reversal pointer.

The design-study Agent Note moves proposed/ -> implemented/ rewritten in
present tense: all review decisions recorded (llm:false seam over the
placeholder-key hack, providers-mode replay, whenIdle barrier stack,
single aria golden + anchors, TUI-style inline modes over a suite
factory, scrub-only header stance, CI deferral) with re-entry triggers
under Deferred.
2026-07-24 19:57:27 +08:00
Turtle 4e295221f3 refactor(persistence): group sessions in project directories 2026-07-24 19:52:47 +08:00
Hypatia May 66585635c8 fix: serialize paginated session searches 2026-07-24 19:47:23 +08:00
Turtle 800bafda3b refactor(cli): parse dsh argv through one Commander adapter
Replace the dsh CLI's three hand-rolled parsing idioms (raw argv[0]/includes
dispatch in bin.ts, per-mode node:util parseArgs in headless.ts/web.ts, and the
bespoke parseResumeArg scanner in dsh-app-boot) with a single Commander adapter
in apps/cli/src/args.ts. parseDshArgs resolves argv into a discriminated
DshInvocation union; bin.ts switches on the mode and dynamic-imports the chosen
module, which now consumes already-parsed values.

- web is a real subcommand; --host uses choices and --port an argParser range
  check, moving validation into the parser.
- --resume rejects empty and repeated forms; --prompt rejects empty; a config
  positional after --prompt and a root flag placed before web fail loud.
- adds --help/--version; removes parseResumeArg from dsh-app-boot.
- new apps/cli/tests/args.spec.ts (apps/*/tests added to vitest include,
  apps/cli/tests to tsconfig.host.json); the tui-agent keyless PTY smoke covers
  bin.ts dispatch end to end unchanged.
2026-07-24 19:43:59 +08:00
Tianyi Cui 51f697db50 test(tui): wait for scoped file completion
The fake terminal retains earlier autocomplete frames, so the directory follow-up assertion could pass before the scoped lookup completed. Wait for the quoted file mention to be applied before submitting.
2026-07-24 19:38:31 +08:00
Turtle 207aab9d8d fix(llm): classify empty model completions as retryable EMPTY_RESPONSE
A well-formed provider stream that ends with finish_reason stop and zero
content blocks previously became a successful empty assistant message: the
turn completed silently, and drivers like goal-session counted the no-op
round. Both adapters now map that degenerate completion to a finish
{kind:'error'} with the new canonical EMPTY_RESPONSE code from dsh-llm, and
dsh-llm-retry adds the code to its default retryable set, so the existing
closed-step recovery path retries it and fails loud once the budget is
exhausted.

Covered by adapter unit tests, an llm-retry default-policy test, and a new
authored keyless ACP snapshot (empty-response-retry) with a deterministic
1 ms zero-jitter retry overlay.
2026-07-24 19:23:12 +08:00
Hypatia May 795af3174e fix: cancel session authorization reads 2026-07-24 19:21:32 +08:00
Tianyi Cui a229b095d6 Merge branch 'master' into fix/goal-completion-strict-fields 2026-07-24 19:19:57 +08:00
Tianyi Cui 9ef0193dd5 feat(host-runtime,llm-replay): keyless llm seam + replay pacing/consumption handle
BootHostOptions.llm: 'deepseek' | false — false mounts no adapter, boots
keyless, and leaves the llm capability seam open for the embedder to fill
on RunningHost.ctx (now the third sanctioned ctx use, JSDoc + README
amended); an unfilled seam fails loud with NO_ADAPTER at the first stream.

dsh-llm-replay grows two additive surfaces for the web browser e2e lane:
paceMs (validated per-chunk delay so a real transport shows incremental
delivery; abort during a pace wait cancels promptly) and a ReplayHandle
return — dispose() plus assertConsumed(), the teardown check that every
recorded script bound and drained, converting silent fixture underruns
into diagnostics. Existing callers updated; config catalog regenerated.
2026-07-24 19:08:10 +08:00
Hypatia May 415948dd7b fix: redact lineage errors and parse precise times 2026-07-24 18:47:41 +08:00
Hypatia May 0d2e74ed48 docs: describe host session-query surface 2026-07-24 18:22:00 +08:00
Hypatia May fec4ce52cc fix: batch cancellable title reads 2026-07-24 18:13:11 +08:00
Hypatia May f5fc7ac04a test: scope event snapshot normalization 2026-07-24 17:25:32 +08:00
Turtle 081bb1a8fd Merge remote-tracking branch 'origin/master' into feat/send-unify
# Conflicts:
#	packages/client/runtime/src/client/sessions/fold-adapter.ts
2026-07-24 17:17:35 +08:00