Merge remote-tracking branch 'origin/master' into codex/agent-session-jsonl-location
# Conflicts: # docs/config-catalog.md # docs/cordis-catalog/services.md # docs/core-data-structures/bash.md # docs/module-graph.md # docs/rfc/INDEX.md # docs/rfc/implemented/architecture/2026-06-30-bash-stdin-env-trusted-plugin-surface.md # docs/rfc/implemented/feature/2026-06-30-hook-bridges.md # examples/acp-agent/tests/snapshots/advanced-toolchain/session.1.jsonl # examples/acp-agent/tests/snapshots/advanced-toolchain/session.2.jsonl # examples/acp-agent/tests/snapshots/advanced-toolchain/session.jsonl # examples/acp-agent/tests/snapshots/advanced-toolchain/system-prompt.golden.md # examples/acp-agent/tests/snapshots/both-mode-turn/session.jsonl # examples/acp-agent/tests/snapshots/both-mode-turn/system-prompt.golden.md # examples/acp-agent/tests/snapshots/code-mode-turn/system-prompt.golden.md # examples/acp-agent/tests/snapshots/skill-load/session.jsonl # examples/acp-agent/tests/snapshots/text-turn/session.jsonl # examples/sandbox-acp-agent/tests/snapshots/mode-switching/session.jsonl # packages/bash/bash-local/README.md # packages/bash/bash-local/src/run.ts # packages/bash/bash-local/tests/run.spec.ts # packages/bash/bash/README.md # packages/bash/tool-bash/README.md # packages/bash/tool-bash/src/index.ts # packages/bash/tool-bash/tests/tools.spec.ts # packages/cordis/tool-cordis/src/api-catalog.ts # packages/core/agent-core/src/index.ts # packages/session-persistence/session-persistence-jsonl/src/index.ts # packages/session-persistence/session-persistence-sqlite/src/index.ts # packages/session-persistence/session-persistence/README.md # packages/session-persistence/session-persistence/src/index.ts # packages/ui/acp-agent/README.md # packages/ui/stdio-agent/README.md
This commit is contained in:
@@ -23,9 +23,9 @@ The persisted unit IS the existing `SessionEvent` (event-sourced model — the l
|
||||
|
||||
## The write coordinator
|
||||
|
||||
The two first-party backends were byte-identical (or same-algorithm) for ALL of their write-path orchestration — the in-memory bookkeeping (per-id state, write-behind buffers, per-id serialization chains, per-session init promises), the `session/event` → buffer → `session/flush` drain, lazy materialization, crash-tail repair on load, the four `session/created` adoption cases (new / HMR-adopt / collision / ownerless-claim), and dispose-time quiescence. Only the STORAGE primitives differed (write bytes vs. INSERT rows).
|
||||
`PersistenceCoordinator` owns per-id state, write-behind buffers and serialization, the `session/event` → `session/flush` drain, lazy materialization, crash-tail repair, session adoption, and quiescent disposal. A first-party backend composes one, implements the small `PersistenceBackend` storage hook interface, and delegates its four public service methods. JSONL and SQLite therefore share lifecycle correctness while retaining different storage primitives; see the [coordinator RFC](../../../docs/rfc/implemented/architecture/2026-06-18-shared-persistence-write-coordinator.md).
|
||||
|
||||
`PersistenceCoordinator` owns that orchestration once. A first-party backend composes one (`new PersistenceCoordinator(ctx, this)`), implements the small `PersistenceBackend` hook interface, and delegates its four stateful service methods to the coordinator; the pure `locate` query stays backend-owned. This keeps the duplicated, correctness-heavy orchestration in a single place (it used to receive the same fixes twice).
|
||||
The side-effect-free `locate` query remains backend-owned because it describes storage topology rather than write orchestration.
|
||||
|
||||
The `PersistenceBackend<TornMarker>` hooks (the only seam between the coordinator and storage):
|
||||
|
||||
@@ -50,3 +50,17 @@ Three backends run these suites: an in-memory reference (in `tests/`), `dsh-sess
|
||||
## Metadata and location types
|
||||
|
||||
Re-exported from `dsh-session`: `SessionHeader` (immutable session metadata: `version`, `id`, `createdAt`, `cwd?`, `parentSession?`, `seedLength?`). `SessionLocation` is `{ readonly kind: string; readonly path: string }`; its path is an absolute backend target, not proof that the artifact exists or contains an unflushed turn.
|
||||
|
||||
## Model Experience
|
||||
|
||||
### Resumed conversation history
|
||||
|
||||
**What the model sees**: This seam adds no prompt or schema. Resume restores stored surface events as message history; stored request headers reconstruct earlier calls, while the new loop composes the current system prompt, tools, and session prefix for its next request. Crash repair inserts exactly `Tool call interrupted by a crash; no result was recorded.` as the error result for each unanswered tool call.
|
||||
|
||||
**Token effect**: Zero tokens during ordinary persistence. Resume restores retained history cost and pays the current request envelope normally; each repaired call adds the quoted retained error text.
|
||||
|
||||
## Known Limitations and Deferred Work
|
||||
|
||||
- **No deletion or retention surface** — the seam is `create`/`append`/`load`/`list` only; pruning stored sessions is out-of-band backend maintenance.
|
||||
- **`list()` is unpaginated and unfiltered** — it returns every stored session's header; fine for local stores, unindexed at scale.
|
||||
- **Repair-time synthetic closers are the only crash story** — a backend must synthesize `tool/result`/`step/end`/`turn/end` closers on load; there is no partial-turn resume that continues an interrupted turn instead of closing it.
|
||||
|
||||
@@ -1,26 +1,7 @@
|
||||
/**
|
||||
* The backend-agnostic write-path orchestration shared by every first-party
|
||||
* {@link SessionPersistence} backend.
|
||||
*
|
||||
* Every durable backend needs the same orchestration: the in-memory bookkeeping
|
||||
* (the per-id state, the write-behind buffers, the per-id serialization chains,
|
||||
* the per-session init promises), the `session/event` → buffer → `session/flush`
|
||||
* drain, lazy materialization, crash-tail repair on load, the four
|
||||
* `session/created` adoption cases (new / HMR-adopt / collision /
|
||||
* ownerless-claim), and dispose-time quiescence. Only the STORAGE primitives are
|
||||
* backend-specific (file bytes for `dsh-session-persistence-jsonl`, `node:sqlite`
|
||||
* rows for `dsh-session-persistence-sqlite`). {@link PersistenceCoordinator} owns
|
||||
* the orchestration; a backend supplies the storage primitives as a small
|
||||
* {@link PersistenceBackend} hook object.
|
||||
*
|
||||
* The abstract {@link SessionPersistence} service's public API is independent of
|
||||
* this: a backend IS a `SessionPersistence` (its four public methods delegate to
|
||||
* a coordinator it composes), so a third-party backend MAY implement the service
|
||||
* directly without using the coordinator at all.
|
||||
*
|
||||
* See the write-coordinator RFC (docs/rfc/implemented/architecture/2026-06-18-shared-persistence-write-coordinator.md)
|
||||
* for the design rationale (composition over inheritance, the opaque torn marker).
|
||||
*
|
||||
* Shared buffering, serialization, adoption, repair, and disposal orchestration
|
||||
* for first-party backends. Third-party backends may implement the public
|
||||
* persistence seam directly.
|
||||
* @module @deepseek-ai/dsh-session-persistence/coordinator
|
||||
*/
|
||||
|
||||
@@ -30,16 +11,9 @@ import type { Session, SessionEvent, SessionId, SessionHeader } from '@deepseek-
|
||||
import { seedCoversPrefix } from './index.ts'
|
||||
|
||||
/**
|
||||
* A stored session's durable prefix as read back from a backend: its
|
||||
* {@link SessionHeader}, the preserved (seq-contiguous, parseable) event prefix,
|
||||
* and an OPAQUE `tornMarker` that is present iff a never-committed torn tail must
|
||||
* be truncated before further writes.
|
||||
*
|
||||
* The coordinator NEVER inspects `tornMarker`'s value — it only tests
|
||||
* `!== undefined` (is there a tail to repair?) and passes the value back to
|
||||
* {@link PersistenceBackend.commitRepair}. Each backend chooses its own marker
|
||||
* type: the JSONL backend uses the byte offset to truncate to, the SQLite
|
||||
* backend uses the seq to delete from (both happen to be `number`).
|
||||
* A stored session's header, valid contiguous event prefix, and optional opaque
|
||||
* torn-tail marker. The coordinator only checks marker presence and returns its
|
||||
* value to {@link PersistenceBackend.commitRepair}; each backend owns the type.
|
||||
*/
|
||||
export interface StoredPrefix<TornMarker = unknown> {
|
||||
meta: SessionHeader
|
||||
@@ -112,16 +86,9 @@ interface SessionState {
|
||||
/** The next seq the backend expects to append (the stored log length). */
|
||||
cursor: number
|
||||
/**
|
||||
* Whether the backend has physically written this session (a JSONL file /
|
||||
* SQLite row exists). `create()` registers state LAZILY — cursor 0,
|
||||
* materialized false, nothing on disk — so an empty session leaves no
|
||||
* artifact and the FIRST `appendBatch` writes the header + its events in ONE
|
||||
* transaction (the "a row exists ⇔ it has events" invariant `list`
|
||||
* relies on; a separate up-front materialize could crash leaving a row with
|
||||
* zero events). The flag is the only signal that distinguishes a session
|
||||
* registered-but-never-written from one durably present, which the reclaim
|
||||
* path needs (an abandoned id with no artifact AND no buffered events is free
|
||||
* to reuse; a materialized one is a real collision).
|
||||
* Whether lazy creation has produced a durable artifact. The first append
|
||||
* atomically materializes the header with events; reclaim logic uses this to
|
||||
* distinguish an unused id from a persisted collision.
|
||||
*/
|
||||
materialized: boolean
|
||||
/**
|
||||
@@ -166,14 +133,9 @@ export class PersistenceCoordinator<TornMarker = unknown> {
|
||||
*/
|
||||
private chains = new Map<SessionId, Promise<unknown>>()
|
||||
/**
|
||||
* Per-session init promise (onCreated). Keyed by the LIVE Session OBJECT, not
|
||||
* its id: a disposed fiber's session can be replaced by a different live
|
||||
* Session reusing the same id (HMR, an ACP reconnect), and an id-keyed cache
|
||||
* would hand the new object the old object's init promise.
|
||||
*
|
||||
* Public (readonly) so a backend can expose it for white-box tests that await
|
||||
* a specific session's init (there is no public API to await one init); the
|
||||
* coordinator itself only ever mutates it internally.
|
||||
* Init promises keyed by live session object, preventing an id-reusing
|
||||
* replacement from inheriting stale initialization. Readonly access supports
|
||||
* backend white-box tests.
|
||||
*/
|
||||
readonly inits = new Map<Session, Promise<void>>()
|
||||
|
||||
@@ -184,16 +146,11 @@ export class PersistenceCoordinator<TornMarker = unknown> {
|
||||
// --- public surface (the backend's service methods delegate here) ---
|
||||
|
||||
/**
|
||||
* Register a new session's metadata (lazy: no physical write until the first
|
||||
* {@link append}). Rejects if the id is already tracked or already persisted.
|
||||
* @param meta - the header (id, version, cwd, lineage) to record; materialized
|
||||
* as a detached lossless-JSON snapshot at call time.
|
||||
* Register detached session metadata for lazy creation on the first append.
|
||||
* @param meta - header to snapshot; duplicate tracked or persisted ids reject.
|
||||
*/
|
||||
create(meta: SessionHeader): Promise<void> {
|
||||
// Snapshot the metadata at call time: the op runs later (behind the
|
||||
// per-session chain) and the snapshot is stored as the lazy state, so keeping
|
||||
// the caller's object by reference would let a later mutation of `id`/`cwd`
|
||||
// register under one key but materialize under a different path/header.
|
||||
// Snapshot before queueing so caller mutation cannot diverge the key and header.
|
||||
const snapshot = snapshotJsonValue(meta)
|
||||
if (snapshot === undefined) {
|
||||
return Promise.reject(new TypeError('session metadata must be losslessly JSON-serializable'))
|
||||
@@ -274,32 +231,20 @@ export class PersistenceCoordinator<TornMarker = unknown> {
|
||||
const { meta, events, tornMarker } = stored
|
||||
this.assertVersion(meta)
|
||||
|
||||
// Crash-recovery: if the log ended mid-turn (real, preserved events but no
|
||||
// closing turn/end), close it durably DURING load so disk, the returned log,
|
||||
// and the cursor all agree. The interrupted turn's real events are preserved,
|
||||
// never truncated (a turn can be huge — the session-persistence RFC); only a
|
||||
// never-fully-written torn tail fragment is discarded.
|
||||
// Preserve complete interrupted events and synthesize only missing closers.
|
||||
const closers = interruptedTurnClosers(events)
|
||||
const balanced = [...events, ...closers]
|
||||
|
||||
// Make the repair durable (truncate the torn tail + append the synthetic
|
||||
// closers) BEFORE recording state — commitRepair takes `meta` directly, so
|
||||
// there is no state-path ordering dependency (uniform across backends).
|
||||
// Repair storage before publishing coordinator state.
|
||||
if (tornMarker !== undefined || closers.length > 0) {
|
||||
await this.backend.commitRepair(meta, tornMarker, closers)
|
||||
}
|
||||
// The state keeps its OWN copy of the meta; the returned value is separate so
|
||||
// a consumer mutating loaded.meta cannot corrupt the backend's metadata.
|
||||
// Keep coordinator metadata detached from the returned record.
|
||||
this.states.set(id, { meta: { ...meta }, cursor: balanced.length, materialized: true })
|
||||
return { meta, events: balanced }
|
||||
}
|
||||
|
||||
// NOTE: there is deliberately no coordinator `list()`. Listing needs none of
|
||||
// the coordinator's orchestration (no per-id serialization, no cursor, no
|
||||
// in-memory state) — it is a pure read of stored metadata. A backend's public
|
||||
// `list()` IS the {@link PersistenceBackend.list} hook (one method); routing it
|
||||
// through the coordinator would only forward to that same hook, so the
|
||||
// coordinator stays out of the listing path entirely.
|
||||
// Listing is a direct backend read and needs no coordinator state.
|
||||
|
||||
// --- per-id serialization + adoption helpers ---
|
||||
|
||||
|
||||
@@ -1,23 +1,7 @@
|
||||
/**
|
||||
* The durable session-persistence seam (`ctx.sessionPersistence`): an abstract
|
||||
* service defining WHAT a persistence backend does — durably store, reload,
|
||||
* and list sessions — without saying HOW. Implementations subclass
|
||||
* {@link SessionPersistence} and register themselves as the
|
||||
* `sessionPersistence` service; `@deepseek-ai/dsh-session-persistence-jsonl`
|
||||
* (an append-only JSONL log per session) is the first and
|
||||
* `@deepseek-ai/dsh-session-persistence-sqlite` (`node:sqlite`, one row per
|
||||
* event) is a second that validates the seam is backend-agnostic by passing
|
||||
* the same `runPersistenceContract` suite. Further backends swap in an object
|
||||
* store or a remote service without touching the consumers (the write-path
|
||||
* plugin, the agent-loop resume seam).
|
||||
*
|
||||
* The persisted unit IS the existing {@link SessionEvent} — there is no
|
||||
* parallel "persisted message" type the log must be converted to and from
|
||||
* (faithful to the event-sourced model: the log is the single source of
|
||||
* truth). Metadata that is NOT replayable conversation state (format version,
|
||||
* cwd, lineage, seed boundary) travels separately as {@link SessionHeader},
|
||||
* which is owned by `dsh-session` and re-exported here.
|
||||
*
|
||||
* Durable session-persistence seam (`ctx.sessionPersistence`). Backends store
|
||||
* {@link SessionEvent}s as the event-sourced log and carry non-replayable
|
||||
* {@link SessionHeader} metadata separately.
|
||||
* @module @deepseek-ai/dsh-session-persistence
|
||||
*/
|
||||
|
||||
@@ -51,12 +35,8 @@ export interface SessionLocation {
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether a live session's seed reproduces a persisted prefix exactly. Backends
|
||||
* use this collision check to distinguish a legitimate resume/HMR rebind from a
|
||||
* different live session reusing an existing session id.
|
||||
*
|
||||
* The comparison includes the full event payload, not just seq/type/time, so a
|
||||
* mutated seed cannot be grafted onto a durable log with the same envelope.
|
||||
* Check whether a live seed exactly reproduces a durable prefix, including full
|
||||
* payloads. This distinguishes resume/HMR rebinding from an id collision.
|
||||
* @param seed - the live session's creation-time event snapshot.
|
||||
* @param prefix - the persisted prefix the seed must reproduce.
|
||||
* @returns `true` when the prefix fits within the seed and every event matches by JSON text.
|
||||
@@ -84,32 +64,10 @@ export function assertSerializable(events: readonly SessionEvent[]): void {
|
||||
}
|
||||
|
||||
/**
|
||||
* Abstract durable session-persistence service. Subclass, implement the
|
||||
* abstract methods, and load the subclass as a plugin — it registers as
|
||||
* `ctx.sessionPersistence` (one implementation per context; loading a second
|
||||
* throws, cordis' standard duplicate-service behavior).
|
||||
*
|
||||
* Contracts every implementation MUST honor (a DB backend asserts them inside
|
||||
* a transaction; a file backend appends at EOF):
|
||||
*
|
||||
* - **Append-only; a crashed turn is closed, not truncated.** Committed events
|
||||
* — those at or below a flushed `turn/end` — are never rewritten. A crash can
|
||||
* leave an unclosed final turn whose events are real (and possibly large);
|
||||
* {@link load} preserves them and closes the orphaned turn with synthetic
|
||||
* boundary events (see {@link load}). Only a never-fully-written torn tail
|
||||
* fragment is discarded.
|
||||
* - **Contiguous seq.** A persisted log is contiguous: `events[i].seq === i`.
|
||||
* {@link load} rejects a parse error or a `seq` gap in the COMMITTED region
|
||||
* (unloadable); {@link append}'s first event `seq` MUST equal the backend's
|
||||
* stored next-seq (after `load` has balanced any interrupted turn).
|
||||
* - **JSON-serializable events.** `SessionEventMap` is merge-extensible, so
|
||||
* {@link append} materializes each complete batch through the shared
|
||||
* lossless-JSON boundary before buffering it. The public `session.events`
|
||||
* view is immutable, but persistence still snapshots direct/replay callers at
|
||||
* this independent trust boundary.
|
||||
* - **Durability.** {@link append} returns only once the batch is durable
|
||||
* (the file backend fsyncs; a DB commits). {@link create} MAY defer the
|
||||
* physical write until the first {@link append} (lazy materialization).
|
||||
* Durable append-only session storage. Implementations preserve contiguous,
|
||||
* losslessly JSON-serializable events; {@link append} resolves only after
|
||||
* durability, and {@link load} balances a complete interrupted tail without
|
||||
* rewriting committed events.
|
||||
*/
|
||||
export abstract class SessionPersistence extends Service {
|
||||
constructor(ctx: Context) {
|
||||
@@ -146,29 +104,12 @@ export abstract class SessionPersistence extends Service {
|
||||
abstract append(id: SessionId, events: readonly SessionEvent[]): Promise<void>
|
||||
|
||||
/**
|
||||
* Reload a session: its {@link SessionHeader} plus the event log up to the last
|
||||
* durable checkpoint. Returns `meta` AND `events` so the live session is
|
||||
* reconstructed with its `cwd`/lineage, not just its log.
|
||||
*
|
||||
* The loop only flushes at `turn/end`, so a crash can leave a durable log
|
||||
* whose final turn never closed: real, fully-written events sit after the last
|
||||
* `turn/end`. Those events are PRESERVED — a single turn can be huge in a
|
||||
* long-horizon task, so truncating it would destroy real work — and `load`
|
||||
* CLOSES the orphaned turn by durably appending the minimal synthetic boundary
|
||||
* events: an error `tool/result` for every `tool-call` the crash left
|
||||
* unanswered (so the rehydrated history is a valid provider transcript — a
|
||||
* dangling assistant tool-call is otherwise rejected), then a `step/end` if a
|
||||
* step was open, then a `turn/end` carrying the `{ kind: 'interrupted' }`
|
||||
* reason. The returned `events` therefore end on a balanced `turn/end` and are
|
||||
* immediately usable as a session seed. Only a never-fully-written TORN tail
|
||||
* fragment (a half-written final record) is discarded. Returned events are
|
||||
* contiguous (`events[i].seq === i`); a parse error or a `seq` gap in the
|
||||
* COMMITTED region (at or before the last real `turn/end`) makes the session
|
||||
* unloadable (reject). Rejects an unknown format `version`. See the session-persistence RFC for
|
||||
* the crash-recovery contract.
|
||||
* Load a header and balanced contiguous log. A complete interrupted final
|
||||
* turn is preserved and durably closed with missing tool errors plus any open
|
||||
* step and turn boundaries; only a torn final record is discarded. Unknown
|
||||
* versions and corruption in the committed prefix reject.
|
||||
* @param id - the persisted session to reload.
|
||||
* @returns the header plus the event log, ending on a balanced `turn/end` —
|
||||
* immediately usable as a session seed.
|
||||
* @returns the header and a log ending on a balanced `turn/end`.
|
||||
*/
|
||||
abstract load(id: SessionId): Promise<{ meta: SessionHeader; events: SessionEvent[] }>
|
||||
|
||||
|
||||
@@ -43,15 +43,9 @@ export function oneTurnLog(): SessionEvent[] {
|
||||
}
|
||||
|
||||
/**
|
||||
* Append a whole event log to a LIVE session, event by event, forwarding the
|
||||
* surface metadata each event already carries. A bare `append(e.type, e.data)`
|
||||
* over a `SessionEvent[]` widens the type argument to the union, where the
|
||||
* typed overload's mandatory-marker rule collapses to optional — and `append`'s
|
||||
* runtime guard then rejects a surface-eligible event with no marker. This
|
||||
* helper forwards the `surfaceOp`/`sourceEventSeqs` VERBATIM from the source
|
||||
* event (it does not synthesize a default), so a well-formed recorded log
|
||||
* round-trips through a live session intact and a fixture that forgot a marker
|
||||
* still trips the guard.
|
||||
* Append recorded events to a live session while forwarding surface metadata verbatim. The broad
|
||||
* `SessionEvent` union makes the typed marker optional, but the runtime guard must still reject a
|
||||
* surface event whose fixture omitted it; this helper never synthesizes a default.
|
||||
*/
|
||||
export function appendLog(session: Session, events: readonly SessionEvent[]): void {
|
||||
for (const e of events) {
|
||||
@@ -222,10 +216,10 @@ export function runPersistenceContract(name: string, make: () => Promise<Contrac
|
||||
it('append rejects non-JSON-serializable event data, naming the event type', async () => {
|
||||
const { persistence, dispose } = await make()
|
||||
try {
|
||||
// Every value `isJsonValue` rejects must be rejected by the backend, not
|
||||
// just BigInt — otherwise a backend could pass this contract while still
|
||||
// accepting values that corrupt the durable round-trip. Each is a
|
||||
// plugin-added `extra` field on a single user/message (seq 0).
|
||||
// Every value `isJsonValue` rejects must be rejected by the backend, not just BigInt —
|
||||
// otherwise a backend could pass this contract while still accepting values that
|
||||
// corrupt the durable round-trip. Each value is carried in a plugin-added field on one
|
||||
// user message so the contract covers the complete JSON-value boundary.
|
||||
const cyclic: Record<string, unknown> = { type: 'text', text: 'x' }
|
||||
cyclic['self'] = cyclic
|
||||
const badValues: unknown[] = [
|
||||
|
||||
@@ -1,28 +1,11 @@
|
||||
/**
|
||||
* Reusable ORCHESTRATION suite for any backend that composes a
|
||||
* {@link PersistenceCoordinator}. Where {@link runPersistenceContract} (in
|
||||
* contract.ts) pins the public read/write SEMANTICS, this suite pins the
|
||||
* coordinator's WRITE-PATH ORCHESTRATION — the behavior that is identical across
|
||||
* every first-party backend because it lives in the shared coordinator, not in
|
||||
* the storage primitives: the `session/created` → `session/event` →
|
||||
* `session/flush` → dispose drain, lazy materialization, fork-seed persistence,
|
||||
* the four `onCreated` adoption cases (new / HMR-adopt / collision /
|
||||
* ownerless-claim), crash-tail repair on load, and dispose-time quiescence.
|
||||
*
|
||||
* A backend imports {@link runCoordinatorContract} and calls it with a
|
||||
* {@link CoordinatorFixture} factory that knows how to (a) mount the REAL
|
||||
* backend plugin on a {@link Context} over a SHARED storage scope (so HMR/reload
|
||||
* tests can dispose one instance and mount another over the same bytes/rows),
|
||||
* and (b) inject a never-committed torn tail for one session
|
||||
* ({@link CoordinatorFixture.corruptTail}) so the through-coordinator torn-tail
|
||||
* repair branch is exercised against real storage. The suite drives everything
|
||||
* through the PUBLIC {@link SessionPersistence} API + the cordis SessionStore
|
||||
* write path — never the storage primitives directly — so it runs unchanged for
|
||||
* every backend (memory / jsonl / sqlite).
|
||||
*
|
||||
* Each scenario lives here once and runs once per backend through the fixture;
|
||||
* the per-backend specs keep ONLY their storage-mechanics tests.
|
||||
* Shared write-path orchestration contract for backends using {@link PersistenceCoordinator}.
|
||||
* Unlike the public storage-semantics suite in `contract.ts`, it covers SessionStore event wiring,
|
||||
* lazy creation, fork seed persistence, four adoption/collision cases, crash-tail repair, reload,
|
||||
* flush, and disposal quiescence through public APIs rather than storage primitives.
|
||||
*
|
||||
* Each real backend supplies a shared storage scope and optional torn-tail injector; backend specs
|
||||
* retain only storage-mechanics tests, while these scenarios run once per backend.
|
||||
* @module @deepseek-ai/dsh-session-persistence/tests/coordinator-contract
|
||||
*/
|
||||
|
||||
@@ -39,25 +22,12 @@ import { meta, oneTurnLog, appendLog } from './contract.ts'
|
||||
* the suite mounts/disposes backend instances on it and cleans it up at the end.
|
||||
*/
|
||||
export interface CoordinatorFixture {
|
||||
/**
|
||||
* Mount the REAL backend plugin (via `ctx.plugin`, the Loader path) on `ctx`,
|
||||
* over THIS fixture's shared storage scope. Returns the plugin fiber so the
|
||||
* suite can dispose a single instance (HMR/reload) while the storage — and any
|
||||
* still-live session in another fiber — survives. The caller has already
|
||||
* mounted `SessionStore` on `ctx`.
|
||||
*/
|
||||
/** Mount the real backend through `ctx.plugin` over shared storage and return only that fiber. */
|
||||
mount: (ctx: Context) => Promise<Fiber>
|
||||
|
||||
/**
|
||||
* Inject a NEVER-COMMITTED torn tail into the backend's storage for `id` at
|
||||
* the given `cwd` (the cwd the session was created with): a half-written
|
||||
* record past the committed region (JSONL: a partial line with no newline;
|
||||
* SQLite: a row with invalid `data` JSON past the committed seq). This drives
|
||||
* the coordinator's `loadCore` `tornMarker !== undefined` → `commitRepair`
|
||||
* branch against real storage.
|
||||
*
|
||||
* OMITTED by a backend that structurally has no torn tails (memory): the
|
||||
* torn-tail scenario then self-skips (asserted explicitly in the suite).
|
||||
* Inject a never-committed partial record after the durable region so `loadCore` reaches
|
||||
* `commitRepair`. Omit only when the backend structurally cannot produce torn tails.
|
||||
*/
|
||||
corruptTail?: (id: SessionId, cwd: string | undefined) => Promise<void>
|
||||
|
||||
@@ -124,10 +94,9 @@ export function runCoordinatorContract(name: string, makeFixture: () => Promise<
|
||||
})
|
||||
|
||||
it('round-trips the seed boundary (seedLength) through persistence', async () => {
|
||||
// A forked child records how many leading events were inherited via the
|
||||
// seed; the boundary must survive a reload (so a resume/replay can tell the
|
||||
// inherited prefix from the child's own events). Both backends carry it on
|
||||
// the header — JSONL on the header line, SQLite in the seed_length column.
|
||||
// A forked child records how many leading events were inherited via the seed; the
|
||||
// boundary must survive a reload (so a resume/replay can tell the inherited prefix from
|
||||
// the child's own events). JSONL stores it in the header; SQLite uses `seed_length`.
|
||||
const fix = await makeFixture()
|
||||
const { ctx, fiber } = await freshCtx(fix)
|
||||
try {
|
||||
@@ -213,10 +182,10 @@ export function runCoordinatorContract(name: string, makeFixture: () => Promise<
|
||||
})
|
||||
|
||||
it('resume: a re-created session seeded with the loaded log does not re-append its seed and continues the seq', async () => {
|
||||
// Separate backend lifecycles distinguish persisted-seed adoption from an in-memory continuation.
|
||||
const fix = await makeFixture()
|
||||
const first = await freshCtx(fix)
|
||||
try {
|
||||
// First lifecycle: persist a session through the store.
|
||||
const s1 = first.ctx.sessions.create(SessionId('resumed'), { meta: { cwd: WORK } })
|
||||
send(s1, oneTurnLog())
|
||||
await first.ctx.parallel('session/flush', s1)
|
||||
@@ -224,9 +193,6 @@ export function runCoordinatorContract(name: string, makeFixture: () => Promise<
|
||||
await first.fiber.dispose()
|
||||
}
|
||||
|
||||
// Second lifecycle: a NEW backend instance + a session re-created with the
|
||||
// same id SEEDED with the loaded events. onCreated adopts the stored log
|
||||
// (does not re-persist the seed); a new turn appends at seq 6.
|
||||
const second = await freshCtx(fix)
|
||||
try {
|
||||
const loaded = await second.ctx.sessionPersistence.load(SessionId('resumed'))
|
||||
@@ -237,7 +203,6 @@ export function runCoordinatorContract(name: string, makeFixture: () => Promise<
|
||||
await second.ctx.parallel('session/flush', s2)
|
||||
|
||||
const reloaded = await second.ctx.sessionPersistence.load(SessionId('resumed'))
|
||||
// 6 original + 2 new, contiguous, no duplicated seed.
|
||||
expect(reloaded.events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4, 5, 6, 7])
|
||||
} finally {
|
||||
await second.fiber.dispose()
|
||||
@@ -304,10 +269,9 @@ export function runCoordinatorContract(name: string, makeFixture: () => Promise<
|
||||
session.append('turn/end', { turn: 1, reason: { kind: 'completed' } })
|
||||
await ctx.parallel('session/flush', session)
|
||||
|
||||
// Hot-reload: dispose instance 1, mount instance 2 over the SAME storage
|
||||
// while the session stays live. Instance 2 has an empty states map but the
|
||||
// log is materialized and is a prefix of the live events — it must ADOPT
|
||||
// (not reject). A second turn appended after reload then persists.
|
||||
// Hot-reload: dispose instance 1, mount instance 2 over the same storage while the
|
||||
// session stays live. The new instance has no coordinator state but must adopt the
|
||||
// materialized prefix, then persist another turn rather than rejecting it as a collision.
|
||||
await backend1.dispose()
|
||||
await fix.mount(ctx)
|
||||
session.append('turn/start', { turn: 2, trigger: { kind: 'message', source: { kind: 'user' } } })
|
||||
@@ -399,9 +363,9 @@ export function runCoordinatorContract(name: string, makeFixture: () => Promise<
|
||||
await first.fiber.dispose()
|
||||
}
|
||||
|
||||
// A FRESH backend + a NEW live session with the same id but NO explicit
|
||||
// resume. onCreated treats it as new; create() rejects because a log already
|
||||
// exists. The rejection surfaces via the init promise (flush awaits it).
|
||||
// A fresh backend + a NEW live session with the same id but NO explicit resume. onCreated
|
||||
// treats it as new; create() rejects because a log already exists, and `flush()` surfaces
|
||||
// that initialization rejection.
|
||||
const second = await freshCtx(fix)
|
||||
try {
|
||||
const s2 = second.ctx.sessions.create(SessionId('collide'), { meta: { cwd: WORK } })
|
||||
|
||||
@@ -16,18 +16,10 @@ type MemoryStore = Map<string, { meta: SessionHeader; events: SessionEvent[] }>
|
||||
interface MemoryConfig { store?: MemoryStore }
|
||||
|
||||
/**
|
||||
* A trivial in-memory {@link SessionPersistence} that composes a
|
||||
* {@link PersistenceCoordinator} over a dependency-free `Map`-backed
|
||||
* {@link PersistenceBackend}. It is BOTH the coordinator's reference vehicle
|
||||
* (the simplest possible storage — a `Map<id, {meta, events}>` with no torn
|
||||
* tails, so `tornMarker` is always undefined) and the cover for the abstract
|
||||
* base's constructor + service registration. The real durable backends are
|
||||
* `@deepseek-ai/dsh-session-persistence-jsonl` / `-sqlite`.
|
||||
*
|
||||
* The store can be supplied via config so two backend instances share one Map —
|
||||
* the in-RAM analogue of two backends over the same file/db, which the
|
||||
* coordinator orchestration suite's HMR/reload tests need (a fresh instance with
|
||||
* an empty in-memory states map adopting an already-materialized session).
|
||||
* Reference {@link PersistenceCoordinator} vehicle and abstract-service coverage, backed by a
|
||||
* dependency-free map with atomic writes and no torn-tail marker. Supplying the map lets multiple
|
||||
* instances share materialized sessions, the in-memory analogue of reload over one file/database;
|
||||
* durable behavior is covered by the JSONL and SQLite backends.
|
||||
*/
|
||||
class MemoryPersistence extends SessionPersistence implements PersistenceBackend<never> {
|
||||
static inject = ['sessions']
|
||||
@@ -92,8 +84,7 @@ class MemoryPersistence extends SessionPersistence implements PersistenceBackend
|
||||
}
|
||||
const existing = this.store.get(m.id)
|
||||
if (!existing) {
|
||||
// First batch: `_isMaterialized` is false (the coordinator only omits
|
||||
// materialization on the first batch); writing the entry IS the materialization.
|
||||
// The coordinator sends the first batch for materialization; later batches append.
|
||||
this.store.set(m.id, { meta: structuredClone(m), events: structuredClone(events) as SessionEvent[] })
|
||||
} else {
|
||||
existing.events.push(...structuredClone(events) as SessionEvent[])
|
||||
@@ -126,12 +117,8 @@ runPersistenceContract('memory', async () => {
|
||||
}
|
||||
})
|
||||
|
||||
// Run the shared coordinator orchestration suite against the in-memory backend.
|
||||
// A per-fixture Map is the shared "storage", so two mounted instances see the
|
||||
// same materialized sessions (HMR/reload). `corruptTail` is OMITTED: a Map store
|
||||
// writes atomically in RAM and has no torn tails, so the suite's torn-tail test
|
||||
// self-skips (and asserts the omission). The real torn-tail repair branch is
|
||||
// covered by the jsonl/sqlite fixtures, which CAN inject one.
|
||||
// Each fixture shares one map across mounts. No `corruptTail` is supplied because map writes are
|
||||
// atomic; the suite asserts that skip while JSONL and SQLite cover the repair branch.
|
||||
runCoordinatorContract('memory', async (): Promise<CoordinatorFixture> => {
|
||||
const store: MemoryStore = new Map()
|
||||
return {
|
||||
|
||||
Reference in New Issue
Block a user