Merge remote-tracking branch 'origin/master' into session-query-trace-filter
# Conflicts: # docs/rfc/INDEX.md # packages/README.md
This commit is contained in:
Hypatia May
@@ -0,0 +1,48 @@
|
||||
# RFC: Interactive side sessions and merge-back
|
||||
|
||||
Status: proposed
|
||||
|
||||
## Problem
|
||||
|
||||
A user deep in a live session often wants to fork the conversation — ask "why did we structure it this way", explore an alternative, get an explanation — without polluting the main context and without abandoning the surface they are in. The harness has every primitive this needs and no product face for it: [the session-store fork API](../../implemented/feature/2026-06-30-session-store-fork-api.md) produces a `Session` with no attached agent and no way for a client to reach it, and [the fork subagent](../../implemented/feature/2026-06-21-subagent-capability-seam.md) seeds a child with the parent's prefix but runs it as a model-driven task whose whole transcript collapses into one tool result — neither yields a forked conversation the USER can talk to.
|
||||
|
||||
The return path is missing entirely: nothing carries a conclusion from one branch back into another. Whatever the user learns in an exploration branch is copy-pasted by hand or lost, with no provenance and no replayable record.
|
||||
|
||||
Terminal-first competitors ship half of each: single-shot side questions with inherited context, and user-switched branch copies that force a client restart. None ship the return verb. A harness that owns its session store and context assembly can do both cheaply — and can do so without breaking the provider prefix cache, which third-party wrappers structurally cannot.
|
||||
|
||||
## Proposal
|
||||
|
||||
A **side session** is an ordinary live session forked from a source session at its last completed turn, attached to its own agent, framed as a read-only advisor, with one new verb — **merge-back** — that hands a condensed note to the parent.
|
||||
|
||||
- **Fork + attach composes existing primitives.** The child is created via `ctx.agents.create({ seed, meta })` with the parent's balanced completed-turn prefix (the same slice the fork subagent takes) and `parentSession`/`seedLength` lineage stamped in `meta`. No new core service and no session-store change, for the same reasons [the fork API RFC](../../implemented/feature/2026-06-30-session-store-fork-api.md) rejected a standalone fork service.
|
||||
- **Advisor framing rides the log, not the system prompt.** The rules ("you are a read-only side advisor; explain, do not mutate; refuse task continuation") are `inject()`ed as a `context/message` with source `{ kind: 'plugin', plugin: 'sidechat' }` immediately after creation. The child's system prompt stays byte-identical to the parent's, so the provider's prefix cache covers the inherited history.
|
||||
- **Merge-back is one condense turn plus one injection.** The child is prompted for a bounded handback note (a hard length cap), which is then `inject()`ed into the PARENT as a `context/message` with the same plugin source. The parent's next request sees it at its chronological position; replay and [reconstructability](../../implemented/architecture/2026-07-05-reconstructable-requests.md) hold by construction; no new event type enters [the session vocabulary](../../../core-data-structures/session.md).
|
||||
- **The surface binding is deliberately unspecified.** How a user invokes the fork and the merge, and how a handback note is presented, are client concerns; while the harness speaks through protocols whose UI it does not control, this RFC pins only the surface-agnostic mechanics above and leaves presentation to the first surface the project owns.
|
||||
|
||||
Scope also excludes: rewind productization (mechanically the same fork-to-an-earlier-boundary, an entirely different product question), session tree views, a model-facing side-session tool, and `forkName`/`mergedInto` metadata.
|
||||
|
||||
## Prototype
|
||||
|
||||
A spike (branch `spike/side-sessions-b`) validates the mechanics against the live adapter: a fork that leaves the source log untouched, a child seeded with the full inherited prefix, a multi-turn advisor exchange that demonstrably uses parent history, and a merge-back the parent's next turn quotes correctly.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **Carry side conversations through the subagent seam.** Rejected: a subagent child is a model-driven run — the parent's model spawns it, drives it, and consumes its result as a tool result. A side session is user-driven, needs its own client-visible session and lifecycle, and must outlive any single parent turn.
|
||||
- **Persona via `system-prompt/assemble` section filtering.** Rejected as the default path: any system-prompt byte change invalidates the provider prefix cache from token 0, forfeiting the cheap fork that makes side sessions attractive on long histories. The filter seam remains available for deployments that prefer hard prompt separation over cache reuse.
|
||||
- **A dedicated `sidechat/*` event family for the handback.** Deferred: `context/message` with a mandatory plugin source already satisfies durability, provenance, and replay. A first-class event earns its catalog, persistence, and snapshot costs only if a UI needs to render handbacks as dedicated cards.
|
||||
- **Binding the proposal to a protocol surface now.** Rejected in review: the harness currently speaks through client-owned UIs it does not control, so any presentation contract written today would be speculative. The RFC pins the surface-agnostic mechanics and defers presentation to the first surface the project owns.
|
||||
- **Surface-level mirroring of the handback.** Rejected in review: a visible record emitted outside the log vanishes on replay while the model still sees it. Whatever surface eventually renders the handback must derive its presentation from the durable `context/message`, so live and replayed views come from the same event.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- Forking a live session yields a child agent seeded with the source's balanced completed-turn prefix, with `parentSession` and `seedLength` in its header and a system prompt byte-identical to the parent's; the source log is untouched by the fork.
|
||||
- The advisor framing is exactly one plugin-sourced `context/message` at the head of the child's appended history — never a system-prompt change.
|
||||
- Merge-back appends exactly one length-capped `context/message` to the parent with source `plugin: sidechat`; the parent's next request sees it, and replay reproduces it at the same position.
|
||||
- Parent and child run concurrently without cross-talk between their logs or streams.
|
||||
- Coverage: unit tests for the fork/attach and merge-back mechanics; surface-level snapshot coverage lands with whichever surface first binds the feature.
|
||||
|
||||
## Risks
|
||||
|
||||
- **Read-only is advisory in v1.** The rules are injected context, not enforcement; a determined prompt can still drive mutating tools. The hard gate is a `tools/pre-execute` deny via [the interception seams](../../implemented/feature/2026-06-30-interception-seams.md), and the RFC's advisor framing is written so that gate can be added without changing the mechanics.
|
||||
- **A compacted source forks its compacted view.** The child inherits the summary, not the original turns; whichever surface binds the feature should disclose this once [compaction](../../implemented/feature/2026-06-18-compaction-capability-seam.md) ships in this path.
|
||||
- **Handback notes spend parent tokens.** The length cap and one-note-per-merge bound the cost, but a user who merges repeatedly accumulates notes; a future consolidation pass belongs to the compaction work, not here.
|
||||
@@ -0,0 +1,41 @@
|
||||
# RFC: Stream workflow progress through tool calls
|
||||
|
||||
Status: proposed
|
||||
|
||||
## Problem
|
||||
|
||||
The workflow engine intentionally emits balanced `workflow/*` observation events for run, phase, narration, and child-agent progress, but no production consumer presents them. Editors therefore show one pending workflow tool card until the final result even while the engine already reports which phase is active, what the script logged, and which children started or settled. The [dynamic-workflows decision](../../implemented/feature/2026-07-05-dynamic-workflows.md) explicitly reserves ACP progress UI for this event stream.
|
||||
|
||||
Making `dsh-acp` listen to workflow events directly would invert the capability boundary: the generic UI bridge would depend on an optional workflow package and special-case one tool name. The tool pipeline already owns the routing facts a live update needs—agent and call id—but exposes only pure pending/final presenters, so a long-running tool has no provider-neutral way to report transient UI state between them.
|
||||
|
||||
## Proposal
|
||||
|
||||
Add a live progress channel to `dsh-tools`. The registry-owned `ToolExecution` gains `reportProgress(view): boolean`, where `view` is a detached provider-neutral generic progress snapshot containing an optional replacement title and UI-facing content blocks. Progress cannot change the call's args-derived card tag, kind, raw input, locations, terminal intent, or diff intent; it updates only the live title/content within the presentation chosen up front. While the execution is active, the method validates and snapshots the view, then dispatches a contained, agent-scoped `tools/progress` observation carrying the authoritative execution identity and snapshot. Once final-result processing begins it returns `false` and emits nothing, so a late asynchronous reporter cannot overwrite a terminal card. Observer exceptions are logged and cannot fail the tool.
|
||||
|
||||
`dsh-acp` consumes `tools/progress` generically. It resolves the execution's agent through its existing agent-to-session map and emits an in-progress `tool_call_update` for the same call id. Because reporting is available only inside the tool execution pipeline, the durable `tool/call` and its ACP `tool_call` always precede the first update; closing the reporter before `tools/result` ensures no progress update follows the completed/failed card. Progress is live UI state rather than model input or durable history: session replay continues to reconstruct the pending and final cards from `tool/call` and `tool/result` without replaying transient updates.
|
||||
|
||||
`dsh-tool-workflow` becomes the first producer. Each tool execution installs a compact event capture before calling `ctx.workflows.start()`, because a valid engine may emit progress synchronously inside `start()`. Until the call returns, the capture reduces observed events into candidate states keyed by `WorkflowRunInfo.id`; it then selects the returned `WorkflowRun.id`, discards other candidates, reports the accumulated snapshot, and routes later matching events directly. If `start()` throws, the capture is disposed and its candidates are dropped. This preserves engine swappability without adding observer correlation to `WorkflowStartRequest` or requiring progress to wait until `start()` returns.
|
||||
|
||||
The reducer consumes the existing start, phase, log, agent-start, agent-end, and end events, reporting a replacement snapshot with the current phase, latest log line, active child labels, and completed/failed/cancelled counts. It does not accumulate a narration transcript; settled children leave the active set and become counters. `workflow/end`, tool settlement, or plugin disposal removes the reducer entry and event capture. The six workflow events, their metadata, paired child lifecycle, run handle, cancellation channels, and observer containment remain unchanged; third-party observers can continue consuming them directly.
|
||||
|
||||
Update the tool execution/presentation docs, generated event and API catalogs, workflow package docs, and the workflow data-structure catalog. ACP integration coverage must exercise the real workflow tool and worker seam with a scripted model boundary; the primary ACP snapshot suite adds one workflow-progress scenario because this changes the editor-facing transcript.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**Delete the workflow observation surface.** Rejected in [the collapse-workflow simplification](../../rejected/simplification/2026-07-12-collapse-workflow-to-foreground-core.md): the events and their balanced lifecycle are intentional, and the missing piece is a consumer.
|
||||
|
||||
**Teach ACP about workflows directly.** This could map `WorkflowRunInfo` to a session and card, but it would make the generic bridge depend on an optional capability and bypass the rule that tools own presentation intent. A tool-progress channel solves the same routing problem for every long-running tool.
|
||||
|
||||
**Persist every progress update as a session event.** That would make live narration replayable, but it would permanently enlarge logs with state whose authoritative durable outcome is already the tool call/result pair. If resumable workflow progress becomes a product requirement, it needs a workflow-journaling design rather than UI snapshots disguised as durable facts.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- `ToolExecution.reportProgress()` is registry-owned, agent-scoped, snapshotting, observer-contained, and returns `false` without dispatch after terminal processing starts.
|
||||
- ACP routes progress to the correct call in the correct live session; concurrent workflows in different sessions cannot cross-talk, and no `tool_call_update` appears before its `tool_call` or after its terminal update.
|
||||
- Workflow progress shows the current phase, latest log line, active children, and outcome counts while preserving all existing `workflow/*` events and run semantics; a seam test engine that emits start, phase, log, child, and end events synchronously inside `start()` loses none of that reducer state.
|
||||
- Cancellation, worker death, tool failure, session close, and plugin disposal release reducer state; replay emits only the durable pending/final card pair.
|
||||
- Unit, workflow integration, ACP integration, snapshot, typecheck, coverage, doc-sync, module-graph, build, and hygiene gates pass.
|
||||
|
||||
## Risks
|
||||
|
||||
This adds a public live-progress method and event to the tool seam, so implementations must keep the active/terminal boundary exact and detach snapshots before observers see them. The pre-start capture can briefly observe unrelated workflow runs, so it holds only compact candidate state keyed by run id and drops every non-matching candidate as soon as `start()` returns. A workflow can emit many progress changes; the bounded reducer avoids transcript growth but still sends one UI update per meaningful event after correlation. If measured clients need coalescing, it must be a defaulted validated bridge configuration rather than a hardcoded throttle. Transient progress intentionally disappears on replay, so the final tool result remains the only durable workflow card content.
|
||||
Reference in New Issue
Block a user