fix(fs-local): bound overwrite contextual diff bases

Rebuild of the fs-overwrite-diff-bound branch on current master. Adds the
diffBasisMaxBytes Config field (10 MiB default, capped by runtime
allocation/decode limits), gates both overwrite sides, and reads the prior
basis from the bounded opened descriptor in cancellation-aware chunks; any
post-stat size change returns a null basis. Also pins the one-extra-byte
growth probe with a regression and drops the now-covered v8 ignore.
This commit is contained in:
ZiyaZhang
2026-08-07 11:05:27 -07:00
parent a29a0a4773
commit 16c1eaf546
19 changed files with 452 additions and 47 deletions
@@ -0,0 +1,6 @@
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write .agents/notes/implemented/bug-fix/2026-07-30-bounded-overwrite-diff-basis.md
2026-07-30-bounded-overwrite-diff-basis.md: 353b538a12b8cf48dfa3a561c62d6d8ab9a8bfcf
2026-07-30-bounded-overwrite-diff-basis.zh.md: e2b473a21d16dc47b5b8bf781b8ddffdf443955b
@@ -0,0 +1,31 @@
# Agent Note: Bound overwrite contextual-diff bases at the provider
Status: implemented
English | [中文](2026-07-30-bounded-overwrite-diff-basis.zh.md)
## Problem
`dsh-fs-local` returned the complete prior file in `FsWriteOutcome.before` so consumers could build a contextual overwrite diff. That presentation-only pre-read was unbounded: a large overwrite could allocate the entire prior file, and checking an earlier path stat alone could not enforce a limit because an external process could replace or grow the file between the stat and the read. A large replacement also made the contextual hunk approach the replacement size even when the prior file was small. This closes the deferred bound recorded by [result-time applied-hunk diffs](../../archived/architecture/2026-07-02-result-time-applied-hunk-diffs.md).
## Decision
`LocalFileSystem.Config.diffBasisMaxBytes` is a positive safe-integer deployment setting no greater than the runtime's Buffer-allocation and string-decoding limits, with a 10 MiB default. An overwrite supplies `before` only when the UTF-8 replacement is strictly below that limit and the prior file opened for the basis also ends below it. The prior read opens a descriptor, checks that descriptor, and reads at most the configured byte count in cancellation-aware chunks; reaching the boundary returns `null`. A size change after descriptor stat also returns `null`, even if the final size remains below the limit, because a partial prefix would be an incorrect diff basis. Binary or invalid UTF-8 prior content likewise returns `null`. These outcomes do not block the atomic write.
The local provider owns this decision because `before` is its optional, best-effort basis: it can avoid acquiring prior content that the configured pair limit has already made ineligible. `tool-fs` continues to own diff computation, retention, and presentation. The setting is independent of `tool-fs.readStreamMinSize`; read routing and overwrite presentation are different policies and need not share a value.
`before: null` asks consumers to use their existing whole-file fallback. The limit bounds only the extra prior-content acquisition and eligibility for a contextual pair. It does not bound the caller-owned replacement, the returned `after` value, or a consumer's fallback rendering.
## Alternatives considered
**Keep a hardcoded threshold equal to the read tool's streaming threshold.** Rejected because the read threshold is deployment-configurable and consumer-owned. Two same-valued constants would create an unenforced cross-package coupling, while the overwrite basis is itself a deployment memory/presentation choice.
**Gate only the prior side in the provider and cap new-content diffing in `tool-fs`.** Rejected because it would acquire prior text even when the provider's configured pair limit already excludes the replacement, and it would split one `before` eligibility rule across two plugins. Consumers remain free to impose additional output limits.
**Trust the initial `probe()` size before using an ordinary whole-file read.** Rejected because that size can become stale before the read. The descriptor reader must enforce the bound on the object it actually reads.
**Stream a contextual diff for arbitrarily large pairs.** Rejected for this bug fix because the current filesystem seam returns complete `before`/`after` strings and the current diff implementation consumes them. A streaming diff would require a separate cross-package protocol and presentation design.
## Consequences
Deployments can tune the extra overwrite-basis cost without changing read routing. At or above the exclusive limit, overwrites still succeed and remain visible through the whole-file fallback, but lose contextual hunks. Below the limit, the provider can still hold almost `diffBasisMaxBytes` of prior text in addition to the caller's replacement. The bounded descriptor read adds an open/stat/read sequence for eligible overwrites, while preventing a stale path probe from turning that sequence into an unbounded allocation.
@@ -0,0 +1,31 @@
# Agent Note: 在提供方限制覆写上下文 diff 基础
Status: implemented
[English](2026-07-30-bounded-overwrite-diff-basis.md) | 中文
## Problem
`dsh-fs-local` 会在 `FsWriteOutcome.before` 中返回完整旧文件,供消费方生成覆写上下文 diff。这个仅用于展示的预读没有上限:大文件覆写可能分配整个旧文件;而仅检查较早的路径 stat 也无法真正实施上限,因为外部进程可以在 stat 与读取之间替换文件或扩大文件。即使旧文件很小,大替换内容也会使上下文 hunk 接近替换内容本身的大小。本改动关闭了 [result-time applied-hunk diff](../../archived/architecture/2026-07-02-result-time-applied-hunk-diffs.md) 中记录的暂缓上限事项。
## Decision
`LocalFileSystem.Config.diffBasisMaxBytes` 是一个不超过运行时 Buffer 分配和字符串解码上限的正安全整数部署配置,默认 10 MiB。只有当 UTF-8 替换内容严格低于该上限,且为生成基础而打开的旧文件最终也低于该上限时,覆写才提供 `before`。旧文件读取会打开文件描述符、检查该描述符,并按可响应取消的分块最多读取配置的字节数;一旦到达边界便返回 `null`。描述符 stat 后发生大小变化时同样返回 `null`,即使最终大小仍低于上限,因为部分前缀会成为错误的 diff 基础。旧内容为二进制或无效 UTF-8 时也返回 `null`。这些结果都不会阻止原子写入。
本地提供方拥有该决策,因为 `before` 是它提供的可选、尽力而为的基础:当配置的成对上限已使替换内容不合格时,它可以避免获取旧内容。`tool-fs` 继续拥有 diff 计算、保留与展示。该配置独立于 `tool-fs.readStreamMinSize`;读取路由与覆写展示是不同策略,无需共享数值。
`before: null` 要求消费方使用既有的整文件回退。该上限只限制额外获取旧内容的成本,以及上下文内容对是否合格;它不限制调用方持有的替换内容、返回的 `after` 值或消费方的回退渲染。
## Alternatives considered
**保留一个与读取工具流式阈值相等的硬编码阈值。** 否决,因为读取阈值可由部署配置,且归消费方所有。两个同值常量会形成无法强制的一致性耦合,而覆写基础本身也是部署层面的内存与展示选择。
**提供方只限制旧内容一侧,并在 `tool-fs` 中限制新内容 diff。** 否决,因为当提供方配置的成对上限已经排除替换内容时,这仍会获取旧文本;同时会把同一条 `before` 合格规则拆到两个插件中。消费方仍可自由施加额外的输出限制。
**信任初次 `probe()` 的大小,再执行普通整文件读取。** 否决,因为该大小可能在读取前变旧;描述符读取必须对它真正读取的对象实施上限。
**为任意大的内容对流式生成上下文 diff。** 本次缺陷修复不采用,因为当前文件系统 seam 返回完整的 `before`/`after` 字符串,当前 diff 实现也消费这两个字符串。流式 diff 需要独立的跨包协议与展示设计。
## Consequences
部署可以调整额外的覆写基础成本,而不改变读取路由。达到或超过排他上限时,覆写仍会成功,并通过整文件回退保持可见,但不再提供上下文 hunk。低于上限时,除调用方的替换内容外,提供方仍可能持有接近 `diffBasisMaxBytes` 的旧文本。对于合格覆写,有上限的描述符读取会增加一次 open/stat/read 序列,同时防止陈旧路径探测把该序列变成无上限分配。
+10 -5
View File
@@ -460,10 +460,15 @@ Source: [`packages/host/frontend-static/src/index.ts:28`](../packages/host/front
export interface Config {
/** Base directory for relative paths. Defaults to `process.cwd()`. */
cwd?: string
/**
* Exclusive UTF-8 byte limit on each overwrite-diff side, capped by the
* runtime's safe allocation/decode maximum. Defaults to 10 MiB.
*/
diffBasisMaxBytes?: number
}
```
Source: [`packages/fs/fs-local/src/index.ts:38`](../packages/fs/fs-local/src/index.ts)
Source: [`packages/fs/fs-local/src/index.ts:39`](../packages/fs/fs-local/src/index.ts)
## `@deepseek-ai/dsh-fs-sandbox`
@@ -471,10 +476,10 @@ Requires: `sandboxPolicy`
```ts config-catalog
/**
* Plugin config: the local backend's knobs, verbatim (only `cwd`, the resolve
* base for relative paths). The sandbox default (mode + `workspace-write`
* fallback root) is NOT here — `ctx.sandboxPolicy` resolves each calling
* session for every enforcing capability.
* Plugin config: the local backend's knobs verbatim (`cwd` resolution default
* and `diffBasisMaxBytes` overwrite-presentation bound). The sandbox default
* (mode + `workspace-write` fallback root) is NOT here — `ctx.sandboxPolicy`
* resolves each calling session for every enforcing capability.
*/
export type Config = LocalConfig
```
@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write docs/core-data-structures/filesystem.md
filesystem.md: 110c1fd428b15c5094f9dcc94050cad61c324373
filesystem.zh.md: 010ec22a5d4a29555425c3deede08b317cba7004
filesystem.md: 4862a25e3b8922a8709b4b025acd5436a2228082
filesystem.zh.md: 2b5c7abec1555cc2cbb909eaae60d9022ae297dd
+5 -4
View File
@@ -134,10 +134,11 @@ interface FsWriteOutcome {
version: FsVersion
/**
* The file's content BEFORE the write, or `null` when the file did not exist
* (a create) or was undiffable (binary/non-UTF-8). LF-normalized storage text
* (the diff basis), never a diff — a consumer computes the result-time
* contextual diff from `before`/`after` when `before` is present, else falls
* back to a whole-file diff.
* (a create) or the backend declined a contextual basis (for example, a
* binary/non-UTF-8 prior file or either overwrite side reaching its exclusive limit).
* LF-normalized storage text (the diff basis), never a diff — a consumer
* computes the result-time contextual diff from `before`/`after` when
* `before` is present, else falls back to a whole-file diff.
*/
before: string | null
/** The file's content AFTER the write, LF-normalized to share `before`'s diff basis. */
+5 -4
View File
@@ -134,10 +134,11 @@ interface FsWriteOutcome {
version: FsVersion
/**
* The file's content BEFORE the write, or `null` when the file did not exist
* (a create) or was undiffable (binary/non-UTF-8). LF-normalized storage text
* (the diff basis), never a diff — a consumer computes the result-time
* contextual diff from `before`/`after` when `before` is present, else falls
* back to a whole-file diff.
* (a create) or the backend declined a contextual basis (for example, a
* binary/non-UTF-8 prior file or either overwrite side reaching its exclusive limit).
* LF-normalized storage text (the diff basis), never a diff — a consumer
* computes the result-time contextual diff from `before`/`after` when
* `before` is present, else falls back to a whole-file diff.
*/
before: string | null
/** The file's content AFTER the write, LF-normalized to share `before`'s diff basis. */
+2 -2
View File
@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write packages/fs/fs-local/README.md
README.md: 6d344fa3fef7f6bda6c0daa50184661156a925a7
README.zh.md: 4c94de64561d805684f91f02d5b0dfc375f0d04f
README.md: 9efee1ed3c33c825b20b4565f3c6cef7200ce4a2
README.zh.md: 5554017f3d528e25decffe2f866c843039bb0457
+2 -2
View File
@@ -18,7 +18,7 @@ await ctx.plugin(LocalFileSystem, { cwd: process.cwd() })
- **`stat` / `lstat`** — return target metadata or `undefined` when absent. `stat` reports `FsInfo` for an already resolved target (`version` = an opaque token derived from bigint `dev:ino:size:mtimeNs:ctimeNs`, `type` of `file`/`directory`/`other`, byte `size`); path-shaped `lstat` reports `FsPathInfo` without following the final symlink and can therefore return `symlink`. Both check cancellation before and after their asynchronous metadata probe, so an abort that lands in flight reports `FS_ABORTED` rather than stale absence.
- **`readText` / `streamText`** — UTF-8 only. `readText` reads the whole file; `streamText` streams it in chunks (cross-chunk decoding) so a huge file never has to be held whole in memory. Both reject invalid UTF-8 and NUL-byte binary samples (`FS_NOT_TEXT`) and non-regular targets. The `read` tool (`@deepseek-ai/dsh-tool-fs`) decides which to call by size and owns the line windowing.
- **`listDir`** — lists one directory level in stable `name.localeCompare()` order. Each entry carries the child basename, type, resolved child target (`displayPath` under the listed directory, `targetKey` as the realpath identity), and cheap stat metadata (`version`, plus `size` for regular files). It never opens or decodes file contents. Missing targets report `FS_NOT_FOUND`, file/special-file targets report `FS_NOT_DIRECTORY`, aborted calls report `FS_ABORTED`, permission failures report `FS_PERMISSION_DENIED`, and other listing or child metadata I/O failures report `FS_IO_ERROR`. Broken/disappeared children are returned as `other` without metadata, but permission/IO failures while resolving a child fail the whole listing with a structured `FsError`.
- **`writeText`** — atomic: writes to a temp file opened exclusively (`wx`, `0o600`) inside a randomly-named private staging dir (`0o700`) next to the target, fsyncs, then renames over the target. An existing file's mode is preserved, while new files default to `0o600`; on Windows a new file inherits the destination directory's DACL, while replacement copies the target DACL onto the empty temp before writing and publishes through `ReplaceFileW` so the original access policy survives ([Windows DACL preservation Agent Note](../../../.agents/notes/implemented/bug-fix/2026-07-19-windows-atomic-write-dacl-preservation.md)). The `expected` guard is OPTIONAL: omitting it unconditionally creates-or-overwrites; `createIfAbsent` creates a missing target and rejects an existing one (`FS_NOT_OBSERVED`); `replaceIfVersion` replaces only at the observed version (a missing target or mismatch is `FS_STALE_VERSION`).
- **`writeText`** — atomic: writes to a temp file opened exclusively (`wx`, `0o600`) inside a randomly-named private staging dir (`0o700`) next to the target, fsyncs, then renames over the target. An existing file's mode is preserved, while new files default to `0o600`; on Windows a new file inherits the destination directory's DACL, while replacement copies the target DACL onto the empty temp before writing and publishes through `ReplaceFileW` so the original access policy survives ([Windows DACL preservation Agent Note](../../../.agents/notes/implemented/bug-fix/2026-07-19-windows-atomic-write-dacl-preservation.md)). The `expected` guard is OPTIONAL: omitting it unconditionally creates-or-overwrites; `createIfAbsent` creates a missing target and rejects an existing one (`FS_NOT_OBSERVED`); `replaceIfVersion` replaces only at the observed version (a missing target or mismatch is `FS_STALE_VERSION`). An overwrite returns the prior text as its contextual diff basis only when both the opened prior file and UTF-8 replacement are strictly below `config.diffBasisMaxBytes` (default 10 MiB). The descriptor read enforces that limit even if an external writer replaces or changes the file size after the initial probe. Otherwise the provider returns `before: null`, so presentation uses its whole-file fallback.
- **`editText`** — atomic literal read-modify-write over the same primitive, serialized per target by a mutation lock. The `expected` guard is OPTIONAL: when supplied it verifies the version BEFORE literal matching (a stale edit reports `FS_STALE_VERSION`, never `FS_EDIT_NOT_FOUND`/`FS_AMBIGUOUS_EDIT` against newer content); omitting it edits the current content unconditionally. A missing target reports `FS_STALE_VERSION` either way. LF-normalizes for matching, restores the file's dominant CRLF/LF style, and rejects empty `oldString` / zero matches (`FS_EDIT_NOT_FOUND`) or ambiguous multi-matches without `replace_all` (`FS_AMBIGUOUS_EDIT`).
The package-root SDK surface is the default/named `LocalFileSystem` class plus `Config`. Raw I/O lives in `src/fsio.ts` (Cordis-free, independently unit-tested); `src/index.ts` is the thin service wiring.
@@ -34,8 +34,8 @@ No direct invalidation; the named consumer owns any request-prefix changes.
## Known Limitations and Deferred Work
- **`config.cwd` is not a sandbox** — it is a resolution default, not containment: absolute paths and `..` escape it. Enforce containment with a stricter `ctx.fs` backend or a permission plugin on the `tools/execute` waterfall ([capability-seam Agent Note](../../../.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.md#consequences)).
- **An overwrite reads the whole prior file into memory** — solely as the UI diff basis; bounding that pre-read above a size threshold is deferred (`TODO(overwrite-diff-bound)`).
- **Version tokens are `mtimeMs:size`** — an external change that preserves both within the filesystem's timestamp granularity defeats the stale guard.
- **`editText` holds the whole file (plus the edited copy) in memory** — streaming exists only on the read path.
- **A sub-limit overwrite still buffers a contextual basis** — `writeText` may retain up to just below `config.diffBasisMaxBytes` of prior text in addition to the caller-owned replacement; the bound does not cap the returned `after` value or presentation's whole-file fallback.
- **Binary detection is asymmetric** — reads NUL-sample only the first 8192 bytes while edits scan the whole buffer, so a file with a late NUL reads fine but rejects edits.
- **The per-target mutation lock is in-process only** — a writer in another process is caught only by the optional version guard, never serialized.
+2 -2
View File
@@ -18,7 +18,7 @@ await ctx.plugin(LocalFileSystem, { cwd: process.cwd() })
- **`stat` / `lstat`**:返回目标元数据;目标不存在时返回 `undefined`。`stat` 为已解析目标报告 `FsInfo`(`version` 是由 bigint `dev:ino:size:mtimeNs:ctimeNs` 派生的不透明 token,`type` 为 `file`/`directory`/`other`,`size` 以字节计);路径形态的 `lstat` 不跟随最后一个符号链接,报告 `FsPathInfo`,因此可以返回 `symlink`。两者都会在异步元数据探测前后检查取消,因此异步探测进行期间发生的中止会报告 `FS_ABORTED`,而非已失效的「不存在」结果。
- **`readText` / `streamText`**:只支持 UTF-8。`readText` 读取整个文件;`streamText` 按分片流式读取(跨分片解码),因此超大文件无需整体保存在内存中。两者都会拒绝无效 UTF-8、包含 NUL 字节的二进制样本(`FS_NOT_TEXT`)以及非普通文件目标。`read` 工具(`@deepseek-ai/dsh-tool-fs`)按大小决定调用哪个方法,并负责行窗口逻辑。
- **`listDir`**:按稳定的 `name.localeCompare()` 顺序列出一层目录。每个条目携带子项 basename、类型、解析后的子目标(`displayPath` 位于所列目录下,`targetKey` 是 realpath 身份)和低成本 stat 元数据(`version`,普通文件另有 `size`)。它绝不会打开或解码文件内容。缺失目标报告 `FS_NOT_FOUND`,文件/特殊文件目标报告 `FS_NOT_DIRECTORY`,已中止调用报告 `FS_ABORTED`,权限失败报告 `FS_PERMISSION_DENIED`,其他列出或子项元数据 I/O 失败报告 `FS_IO_ERROR`。损坏/消失的子项以无元数据的 `other` 返回,但解析子项时出现权限/I/O 失败会让整个列表以结构化 `FsError` 失败。
- **`writeText`**:原子写入。它会向排他打开的临时文件(`wx`、`0o600`)写入;该文件位于目标旁随机命名的私有暂存目录(`0o700`)内。完成写入和 fsync 后,以 rename 覆盖目标。现有文件的 mode 会保留,新文件默认为 `0o600`;Windows 上的新文件继承目标目录的 DACL,而替换会在写入前把目标 DACL 复制到空临时文件,并通过 `ReplaceFileW` 发布,使原访问策略得以保留(见 [Windows DACL 保留 Agent Note](../../../.agents/notes/implemented/bug-fix/2026-07-19-windows-atomic-write-dacl-preservation.md))。`expected` 防护是可选(OPTIONAL)的:省略时无条件创建或覆盖;`createIfAbsent` 创建缺失目标并拒绝现有目标(`FS_NOT_OBSERVED`);`replaceIfVersion` 只在观察到的版本上替换(目标缺失或版本不匹配均为 `FS_STALE_VERSION`)。
- **`writeText`**:原子写入。它会向排他打开的临时文件(`wx`、`0o600`)写入;该文件位于目标旁随机命名的私有暂存目录(`0o700`)内。完成写入和 fsync 后,以 rename 覆盖目标。现有文件的 mode 会保留,新文件默认为 `0o600`;Windows 上的新文件继承目标目录的 DACL,而替换会在写入前把目标 DACL 复制到空临时文件,并通过 `ReplaceFileW` 发布,使原访问策略得以保留(见 [Windows DACL 保留 Agent Note](../../../.agents/notes/implemented/bug-fix/2026-07-19-windows-atomic-write-dacl-preservation.md))。`expected` 防护是可选(OPTIONAL)的:省略时无条件创建或覆盖;`createIfAbsent` 创建缺失目标并拒绝现有目标(`FS_NOT_OBSERVED`);`replaceIfVersion` 只在观察到的版本上替换(目标缺失或版本不匹配均为 `FS_STALE_VERSION`)。仅当打开后的旧文件和 UTF-8 替换内容都严格低于 `config.diffBasisMaxBytes`(默认 10 MiB)时,覆写才返回旧文本作为上下文 diff 基础。即使外部写入方在初次探测后替换文件或改变文件大小,文件描述符读取仍会强制执行该上限;否则提供方返回 `before: null`,由展示层使用整文件回退。
- **`editText`**:在同一原语之上执行原子式的字面量读取-修改-写入,并通过变更锁按目标串行化。`expected` 防护是可选(OPTIONAL)的:提供时,会在字面量匹配之前校验版本(陈旧编辑报告 `FS_STALE_VERSION`,绝不会针对较新内容报告 `FS_EDIT_NOT_FOUND`/`FS_AMBIGUOUS_EDIT`);省略时,无条件编辑当前内容。无论哪种情况,目标缺失都报告 `FS_STALE_VERSION`。匹配时规范化为 LF,随后恢复文件主要的 CRLF/LF 风格;空 `oldString` / 零匹配报告 `FS_EDIT_NOT_FOUND`,未设置 `replace_all` 的多个匹配则报告 `FS_AMBIGUOUS_EDIT`。
包根目录的 SDK 接口包含默认/具名 `LocalFileSystem` 类和 `Config`。原始 I/O 位于 `src/fsio.ts`(不依赖 Cordis,单独进行单元测试);`src/index.ts` 是轻量服务接线。
@@ -34,8 +34,8 @@ await ctx.plugin(LocalFileSystem, { cwd: process.cwd() })
## 已知限制与暂缓事项
- **`config.cwd` 不是沙箱**:它是解析默认值,而非约束;绝对路径和 `..` 可以逃逸。请使用更严格的 `ctx.fs` 后端或 `tools/execute` waterfall(瀑布式事件)上的权限插件实施约束(见[能力 seam Agent Note](../../../.agents/notes/implemented/architecture/2026-06-17-filesystem-capability-seam.md#consequences))。
- **覆盖会把整个旧文件读入内存**:只用于 UI diff;在大小阈值之上限制这次预读取的工作延期处理(`TODO(overwrite-diff-bound)`)。
- **版本 token 是 `mtimeMs:size`**:如果外部变更在文件系统时间戳粒度内保持两者不变,就能绕过陈旧防护。
- **`editText` 会把整个文件及编辑后的副本保存在内存中**:只有读取路径支持流式处理。
- **低于上限的覆写仍会缓冲上下文基础**:`writeText` 除调用方持有的替换内容外,最多还会保留略低于 `config.diffBasisMaxBytes` 的旧文本;该上限不限制返回的 `after` 值,也不限制展示层的整文件回退。
- **二进制检测不对称**:读取只对前 8192 字节执行 NUL 采样,编辑则扫描整个 buffer,因此 NUL 出现在后部的文件可以读取,但编辑会被拒绝。
- **每目标变更锁仅限进程内**:其他进程中的写入方只会被可选版本防护发现,绝不会被串行化。
+47 -10
View File
@@ -15,6 +15,8 @@ import { FsError, FsTargetKey, FsVersion } from '@deepseek-ai/dsh-fs'
import { copyFileDaclWin32, replaceFileWin32 } from './win32.ts'
const BINARY_SAMPLE_BYTES = 8192
// Bound one non-abortable FileHandle.read so cancellation is observed between chunks.
const DIFF_BASIS_READ_CHUNK_BYTES = 64 * 1024
function isENOENT(error: unknown): boolean {
return error instanceof Error && 'code' in error && error.code === 'ENOENT'
@@ -70,9 +72,8 @@ function versionOf(info: BigIntStats): FsVersion {
}
/**
* Test seam: lets specs pin the atomic-write temp names (to prove
* exclusive-open behavior without a name race) and observe the staged temp
* file before it is renamed over the target.
* Test seam: lets specs pin the atomic-write temp names (to prove exclusive-open behavior without
* a name race), override native boundaries, and observe the staged temp file before publication.
*/
export interface FsIoInternals {
/** Override the host platform for native-publication unit coverage. */
@@ -564,17 +565,53 @@ export async function readForEdit(
}
/**
* Best-effort overwrite diff basis. Binary or invalid UTF-8 returns `null` so the write still
* succeeds and presentation falls back to a whole-file diff.
* Best-effort overwrite diff basis. Binary, invalid UTF-8, or a file at/above the byte limit
* returns `null` so the write still succeeds and presentation falls back to a whole-file diff.
* The bound is enforced on the opened descriptor rather than a prior path stat, so concurrent
* external replacement or size changes cannot make this helper buffer more than `maxBytes`.
* @param absolutePath - the file to read (typically a target key); it must exist.
* @param maxBytes - exclusive upper bound for bytes held as the contextual-diff basis.
* @param signal - aborts the read (`FS_ABORTED`).
* @returns the LF-normalized text, or null for a binary or non-UTF-8 file.
* @returns the LF-normalized text, or null for a non-regular, at/above-limit, binary, non-UTF-8,
* or descriptor-size-changed file.
*/
export async function readTextForDiff(absolutePath: string, signal?: AbortSignal): Promise<string | null> {
const buffer = await readFileAbortable(absolutePath, 'read', signal)
if (buffer.includes(0)) return null
export async function readTextForDiff(
absolutePath: string,
maxBytes: number,
signal?: AbortSignal,
): Promise<string | null> {
throwIfAborted(signal, 'read')
const handle = await open(absolutePath, 'r')
let buffer: Buffer
let total = 0
let openedSize = 0
try {
return normalizeLineEndings(new TextDecoder('utf-8', { fatal: true }).decode(buffer))
throwIfAborted(signal, 'read')
const info = await handle.stat()
throwIfAborted(signal, 'read')
/* v8 ignore next -- requires a post-preflight replacement with a non-file;
* direct coverage is not portable to Windows. */
if (!info.isFile()) return null
if (info.size >= maxBytes) return null
openedSize = info.size
// One extra byte detects growth after stat without retaining per-read backing buffers.
buffer = Buffer.allocUnsafe(openedSize + 1)
while (total < buffer.length) {
throwIfAborted(signal, 'read')
const length = Math.min(buffer.length - total, DIFF_BASIS_READ_CHUNK_BYTES)
const { bytesRead } = await handle.read(buffer, total, length, null)
if (bytesRead === 0) break
total += bytesRead
}
} finally {
await handle.close()
}
throwIfAborted(signal, 'read')
if (total !== openedSize) return null
const basis = buffer.subarray(0, total)
if (basis.includes(0)) return null
try {
return normalizeLineEndings(new TextDecoder('utf-8', { fatal: true }).decode(basis))
} catch (error: unknown) {
/* v8 ignore next 2 -- TextDecoder({fatal}) only throws TypeError on invalid bytes; any other throw is an unreachable runtime fault. */
if (!(error instanceof TypeError)) throw error
+30 -5
View File
@@ -5,6 +5,7 @@
*/
import { Context } from 'cordis'
import { constants as bufferConstants } from 'node:buffer'
import { resolve } from 'node:path'
import z from 'schemastery'
import { FileSystem, FsError, FsVersion } from '@deepseek-ai/dsh-fs'
@@ -38,9 +39,19 @@ import type { FsIoInternals } from './fsio.ts'
export interface Config {
/** Base directory for relative paths. Defaults to `process.cwd()`. */
cwd?: string
/**
* Exclusive UTF-8 byte limit on each overwrite-diff side, capped by the
* runtime's safe allocation/decode maximum. Defaults to 10 MiB.
*/
diffBasisMaxBytes?: number
}
type ResolvedConfig = Required<Config>
const DEFAULT_DIFF_BASIS_MAX_BYTES = 10 * 1024 * 1024
const MAX_DIFF_BASIS_BYTES = Math.min(
bufferConstants.MAX_LENGTH,
bufferConstants.MAX_STRING_LENGTH,
)
/**
* The host-filesystem backend. Reads resolve relative paths from {@link Config.cwd}
@@ -51,11 +62,12 @@ type ResolvedConfig = Required<Config>
export class LocalFileSystem extends FileSystem {
static Config: z<Config> = z.object({
cwd: z.string().default(process.cwd()),
diffBasisMaxBytes: z.number().default(DEFAULT_DIFF_BASIS_MAX_BYTES),
})
/** Validated config (schemastery applied the defaults before construction). */
readonly config: ResolvedConfig
/** Test seam forwarded to fsio (force streaming path, pin temp names). */
/** Test seam forwarded to fsio for atomic-publication boundaries. */
internals: FsIoInternals = {}
/** Per-targetKey tail promise: serializes mutating ops so the read→guard→write
* window can't interleave, making concurrent writes/edits deterministically
@@ -64,7 +76,13 @@ export class LocalFileSystem extends FileSystem {
constructor(ctx: Context, config: Config) {
super(ctx)
this.config = config as ResolvedConfig
const resolved = config as ResolvedConfig
if (!Number.isSafeInteger(resolved.diffBasisMaxBytes)
|| resolved.diffBasisMaxBytes <= 0
|| resolved.diffBasisMaxBytes > MAX_DIFF_BASIS_BYTES) {
throw new Error(`fs-local: diffBasisMaxBytes must be a positive safe integer no greater than ${MAX_DIFF_BASIS_BYTES}`)
}
this.config = resolved
}
/** Run `op` with exclusive access to `targetKey` (FIFO per key). */
@@ -150,9 +168,16 @@ export class LocalFileSystem extends FileSystem {
}
// No expectation means an unconditional but still atomic write.
// Preserve prior text for contextual diffs; null falls back to a whole-file diff.
// TODO(overwrite-diff-bound): cap this UI-only pre-read for large files.
const before = existing ? await readTextForDiff(target.targetKey, signal) : null
// Capture an optional contextual-diff basis before the write. The bounded
// reader checks the opened file itself, so an external replacement after
// `probe()` cannot turn this best-effort presentation read into an
// unbounded allocation. Either side at/above the configured limit yields
// `before: null`; consumers retain their whole-file fallback.
const diffable = existing !== null
&& Buffer.byteLength(content, 'utf8') < this.config.diffBasisMaxBytes
const before = diffable
? await readTextForDiff(target.targetKey, this.config.diffBasisMaxBytes, signal)
: null
await writeFileAtomic(target.targetKey, content, existing?.mode, signal, this.internals)
const after = await probe(target.targetKey)
return {
@@ -7,6 +7,7 @@
*/
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
import { constants as bufferConstants } from 'node:buffer'
import { mkdir, mkdtemp, readFile, realpath, rm, stat, symlink, unlink, utimes, writeFile } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
@@ -42,13 +43,39 @@ async function versionOf(target: FsTarget): Promise<FsVersion> {
return info.version
}
async function remountWithDiffLimit(diffBasisMaxBytes: number): Promise<void> {
await fiber.dispose()
fiber = await ctx.plugin(LocalFileSystem, { cwd: dir, diffBasisMaxBytes })
fs = ctx.fs as LocalFileSystem
}
describe('registration', () => {
it('registers LocalFileSystem as ctx.fs with a default cwd', async () => {
const bare = new Context()
const bareFiber = await bare.plugin(LocalFileSystem)
expect((bare.fs as LocalFileSystem).config.cwd).toBe(process.cwd())
expect((bare.fs as LocalFileSystem).config.diffBasisMaxBytes).toBe(10 * 1024 * 1024)
await bareFiber.dispose()
})
it('rejects non-positive, fractional, unsafe, or unallocatable diff-basis limits', async () => {
const maxDiffBasisBytes = Math.min(
bufferConstants.MAX_LENGTH,
bufferConstants.MAX_STRING_LENGTH,
)
const valid = new Context()
const validFiber = await valid.plugin(LocalFileSystem, { diffBasisMaxBytes: maxDiffBasisBytes })
expect((valid.fs as LocalFileSystem).config.diffBasisMaxBytes).toBe(maxDiffBasisBytes)
await validFiber.dispose()
for (const diffBasisMaxBytes of [0, -1, 1.5, maxDiffBasisBytes + 1, Number.MAX_SAFE_INTEGER + 1]) {
const invalid = new Context()
await expect(invalid.plugin(LocalFileSystem, { diffBasisMaxBytes })).rejects.toThrow(
`fs-local: diffBasisMaxBytes must be a positive safe integer no greater than ${maxDiffBasisBytes}`,
)
await invalid.fiber.dispose()
}
})
})
describe('resolve', () => {
@@ -384,6 +411,42 @@ describe('writeText', () => {
expect(outcome.after).toBe('now valid')
})
it('an overwrite of a prior file AT the whole-file bound reports before:null (undiffable), still succeeds', async () => {
// The configured bound keeps the fixture small; 8 bytes at a bound of 8
// pins the exclusive edge without coupling this provider to a read tool.
await remountWithDiffLimit(8)
await writeFile(join(dir, 'big.txt'), '12345678')
const target = await fs.resolve('big.txt')
const outcome = await fs.writeText(target, 'tiny')
expect(outcome.operation).toBe('update')
expect(outcome.before).toBeNull()
expect(outcome.after).toBe('tiny')
})
it('an overwrite whose NEW content is at the whole-file bound reports before:null (no huge contextual diff)', async () => {
// The bound gates BOTH sides of the diff pair: a small prior file rewritten
// with at/above-bound content yields no contextual-hunk basis either, since
// a small-to-huge rewrite's hunk is as large as the new content — the
// consumer must fall back to the whole-file diff card, exactly like a
// create of the same size.
await remountWithDiffLimit(8)
await writeFile(join(dir, 'grow.txt'), 'tiny')
const target = await fs.resolve('grow.txt')
const outcome = await fs.writeText(target, '12345678')
expect(outcome.operation).toBe('update')
expect(outcome.before).toBeNull()
expect(outcome.after).toBe('12345678')
})
it('an overwrite with BOTH sides below the whole-file bound keeps its contextual before basis', async () => {
await remountWithDiffLimit(8)
await writeFile(join(dir, 'small.txt'), '1234567')
const target = await fs.resolve('small.txt')
const outcome = await fs.writeText(target, 'new')
expect(outcome.before).toBe('1234567')
expect(outcome.after).toBe('new')
})
it('releases per-target mutation locks after success and failure', async () => {
const target = await fs.resolve('a.txt')
await fs.writeText(target, 'created', { kind: 'createIfAbsent' })
+201 -1
View File
@@ -5,7 +5,7 @@
* policy and lives in `dsh-fs-policy`, so it is not tested here.
*/
import { afterEach, beforeEach, describe, expect, it } from 'vitest'
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
import { chmod, mkdtemp, readFile, rename, rm, stat, symlink, unlink, writeFile, mkdir, readdir, realpath } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
@@ -16,6 +16,7 @@ import {
probe,
probeNoFollow,
readForEdit,
readTextForDiff,
readWholeText,
resolveLocalTarget,
restoreLineEndings,
@@ -316,6 +317,205 @@ describe('readWholeText', () => {
})
})
describe('readTextForDiff', () => {
it('returns normalized text only when the opened file is strictly below the limit', async () => {
const file = join(dir, 'basis.txt')
await writeFile(file, 'a\r\nb')
expect(await readTextForDiff(file, 5)).toBe('a\nb')
expect(await readTextForDiff(file, 4)).toBeNull()
})
it('bounds the actual opened file rather than trusting an earlier path size', async () => {
const file = join(dir, 'replaced.txt')
await writeFile(file, 'tiny')
const earlierSize = (await stat(file)).size
await writeFile(file, '123456789')
expect(earlierSize).toBeLessThan(8)
expect(await readTextForDiff(file, 8)).toBeNull()
})
it('returns null when the opened file shrinks after descriptor stat', async () => {
const file = join(dir, 'shrinking.txt')
await writeFile(file, 'abcdef')
vi.resetModules()
vi.doMock('node:fs/promises', async (importOriginal) => {
const actual = await importOriginal<typeof import('node:fs/promises')>()
return {
...actual,
async open(...args: Parameters<typeof actual.open>) {
const handle = await actual.open(...args)
return {
close: handle.close.bind(handle),
read: handle.read.bind(handle),
async stat(...statArgs: Parameters<typeof handle.stat>) {
const info = await handle.stat(...statArgs)
await writeFile(file, 'abc')
return info
},
}
},
}
})
try {
const { readTextForDiff: isolatedReadTextForDiff } = await import('../src/fsio.ts')
expect(await isolatedReadTextForDiff(file, 8)).toBeNull()
} finally {
vi.doUnmock('node:fs/promises')
vi.resetModules()
}
})
it('returns null when the opened file grows after descriptor stat', async () => {
// Pins the one-extra-byte EOF probe: with a buffer of exactly openedSize a
// grown file would read openedSize bytes and pass the consistency check.
const file = join(dir, 'growing.txt')
await writeFile(file, 'abcdef')
vi.resetModules()
vi.doMock('node:fs/promises', async (importOriginal) => {
const actual = await importOriginal<typeof import('node:fs/promises')>()
return {
...actual,
async open(...args: Parameters<typeof actual.open>) {
const handle = await actual.open(...args)
return {
close: handle.close.bind(handle),
read: handle.read.bind(handle),
async stat(...statArgs: Parameters<typeof handle.stat>) {
const info = await handle.stat(...statArgs)
await writeFile(file, 'abcdef-grown')
return info
},
}
},
}
})
try {
const { readTextForDiff: isolatedReadTextForDiff } = await import('../src/fsio.ts')
expect(await isolatedReadTextForDiff(file, 32)).toBeNull()
} finally {
vi.doUnmock('node:fs/promises')
vi.resetModules()
}
})
it('returns null for binary and invalid UTF-8 without blocking the caller write', async () => {
await writeFile(join(dir, 'bin'), Buffer.from([0x68, 0x00, 0x69]))
await writeFile(join(dir, 'bad'), Buffer.from([0x68, 0xff, 0x69]))
expect(await readTextForDiff(join(dir, 'bin'), 8)).toBeNull()
expect(await readTextForDiff(join(dir, 'bad'), 8)).toBeNull()
})
it('honors a pre-aborted signal', async () => {
const file = join(dir, 'basis.txt')
await writeFile(file, 'text')
await expect(readTextForDiff(file, 8, AbortSignal.abort())).rejects.toMatchObject({ code: 'FS_ABORTED' })
})
it.each(['open', 'stat'] as const)('observes cancellation immediately after %s', async (stage) => {
const file = join(dir, 'basis.txt')
await writeFile(file, 'text')
const reached = Promise.withResolvers<undefined>()
const release = Promise.withResolvers<undefined>()
let statCalls = 0
const allocate = vi.spyOn(Buffer, 'allocUnsafe')
vi.resetModules()
vi.doMock('node:fs/promises', async (importOriginal) => {
const actual = await importOriginal<typeof import('node:fs/promises')>()
return {
...actual,
async open(...args: Parameters<typeof actual.open>) {
const handle = await actual.open(...args)
if (stage === 'open') {
reached.resolve(undefined)
await release.promise
}
return {
close: handle.close.bind(handle),
read: handle.read.bind(handle),
async stat(...statArgs: Parameters<typeof handle.stat>) {
statCalls += 1
const info = await handle.stat(...statArgs)
if (stage === 'stat') {
reached.resolve(undefined)
await release.promise
}
return info
},
}
},
}
})
try {
const { readTextForDiff: isolatedReadTextForDiff } = await import('../src/fsio.ts')
const controller = new AbortController()
const pending = isolatedReadTextForDiff(file, 8, controller.signal)
await reached.promise
const allocationCalls = allocate.mock.calls.length
controller.abort()
release.resolve(undefined)
await expect(pending).rejects.toMatchObject({ code: 'FS_ABORTED' })
expect(statCalls).toBe(stage === 'open' ? 0 : 1)
expect(allocate).toHaveBeenCalledTimes(allocationCalls)
} finally {
release.resolve(undefined)
allocate.mockRestore()
vi.doUnmock('node:fs/promises')
vi.resetModules()
}
})
it('bounds descriptor reads and observes cancellation before the next chunk', async () => {
const file = join(dir, 'large-basis.txt')
const fileBytes = 200 * 1024
await writeFile(file, 'x'.repeat(fileBytes))
const firstRead = Promise.withResolvers<undefined>()
const releaseFirstRead = Promise.withResolvers<undefined>()
const readLengths: number[] = []
vi.resetModules()
vi.doMock('node:fs/promises', async (importOriginal) => {
const actual = await importOriginal<typeof import('node:fs/promises')>()
return {
...actual,
async open(...args: Parameters<typeof actual.open>) {
const handle = await actual.open(...args)
return {
stat: handle.stat.bind(handle),
close: handle.close.bind(handle),
async read(buffer: Buffer, offset: number, length: number, position: number | null) {
readLengths.push(length)
const result = await handle.read(buffer, offset, length, position)
if (readLengths.length === 1) {
firstRead.resolve(undefined)
await releaseFirstRead.promise
}
return result
},
}
},
}
})
try {
const { readTextForDiff: isolatedReadTextForDiff } = await import('../src/fsio.ts')
const controller = new AbortController()
const pending = isolatedReadTextForDiff(file, fileBytes + 1, controller.signal)
await firstRead.promise
expect(readLengths).toEqual([64 * 1024])
controller.abort()
releaseFirstRead.resolve(undefined)
await expect(pending).rejects.toMatchObject({ code: 'FS_ABORTED' })
expect(readLengths).toHaveLength(1)
} finally {
releaseFirstRead.resolve(undefined)
vi.doUnmock('node:fs/promises')
vi.resetModules()
}
})
})
describe('streamWholeText', () => {
it('streams the whole file as decoded text', async () => {
const file = join(dir, 'a.txt')
+2 -2
View File
@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write packages/fs/fs-sandbox/README.md
README.md: c40fc7999ab85a70702f65a5675208163f5fc351
README.zh.md: 15db5abbfc5307c0570925026ec435d8dbb51bf2
README.md: ae1fd746c711a86e308a02e0054ba478e8d913c0
README.zh.md: e25a3467c06fbd93de9bb75d6b364e8da5a451fe
+2
View File
@@ -4,6 +4,8 @@ English | [中文](README.zh.md)
`SandboxedFileSystem` extends [`LocalFileSystem`](../fs-local/README.md) and registers as `ctx.fs`. It inherits every text-storage mechanic verbatim (resolve, stat, read/stream, list, the atomic write, the read-match-write edit critical section) and adds only a per-call MODE fence on `writeText`/`editText`. Reads always pass through — every mode permits reading.
Its plugin config is the local backend config unchanged: `cwd` remains the relative-path resolution default, and `diffBasisMaxBytes` bounds the optional overwrite contextual-diff basis.
Loading it INSTEAD OF `dsh-fs-local`, together with a [`ctx.sandboxPolicy`](../../sandbox/sandbox-policy/README.md), is the whole swap; the model-facing tools (`dsh-tool-fs`) are untouched. The tool layer resolves the calling session's mode and cwd into the SAME per-call policy bash receives, so the two families never confine to different roots.
## The fence
+2
View File
@@ -4,6 +4,8 @@
`SandboxedFileSystem` 扩展 [`LocalFileSystem`](../fs-local/README.md) 并注册为 `ctx.fs`。它逐字继承全部文本存储机制(解析、stat、读取/流式读取、列出、原子写入、按读取、匹配、写入顺序执行的编辑临界区),只为 `writeText`/`editText` 增加按调用的模式围栏。读取始终直接通过:所有模式都允许读取。
它原样复用本地后端配置:`cwd` 仍是相对路径的解析默认值,`diffBasisMaxBytes` 则限制可选的覆写上下文 diff 基础。
只需加载它来替代 `dsh-fs-local`,并同时加载 [`ctx.sandboxPolicy`](../../sandbox/sandbox-policy/README.md),即可完成替换;面向模型的工具(`dsh-tool-fs`)无需改动。工具层把调用会话的模式和 cwd 解析为与 bash 相同的按调用策略,因此两个能力族绝不会约束到不同根目录。
## 围栏
+4 -4
View File
@@ -41,10 +41,10 @@ import type {} from '@deepseek-ai/dsh-sandbox-policy'
import { isPathUnder } from './containment.ts'
/**
* Plugin config: the local backend's knobs, verbatim (only `cwd`, the resolve
* base for relative paths). The sandbox default (mode + `workspace-write`
* fallback root) is NOT here — `ctx.sandboxPolicy` resolves each calling
* session for every enforcing capability.
* Plugin config: the local backend's knobs verbatim (`cwd` resolution default
* and `diffBasisMaxBytes` overwrite-presentation bound). The sandbox default
* (mode + `workspace-write` fallback root) is NOT here — `ctx.sandboxPolicy`
* resolves each calling session for every enforcing capability.
*/
export type Config = LocalConfig
+5 -4
View File
@@ -123,10 +123,11 @@ export interface FsWriteOutcome {
version: FsVersion
/**
* The file's content BEFORE the write, or `null` when the file did not exist
* (a create) or was undiffable (binary/non-UTF-8). LF-normalized storage text
* (the diff basis), never a diff — a consumer computes the result-time
* contextual diff from `before`/`after` when `before` is present, else falls
* back to a whole-file diff.
* (a create) or the backend declined a contextual basis (for example, a
* binary/non-UTF-8 prior file or either overwrite side reaching its exclusive limit).
* LF-normalized storage text (the diff basis), never a diff — a consumer
* computes the result-time contextual diff from `before`/`after` when
* `before` is present, else falls back to a whole-file diff.
*/
before: string | null
/** The file's content AFTER the write, LF-normalized to share `before`'s diff basis. */