fix(fs): harden guarded-create publication

This commit is contained in:
Tianyi Cui
2026-08-09 17:40:41 +08:00
parent ceba53edd7
commit 7132b9730e
15 changed files with 207 additions and 40 deletions
+2 -2
View File
@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write packages/fs/fs-local/README.md
README.md: ac2f271ad9651797e1aceeba49e7a52533405f10
README.zh.md: 633d55fe06b2261d9d5b52184987811c87c8f155
README.md: 7b993fa123d13833313ecf3f78c64e466b960d5d
README.zh.md: 428719137f988395b76513eab2c3f76ce4f331e5
+3 -1
View File
@@ -19,7 +19,7 @@ await ctx.plugin(LocalFileSystem, { cwd: process.cwd() })
- **`stat` / `lstat`** — return target metadata or `undefined` when absent. `stat` reports `FsInfo` for an already resolved target (`version` = an opaque token derived from bigint `dev:ino:size:mtimeNs:ctimeNs`, `type` of `file`/`directory`/`other`, byte `size`); path-shaped `lstat` reports `FsPathInfo` without following the final symlink and can therefore return `symlink`. Both check cancellation before and after their asynchronous metadata probe, so an abort that lands in flight reports `FS_ABORTED` rather than stale absence.
- **`readText` / `streamText`** — UTF-8 only. `readText` reads the whole file; `streamText` decodes chunks so a huge file need not be held whole in memory and consumers can enforce their own retention bounds. Both reject invalid UTF-8 and NUL-byte binary samples (`FS_NOT_TEXT`) and non-regular targets. The `read` tool (`@deepseek-ai/dsh-tool-fs`) owns line windowing.
- **`listDir`** — lists one directory level in stable `name.localeCompare()` order. Each entry carries the child basename, type, resolved child target (`displayPath` under the listed directory, `targetKey` as the realpath identity), and cheap stat metadata (`version`, plus `size` for regular files). It never opens or decodes file contents. Missing targets report `FS_NOT_FOUND`, file/special-file targets report `FS_NOT_DIRECTORY`, aborted calls report `FS_ABORTED`, permission failures report `FS_PERMISSION_DENIED`, and other listing or child metadata I/O failures report `FS_IO_ERROR`. Broken/disappeared children are returned as `other` without metadata, but permission/IO failures while resolving a child fail the whole listing with a structured `FsError`.
- **`writeText`** — atomic: writes to a temp file opened exclusively (`wx`, `0o600`) inside a randomly-named private staging dir (`0o700`) next to the target, then fsyncs and publishes. An existing file's mode is preserved, while new files default to `0o600`; on Windows a new file inherits the destination directory's DACL, while replacement copies the target DACL onto the empty temp before writing and publishes through `ReplaceFileW` so the original access policy survives ([Windows DACL preservation Agent Note](../../../.agents/notes/implemented/bug-fix/2026-07-19-windows-atomic-write-dacl-preservation.md)). The `expected` guard is OPTIONAL: omitting it unconditionally creates-or-overwrites; `createIfAbsent` hard-links the staged file into place as an atomic no-replace publication, so a file created after the initial probe is preserved and rejected with `FS_NOT_OBSERVED`; `replaceIfVersion` replaces only at the observed version (a missing target or mismatch is `FS_STALE_VERSION`).
- **`writeText`** — atomic: writes to a temp file opened exclusively (`wx`, `0o600`) inside a randomly-named private staging dir (`0o700`) next to the target, then fsyncs and publishes. An existing file's mode is preserved, while new files default to `0o600`; on Windows a new file inherits the destination directory's DACL, while replacement copies the target DACL onto the empty temp before writing and publishes through `ReplaceFileW` so the original access policy survives ([Windows DACL preservation Agent Note](../../../.agents/notes/implemented/bug-fix/2026-07-19-windows-atomic-write-dacl-preservation.md)). The `expected` guard is OPTIONAL: omitting it unconditionally creates-or-overwrites; `createIfAbsent` hard-links the staged file into place as an atomic no-replace publication, so a regular file created after the initial probe is preserved and rejected with `FS_NOT_OBSERVED`, while a non-regular path entry is preserved and rejected with `FS_NOT_REGULAR_FILE`; `replaceIfVersion` replaces only at the observed version (a missing target or mismatch is `FS_STALE_VERSION`).
- **`editText`** — atomic literal read-modify-write over the same primitive, serialized per target by a mutation lock. The `expected` guard is OPTIONAL: when supplied it verifies the version BEFORE literal matching (a stale edit reports `FS_STALE_VERSION`, never `FS_EDIT_NOT_FOUND`/`FS_AMBIGUOUS_EDIT` against newer content); omitting it edits the current content unconditionally. A missing target reports `FS_STALE_VERSION` either way. LF-normalizes for matching, restores the file's dominant CRLF/LF style, and rejects empty `oldString` / zero matches (`FS_EDIT_NOT_FOUND`) or ambiguous multi-matches without `replace_all` (`FS_AMBIGUOUS_EDIT`).
The package-root SDK surface is the default/named `LocalFileSystem` class plus `Config`. Raw I/O lives in `src/fsio.ts` (Cordis-free, independently unit-tested); `src/index.ts` is the thin service wiring.
@@ -40,3 +40,5 @@ No direct invalidation; the named consumer owns any request-prefix changes.
- **`editText` holds the whole file (plus the edited copy) in memory** — streaming exists only on the read path.
- **Binary detection is asymmetric** — reads NUL-sample only the first 8192 bytes while edits scan the whole buffer, so a file with a late NUL reads fine but rejects edits.
- **The per-target mutation lock is in-process only** — guarded create still uses an atomic no-replace publication across processes, but replacement writers in another process are caught only when the optional version guard observes their metadata change; they are never serialized.
- **Guarded creation requires hard-link support** — filesystems or mounts that reject hard-link publication cannot serve `createIfAbsent`; the provider preserves the missing target and reports `FS_IO_ERROR`.
- **Post-commit cleanup is best effort** — a successful publication remains successful if removal of its owner-only staging directory fails, leaving private residue for later operator cleanup.
+3 -1
View File
@@ -19,7 +19,7 @@ await ctx.plugin(LocalFileSystem, { cwd: process.cwd() })
- **`stat` / `lstat`**:返回目标元数据;目标不存在时返回 `undefined`。`stat` 为已解析目标报告 `FsInfo`(`version` 是由 bigint `dev:ino:size:mtimeNs:ctimeNs` 派生的不透明 token,`type` 为 `file`/`directory`/`other`,`size` 以字节计);路径形态的 `lstat` 不跟随最后一个符号链接,报告 `FsPathInfo`,因此可以返回 `symlink`。两者都会在异步元数据探测前后检查取消,因此飞行中的中止会报告 `FS_ABORTED`,而非陈旧的不存在结果。
- **`readText` / `streamText`**:只支持 UTF-8。`readText` 读取整个文件;`streamText` 按分片解码,因此超大文件无需整体保存在内存中,消费方也可以执行各自的保留上限。两者都会拒绝无效 UTF-8、包含 NUL 字节的二进制样本(`FS_NOT_TEXT`)以及非普通文件目标。`read` 工具(`@deepseek-ai/dsh-tool-fs`)拥有行窗口逻辑。
- **`listDir`**:按稳定的 `name.localeCompare()` 顺序列出一层目录。每个条目携带子项 basename、类型、解析后的子目标(`displayPath` 位于所列目录下,`targetKey` 是 realpath 身份)和低成本 stat 元数据(`version`,普通文件另有 `size`)。它绝不会打开或解码文件内容。缺失目标报告 `FS_NOT_FOUND`,文件/特殊文件目标报告 `FS_NOT_DIRECTORY`,已中止调用报告 `FS_ABORTED`,权限失败报告 `FS_PERMISSION_DENIED`,其他列出或子项元数据 I/O 失败报告 `FS_IO_ERROR`。损坏/消失的子项以无元数据的 `other` 返回,但解析子项时出现权限/I/O 失败会让整个列表以结构化 `FsError` 失败。
- **`writeText`**:原子写入。它会向排他打开的临时文件(`wx`、`0o600`)写入;该文件位于目标旁随机命名的私有暂存目录(`0o700`)内,随后执行 fsync 并发布。现有文件的 mode 会保留,新文件默认为 `0o600`;Windows 上的新文件继承目标目录的 DACL,而替换会在写入前把目标 DACL 复制到空临时文件,并通过 `ReplaceFileW` 发布,使原访问政策得以保留(见 [Windows DACL 保留 Agent Note](../../../.agents/notes/implemented/bug-fix/2026-07-19-windows-atomic-write-dacl-preservation.md))。`expected` 防护是可选的:省略时无条件创建或覆盖;`createIfAbsent` 通过硬链接把暂存文件发布到目标位置,以实现原子且不替换的发布,因此初始探测后创建的文件会被保留,并以 `FS_NOT_OBSERVED` 拒绝本次写入;`replaceIfVersion` 只在观察到的版本上替换(目标缺失或版本不匹配均为 `FS_STALE_VERSION`)。
- **`writeText`**:原子写入。它会向排他打开的临时文件(`wx`、`0o600`)写入;该文件位于目标旁随机命名的私有暂存目录(`0o700`)内,随后执行 fsync 并发布。现有文件的 mode 会保留,新文件默认为 `0o600`;Windows 上的新文件继承目标目录的 DACL,而替换会在写入前把目标 DACL 复制到空临时文件,并通过 `ReplaceFileW` 发布,使原访问政策得以保留(见 [Windows DACL 保留 Agent Note](../../../.agents/notes/implemented/bug-fix/2026-07-19-windows-atomic-write-dacl-preservation.md))。`expected` 防护是可选的:省略时无条件创建或覆盖;`createIfAbsent` 通过硬链接把暂存文件发布到目标位置,以实现原子且不替换的发布,因此初始探测后创建的普通文件会被保留,并以 `FS_NOT_OBSERVED` 拒绝本次写入;非普通路径条目也会被保留,并以 `FS_NOT_REGULAR_FILE` 拒绝;`replaceIfVersion` 只在观察到的版本上替换(目标缺失或版本不匹配均为 `FS_STALE_VERSION`)。
- **`editText`**:在同一原语之上依次执行原子的字面量读取、修改和写入,并通过变更锁按目标串行化。`expected` 防护是可选的:提供时,会在字面量匹配之前校验版本(陈旧编辑报告 `FS_STALE_VERSION`,绝不会针对较新内容报告 `FS_EDIT_NOT_FOUND`/`FS_AMBIGUOUS_EDIT`);省略时,无条件编辑当前内容。无论哪种情况,目标缺失都报告 `FS_STALE_VERSION`。匹配时规范化为 LF,随后恢复文件主要的 CRLF/LF 风格;空 `oldString` / 零匹配报告 `FS_EDIT_NOT_FOUND`,未设置 `replace_all` 的多个匹配则报告 `FS_AMBIGUOUS_EDIT`。
包根 SDK 接口包含默认/具名 `LocalFileSystem` 类和 `Config`。原始 I/O 位于 `src/fsio.ts`(不依赖 Cordis,单独进行单元测试);`src/index.ts` 是轻量服务接线。
@@ -40,3 +40,5 @@ await ctx.plugin(LocalFileSystem, { cwd: process.cwd() })
- **`editText` 会把整个文件及编辑后的副本保存在内存中**:只有读取路径支持流式处理。
- **二进制检测不对称**:读取只对前 8192 字节执行 NUL 采样,编辑则扫描整个 buffer,因此 NUL 出现在后部的文件可以读取,但编辑会被拒绝。
- **每目标变更锁仅限进程内**:即使跨进程,带防护的创建仍采用原子且不替换的发布方式;但只有当可选版本防护观察到元数据变化时,系统才能发现其他进程中的替换写入方,且绝不会将其串行化。
- **带防护的创建要求支持硬链接**:拒绝硬链接发布的文件系统或挂载点无法支持 `createIfAbsent`;提供方会使目标保持缺失状态并报告 `FS_IO_ERROR`。
- **提交后清理采用尽力而为语义**:如果移除仅所有者可访问的暂存目录失败,成功发布仍视为成功,并留下私有残留供运维人员后续清理。
+63 -14
View File
@@ -91,6 +91,10 @@ export interface FsIoInternals {
replaceFile?: (replaced: string, replacement: string) => Promise<void>
/** Override the hard-link no-replace publication boundary. */
linkFile?: (existingPath: string, newPath: string) => Promise<void>
/** Override target inspection after guarded publication fails. */
inspectPublicationTarget?: (path: string) => Promise<BigIntStats>
/** Override staging-directory removal for commit-point failure coverage. */
removeStagingDir?: (stagingDir: string) => Promise<void>
/** Test hook after the temp file is written/synced but before final chmod+publication. */
inspectTemp?: (paths: { stagingDir: string; tempPath: string }) => void | Promise<void>
}
@@ -413,9 +417,13 @@ export async function* streamWholeText(target: LocalTarget, signal?: AbortSignal
// --- Writing ---
async function removeStagingDirOrThrow(stagingDir: string, originalError: unknown): Promise<never> {
async function removeStagingDirOrThrow(
stagingDir: string,
originalError: unknown,
removeStagingDir: (path: string) => Promise<void>,
): Promise<never> {
try {
await rm(stagingDir, { recursive: true, force: true })
await removeStagingDir(stagingDir)
} catch (cleanupError: unknown) {
/* v8 ignore next 1 -- cleanup failure here needs a second filesystem fault after the primary write failure. */
throw new FsError(`write failed (${errorMessage(originalError)}) and temp cleanup failed (${errorMessage(cleanupError)})`, 'FS_NOT_FOUND', { cause: originalError })
@@ -423,6 +431,43 @@ async function removeStagingDirOrThrow(stagingDir: string, originalError: unknow
throw originalError
}
async function throwGuardedCreateFailure(
error: unknown,
absolutePath: string,
displayPath: string,
inspectPublicationTarget: (path: string) => Promise<BigIntStats>,
): Promise<never> {
let existing: BigIntStats | undefined
try {
existing = await inspectPublicationTarget(absolutePath)
} catch (metadataError: unknown) {
if (!isENOENT(metadataError) && !isENOTDIR(metadataError)) {
throw new FsError(`cannot write "${displayPath}": ${errorMessage(metadataError)}`, 'FS_IO_ERROR', { cause: metadataError })
}
}
// Link errno values vary by platform and filesystem. Inspect the target entry
// after failure so a collision is not confused with missing hard-link support.
if (existing !== undefined) {
if (!existing.isFile()) {
throw new FsError(`cannot write "${displayPath}": not a regular file`, 'FS_NOT_REGULAR_FILE', { cause: error })
}
throw new FsError(
`cannot overwrite existing "${displayPath}" without reading it first`,
'FS_NOT_OBSERVED',
{ cause: error },
)
}
if (isEEXIST(error)) {
throw new FsError(
`cannot overwrite existing "${displayPath}" without reading it first`,
'FS_NOT_OBSERVED',
{ cause: error },
)
}
throw new FsError(`cannot write "${displayPath}": ${errorMessage(error)}`, 'FS_IO_ERROR', { cause: error })
}
/**
* Atomically replace a file through a private, synced staging file in the same directory.
* POSIX protects the staging directory and file with `0o700` and `0o600`. A new Windows file
@@ -434,8 +479,9 @@ async function removeStagingDirOrThrow(stagingDir: string, originalError: unknow
* inert as a mode on Windows but identifies replacement security semantics.
* @param signal - cancellation checked before final publication.
* @param internals - Test hook for pinning temp names and observing the staged file.
* @param createIfAbsent - publish with a hard-link no-replace primitive; a
* concurrent creator is preserved and rejected with `FS_NOT_OBSERVED`.
* @param createIfAbsent - when provided, publish with a hard-link no-replace
* primitive; a concurrent creator's file is preserved and this write is
* rejected with `FS_NOT_OBSERVED` using the supplied display path.
*/
export async function writeFileAtomic(
absolutePath: string,
@@ -443,7 +489,7 @@ export async function writeFileAtomic(
mode: number | undefined,
signal: AbortSignal | undefined,
internals: FsIoInternals = {},
createIfAbsent = false,
createIfAbsent?: { displayPath: string },
): Promise<void> {
throwIfAborted(signal, 'write')
const directory = dirname(absolutePath)
@@ -458,6 +504,10 @@ export async function writeFileAtomic(
const copyFileDacl = internals.copyFileDacl ?? copyFileDaclWin32
const replaceFile = internals.replaceFile ?? replaceFileWin32
const linkFile = internals.linkFile ?? link
const inspectPublicationTarget = internals.inspectPublicationTarget
?? (path => lstat(path, { bigint: true }))
const removeStagingDir = internals.removeStagingDir
?? (path => rm(path, { recursive: true, force: true }))
let handle: Awaited<ReturnType<typeof open>> | undefined
let stagingCreated = false
try {
@@ -478,16 +528,11 @@ export async function writeFileAtomic(
handle = undefined
throwIfAborted(signal, 'write')
if (createIfAbsent) {
if (createIfAbsent !== undefined) {
try {
await linkFile(tempPath, absolutePath)
} catch (error: unknown) {
if (!isEEXIST(error)) throw error
throw new FsError(
`cannot overwrite existing "${absolutePath}" without reading it first`,
'FS_NOT_OBSERVED',
{ cause: error },
)
await throwGuardedCreateFailure(error, absolutePath, createIfAbsent.displayPath, inspectPublicationTarget)
}
} else if (platform === 'win32' && mode !== undefined) {
try {
@@ -501,7 +546,11 @@ export async function writeFileAtomic(
} else {
await rename(tempPath, absolutePath)
}
await rm(stagingDir, { recursive: true, force: true })
try {
await removeStagingDir(stagingDir)
} catch (_committedStagingCleanupFailure) {
// The target is committed; owner-only staging residue cannot turn that write into a failure.
}
} catch (error: unknown) {
/* v8 ignore next -- abort-mid-write needs a writeFile/signal race; the non-abort (rename/open) side is tested. */
let failure: unknown = isAbortError(error) ? new FsError('write aborted', 'FS_ABORTED') : error
@@ -514,7 +563,7 @@ export async function writeFileAtomic(
}
}
if (!stagingCreated) throw failure
return removeStagingDirOrThrow(stagingDir, failure)
return removeStagingDirOrThrow(stagingDir, failure, removeStagingDir)
}
}
+1 -1
View File
@@ -173,7 +173,7 @@ export class LocalFileSystem extends FileSystem {
existing?.mode,
signal,
this.internals,
expected?.kind === 'createIfAbsent',
expected?.kind === 'createIfAbsent' ? { displayPath: target.displayPath } : undefined,
)
const after = await probe(target.targetKey)
return {
@@ -308,6 +308,41 @@ describe('writeText', () => {
expect(await readFile(path, 'utf8')).toBe('competitor')
})
it('reports a createIfAbsent race with the unresolved display path', async () => {
const realDirectory = join(dir, 'real-workspace')
const linkedDirectory = join(dir, 'linked-workspace')
await mkdir(realDirectory)
await symlink(realDirectory, linkedDirectory, process.platform === 'win32' ? 'junction' : 'dir')
const target = await fs.resolve('linked-workspace/a.txt')
fs.internals.inspectTemp = async () => { await writeFile(join(realDirectory, 'a.txt'), 'competitor') }
await expect(fs.writeText(target, 'ours', { kind: 'createIfAbsent' })).rejects.toMatchObject({
code: 'FS_NOT_OBSERVED',
message: `cannot overwrite existing "${join(linkedDirectory, 'a.txt')}" without reading it first`,
})
expect(await readFile(join(realDirectory, 'a.txt'), 'utf8')).toBe('competitor')
})
it('createIfAbsent rejects a competing directory as not a regular file', async () => {
const path = join(dir, 'a.txt')
const target = await fs.resolve('a.txt')
fs.internals.inspectTemp = async () => { await mkdir(path) }
await expect(fs.writeText(target, 'ours', { kind: 'createIfAbsent' }))
.rejects.toMatchObject({ code: 'FS_NOT_REGULAR_FILE' })
expect((await stat(path)).isDirectory()).toBe(true)
})
it('createIfAbsent rejects and preserves a dangling symbolic link', async () => {
const path = join(dir, 'dangling')
await symlink(join(dir, 'missing-target'), path)
const target = await fs.resolve('dangling')
await expect(fs.writeText(target, 'ours', { kind: 'createIfAbsent' }))
.rejects.toMatchObject({ code: 'FS_NOT_REGULAR_FILE' })
await expect(readFile(path, 'utf8')).rejects.toMatchObject({ code: 'ENOENT' })
})
it('replaceIfVersion replaces when the version matches', async () => {
await writeFile(join(dir, 'a.txt'), 'old')
const target = await fs.resolve('a.txt')
+62 -2
View File
@@ -496,17 +496,77 @@ describe('writeFileAtomic — temp-file safety', () => {
expect((await readdir(dir)).filter(name => name.includes('.tmp'))).toEqual([])
})
it('surfaces a non-collision guarded-create publication failure and cleans staging', async () => {
it('maps a non-collision guarded-create publication failure and cleans staging', async () => {
const file = join(dir, 'a.txt')
const denied = Object.assign(new Error('link denied'), { code: 'EACCES' })
await expect(writeFileAtomic(file, 'ours', undefined, undefined, {
linkFile: async () => { throw denied },
}, true)).rejects.toBe(denied)
}, { displayPath: file })).rejects.toMatchObject({ code: 'FS_IO_ERROR', cause: denied })
await expect(stat(file)).rejects.toMatchObject({ code: 'ENOENT' })
expect((await readdir(dir)).filter(name => name.includes('.tmp'))).toEqual([])
})
it('maps a guarded-create target-inspection failure and cleans staging', async () => {
const file = join(dir, 'a.txt')
const linkFailure = Object.assign(new Error('link failed'), { code: 'EIO' })
const inspectionFailure = Object.assign(new Error('inspection denied'), { code: 'EACCES' })
await expect(writeFileAtomic(file, 'ours', undefined, undefined, {
linkFile: async () => { throw linkFailure },
inspectPublicationTarget: async () => { throw inspectionFailure },
}, { displayPath: file })).rejects.toMatchObject({ code: 'FS_IO_ERROR', cause: inspectionFailure })
await expect(stat(file)).rejects.toMatchObject({ code: 'ENOENT' })
expect((await readdir(dir)).filter(name => name.includes('.tmp'))).toEqual([])
})
it('rejects a guarded-create collision that vanishes before inspection', async () => {
const file = join(dir, 'a.txt')
const collision = Object.assign(new Error('target existed'), { code: 'EEXIST' })
await expect(writeFileAtomic(file, 'ours', undefined, undefined, {
linkFile: async () => { throw collision },
}, { displayPath: file })).rejects.toMatchObject({
code: 'FS_NOT_OBSERVED',
message: `cannot overwrite existing "${file}" without reading it first`,
cause: collision,
})
await expect(stat(file)).rejects.toMatchObject({ code: 'ENOENT' })
expect((await readdir(dir)).filter(name => name.includes('.tmp'))).toEqual([])
})
it('uses the display path and target type when guarded publication finds a competitor', async () => {
const file = join(dir, 'a.txt')
const displayPath = join(dir, 'linked-workspace', 'a.txt')
await expect(writeFileAtomic(file, 'ours', undefined, undefined, {
inspectTemp: async () => { await writeFile(file, 'competitor') },
}, { displayPath })).rejects.toMatchObject({
code: 'FS_NOT_OBSERVED',
message: `cannot overwrite existing "${displayPath}" without reading it first`,
})
expect(await readFile(file, 'utf8')).toBe('competitor')
await rm(file)
await expect(writeFileAtomic(file, 'ours', undefined, undefined, {
inspectTemp: async () => { await mkdir(file) },
}, { displayPath })).rejects.toMatchObject({
code: 'FS_NOT_REGULAR_FILE',
message: `cannot write "${displayPath}": not a regular file`,
})
expect((await stat(file)).isDirectory()).toBe(true)
})
it('does not turn post-commit staging cleanup failure into a failed guarded write', async () => {
const file = join(dir, 'a.txt')
const cleanupFailure = new Error('staging cleanup failed')
await expect(writeFileAtomic(file, 'ours', undefined, undefined, {
removeStagingDir: async () => { throw cleanupFailure },
}, { displayPath: file })).resolves.toBeUndefined()
expect(await readFile(file, 'utf8')).toBe('ours')
})
it.skipIf(!posixModes)('creates new files owner-only by default', async () => {
const file = join(dir, 'a.txt')
await writeFileAtomic(file, 'hello', undefined, undefined)