refactor(fs): split filesystem seam into provider ctx.fs + policy ctx.fileContext
Implements the split-the-filesystem-seam RFC. ctx.fs shrinks to a text-storage provider seam (resolve/stat/readText/streamText/writeText/editText with branded FsTargetKey/FsVersion and an explicit FsWriteExpectation); the new dsh-file-context package owns the model-facing policy (read windowing, observed-state, write/edit freshness) as the concrete ctx.fileContext service. Authorization is now freshness-based rather than full/partial view: a windowed read records the file version and authorizes a later edit when the file is unchanged, removing the dead-end where reading lines 100-150 of a large file could not edit line 120. editText stays a provider primitive so version guard + literal match + atomic rewrite remain one critical section, and the stale check runs before matching so a stale edit reports FS_STALE_VERSION. tool-fs injects fileContext, never reaching around to ctx.fs (the no-bypass contract).
This commit is contained in:
Dudu-0223
@@ -1,20 +1,22 @@
|
||||
# @deepseek-ai/dsh-fs-local
|
||||
|
||||
The **local-filesystem implementation** of the `ctx.fs` seam ([`@deepseek-ai/dsh-fs`](../fs)). Backs the four `FileSystem` primitives with the host filesystem; loading it as a plugin populates `ctx.fs`.
|
||||
The **local-filesystem implementation** of the `ctx.fs` provider seam ([`@deepseek-ai/dsh-fs`](../fs)). Backs the six `FileSystem` primitives with the host filesystem; loading it as a plugin populates `ctx.fs`.
|
||||
|
||||
```ts ignore-check
|
||||
import { LocalFileSystem } from '@deepseek-ai/dsh-fs-local'
|
||||
|
||||
await ctx.plugin(LocalFileSystem, { cwd: process.cwd() })
|
||||
// ctx.fs is now the local backend; load @deepseek-ai/dsh-tool-fs to expose read/write/edit to the model.
|
||||
// ctx.fs is now the local backend; load @deepseek-ai/dsh-file-context for policy
|
||||
// and @deepseek-ai/dsh-tool-fs to expose read/write/edit to the model.
|
||||
```
|
||||
|
||||
## Behavior
|
||||
|
||||
- **`resolve(path)`** — relative paths resolve from `config.cwd` (default `process.cwd()`). The `targetKey` is the file's `realpath`, so two input paths reaching the same file through symlinks share one identity, and writes/edits land on the link target (preserving the link). A not-yet-existing path uses the realpathed parent directory plus basename when the parent exists; only an unresolvable parent falls back to the absolute path. `displayPath` is the absolute (un-resolved) path.
|
||||
- **`readPage`** — UTF-8 only. A fast path (`readFile`) handles files under `FAST_PATH_MAX_SIZE` (10 MB); larger files stream with a capped line buffer so a newline-free giant file can't exhaust memory. Invalid UTF-8 and NUL-byte samples are rejected (`FS_NOT_TEXT`). Output is bounded to `READ_LIMIT` (2000) lines, `READ_MAX_BYTES` (50 KB), and `READ_MAX_LINE_LENGTH` (2000) chars per line; hitting any bound records a `partial` view. The `version` is `mtimeMs:size`.
|
||||
- **`createOrReplace`** — atomic: writes to a temp file opened exclusively (`wx`, `0o600`) inside a randomly-named private staging dir (`0o700`) next to the target, fsyncs, then renames over the target. An existing file's mode is preserved, while new files default to `0o600`. Honors the `FsExpectation`: an `observed` write must match the recorded version (else `FS_STALE_VERSION`); a `partial` write onto an existing file is rejected (`FS_PARTIAL_OBSERVATION`); an `unobserved` write onto an existing file is rejected (`FS_NOT_OBSERVED`).
|
||||
- **`applyEdit`** — atomic literal read-modify-write over the same primitive. Verifies the expected version, LF-normalizes for matching, restores the file's dominant CRLF/LF style, and rejects empty `oldString` / zero matches (`FS_EDIT_NOT_FOUND`) or ambiguous multi-matches without `replace_all` (`FS_AMBIGUOUS_EDIT`).
|
||||
- **`stat`** — returns `FsInfo` (`version` = `mtimeMs:size`, `type` of `file`/`directory`/`other`, byte `size`) or `undefined` when the target is absent.
|
||||
- **`readText` / `streamText`** — UTF-8 only. `readText` reads the whole file; `streamText` streams it in chunks (cross-chunk decoding) so a huge file never has to be held whole in memory. Both reject invalid UTF-8 and NUL-byte binary samples (`FS_NOT_TEXT`) and non-regular targets. The policy layer (`ctx.fileContext`) decides which to call by size and owns the line windowing.
|
||||
- **`writeText`** — atomic: writes to a temp file opened exclusively (`wx`, `0o600`) inside a randomly-named private staging dir (`0o700`) next to the target, fsyncs, then renames over the target. An existing file's mode is preserved, while new files default to `0o600`. Honors the `FsWriteExpectation`: `createIfAbsent` creates a missing target and rejects an existing one (`FS_NOT_OBSERVED`); `replaceIfVersion` replaces only at the observed version (a missing target or mismatch is `FS_STALE_VERSION`).
|
||||
- **`editText`** — atomic literal read-modify-write over the same primitive, serialized per target by a mutation lock. Verifies the expected version BEFORE literal matching (a stale edit reports `FS_STALE_VERSION`, never `FS_EDIT_NOT_FOUND`/`FS_AMBIGUOUS_EDIT` against newer content), LF-normalizes for matching, restores the file's dominant CRLF/LF style, and rejects empty `oldString` / zero matches (`FS_EDIT_NOT_FOUND`) or ambiguous multi-matches without `replace_all` (`FS_AMBIGUOUS_EDIT`).
|
||||
|
||||
## `cwd` is not a sandbox
|
||||
|
||||
|
||||
@@ -1,13 +1,13 @@
|
||||
/**
|
||||
* Cordis-free local-filesystem I/O for `@deepseek-ai/dsh-fs-local`. Kept
|
||||
* separate from the service class (mirroring `dsh-bash-local`'s `run.ts`) so
|
||||
* the raw read/write/edit mechanics can be unit-tested without a Context.
|
||||
* the raw stat/read/write/edit mechanics can be unit-tested without a Context.
|
||||
*
|
||||
* The reader uses two code paths so a single huge line can never balloon
|
||||
* memory: a **fast path** (`readFile` + in-memory split) for files under
|
||||
* {@link FAST_PATH_MAX_SIZE}, and a **streaming path** (manual newline scan
|
||||
* with a capped line buffer) for larger files. Both reject invalid UTF-8 and
|
||||
* NUL-byte binary samples, and keep only the requested page in memory.
|
||||
* This is the PROVIDER layer: it hands back decoded whole-file text (validated
|
||||
* UTF-8, binary rejected) — never line windows or numbered lines, which are
|
||||
* model-facing read policy owned by `@deepseek-ai/dsh-file-context`. Large files
|
||||
* stream their text in chunks so a huge file never has to be held whole in
|
||||
* memory; the binary/NUL sample and cross-chunk UTF-8 decoding stay here.
|
||||
*
|
||||
* Writes are atomic: content goes to a temp file opened exclusively (`wx`,
|
||||
* `0o600`, so a pre-existing path can never be clobbered and write-in-progress
|
||||
@@ -24,56 +24,12 @@ import { chmod, mkdir, open, readFile, realpath, rename, rm, stat } from 'node:f
|
||||
import type { Stats } from 'node:fs'
|
||||
import { basename, dirname, join, resolve } from 'node:path'
|
||||
import { TextDecoder } from 'node:util'
|
||||
import { FsError } from '@deepseek-ai/dsh-fs'
|
||||
import type { FsReadRequest, FsTextLine, FsView } from '@deepseek-ai/dsh-fs'
|
||||
import { FsError, FsTargetKey, FsVersion } from '@deepseek-ai/dsh-fs'
|
||||
|
||||
/** Default and maximum number of lines returned by one read. */
|
||||
export const READ_LIMIT = 2000
|
||||
/** Files at or above this size stream their text; smaller files read whole. */
|
||||
export const STREAM_MIN_SIZE = 10 * 1024 * 1024
|
||||
|
||||
/** Maximum characters returned for a single line. */
|
||||
export const READ_MAX_LINE_LENGTH = 2000
|
||||
|
||||
/** Maximum bytes returned for selected file lines. */
|
||||
export const READ_MAX_BYTES = 50 * 1024
|
||||
|
||||
/** Files smaller than this use the in-memory fast path; larger files stream. */
|
||||
export const FAST_PATH_MAX_SIZE = 10 * 1024 * 1024
|
||||
|
||||
const READ_MAX_BYTES_LABEL = `${READ_MAX_BYTES / 1024} KB`
|
||||
const READ_MAX_LINE_SUFFIX = `... (line truncated to ${READ_MAX_LINE_LENGTH} chars)`
|
||||
const BINARY_SAMPLE_BYTES = 8192
|
||||
const LINE_BUFFER_CAP = READ_MAX_LINE_LENGTH + 1
|
||||
|
||||
/**
|
||||
* Test seam: lets specs force the streaming path (via a small
|
||||
* `fastPathMaxSize`) and pin the temp-file name (to prove exclusive-open
|
||||
* behavior) without a 10 MB fixture or a name race.
|
||||
*/
|
||||
export interface FsIoInternals {
|
||||
/** Override {@link FAST_PATH_MAX_SIZE} for routing. */
|
||||
fastPathMaxSize?: number
|
||||
/** Override the generated private staging-dir name (relative to the target dir). */
|
||||
tempDirName?: (writePath: string) => string
|
||||
/** Override the generated temp-file name (relative to the private staging dir). */
|
||||
tempName?: (writePath: string) => string
|
||||
/** Test hook after the temp file is written/synced but before final chmod+rename. */
|
||||
inspectTemp?: (paths: { stagingDir: string; tempPath: string }) => void | Promise<void>
|
||||
}
|
||||
|
||||
/** A resolved local path: the absolute path shown to callers and its realpath identity. */
|
||||
export interface LocalTarget {
|
||||
/** Absolute path (symlinks not resolved) — used for display. */
|
||||
displayPath: string
|
||||
/** Realpath identity — used as the stable target key and the I/O path. */
|
||||
targetKey: string
|
||||
}
|
||||
|
||||
/** Result of probing a path: null when it does not exist. */
|
||||
export interface PathInfo {
|
||||
version: string
|
||||
mode: number
|
||||
isFile: boolean
|
||||
}
|
||||
|
||||
function isENOENT(error: unknown): boolean {
|
||||
return error instanceof Error && 'code' in error && error.code === 'ENOENT'
|
||||
@@ -94,8 +50,40 @@ function throwIfAborted(signal: AbortSignal | undefined, verb: string): void {
|
||||
}
|
||||
|
||||
/** Opaque version token from a stat: mtime (ns precision) + size. */
|
||||
function versionOf(info: Stats): string {
|
||||
return `${info.mtimeMs}:${info.size}`
|
||||
function versionOf(info: Stats): FsVersion {
|
||||
return FsVersion(`${info.mtimeMs}:${info.size}`)
|
||||
}
|
||||
|
||||
/**
|
||||
* Test seam: lets specs force the streaming read path (via a small
|
||||
* `streamMinSize`) and pin the temp-file name (to prove exclusive-open
|
||||
* behavior) without a 10 MB fixture or a name race.
|
||||
*/
|
||||
export interface FsIoInternals {
|
||||
/** Override {@link STREAM_MIN_SIZE} for read routing. */
|
||||
streamMinSize?: number
|
||||
/** Override the generated private staging-dir name (relative to the target dir). */
|
||||
tempDirName?: (writePath: string) => string
|
||||
/** Override the generated temp-file name (relative to the private staging dir). */
|
||||
tempName?: (writePath: string) => string
|
||||
/** Test hook after the temp file is written/synced but before final chmod+rename. */
|
||||
inspectTemp?: (paths: { stagingDir: string; tempPath: string }) => void | Promise<void>
|
||||
}
|
||||
|
||||
/** A resolved local path: the absolute path shown to callers and its realpath identity. */
|
||||
export interface LocalTarget {
|
||||
/** Absolute path (symlinks not resolved) — used for display. */
|
||||
displayPath: string
|
||||
/** Realpath identity — used as the stable target key and the I/O path. */
|
||||
targetKey: FsTargetKey
|
||||
}
|
||||
|
||||
/** Result of probing a path: null when it does not exist. */
|
||||
export interface PathInfo {
|
||||
version: FsVersion
|
||||
mode: number
|
||||
type: 'file' | 'directory' | 'other'
|
||||
size: number
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -111,26 +99,27 @@ export async function resolveLocalTarget(cwd: string, path: string): Promise<Loc
|
||||
const displayPath = resolve(cwd, path)
|
||||
try {
|
||||
// Prefer the file's own realpath (resolves a symlinked file to its target).
|
||||
return { displayPath, targetKey: await realpath(displayPath) }
|
||||
return { displayPath, targetKey: FsTargetKey(await realpath(displayPath)) }
|
||||
} catch (error: unknown) {
|
||||
/* v8 ignore next -- non-ENOENT realpath failure needs a permission/IO fault; ENOENT falls through to parent-dir resolution. */
|
||||
if (!isENOENT(error)) throw error
|
||||
}
|
||||
try {
|
||||
// File absent: realpath the parent dir + basename so creates get a stable key.
|
||||
return { displayPath, targetKey: join(await realpath(dirname(displayPath)), basename(displayPath)) }
|
||||
return { displayPath, targetKey: FsTargetKey(join(await realpath(dirname(displayPath)), basename(displayPath))) }
|
||||
} catch (error: unknown) {
|
||||
/* v8 ignore next -- parent-dir realpath failing needs the dir itself to be missing/unreadable; fall back to the absolute path. */
|
||||
if (!isENOENT(error)) throw error
|
||||
return { displayPath, targetKey: displayPath }
|
||||
return { displayPath, targetKey: FsTargetKey(displayPath) }
|
||||
}
|
||||
}
|
||||
|
||||
/** Probe a path for its version, mode, and regular-file status. Null if absent. */
|
||||
/** Probe a path for its version, mode, type, and size. Null if absent. */
|
||||
export async function probe(absolutePath: string): Promise<PathInfo | null> {
|
||||
try {
|
||||
const info = await stat(absolutePath)
|
||||
return { version: versionOf(info), mode: info.mode & 0o777, isFile: info.isFile() }
|
||||
const type = info.isFile() ? 'file' : info.isDirectory() ? 'directory' : 'other'
|
||||
return { version: versionOf(info), mode: info.mode & 0o777, type, size: info.size }
|
||||
} catch (error: unknown) {
|
||||
/* v8 ignore next 2 -- a non-ENOENT stat failure needs a permission/IO fault; surface it. */
|
||||
if (!isENOENT(error)) throw error
|
||||
@@ -140,67 +129,6 @@ export async function probe(absolutePath: string): Promise<PathInfo | null> {
|
||||
|
||||
// --- Reading ---
|
||||
|
||||
interface PageAccumulator {
|
||||
lines: FsTextLine[]
|
||||
totalLines: number
|
||||
outputBytes: number
|
||||
truncatedByBytes: boolean
|
||||
truncatedByLine: boolean
|
||||
done: boolean
|
||||
}
|
||||
|
||||
function newAccumulator(): PageAccumulator {
|
||||
return { lines: [], totalLines: 0, outputBytes: 0, truncatedByBytes: false, truncatedByLine: false, done: false }
|
||||
}
|
||||
|
||||
function truncateReadLine(line: string): { text: string; truncated: boolean } {
|
||||
return line.length > READ_MAX_LINE_LENGTH
|
||||
? { text: `${line.substring(0, READ_MAX_LINE_LENGTH)}${READ_MAX_LINE_SUFFIX}`, truncated: true }
|
||||
: { text: line, truncated: false }
|
||||
}
|
||||
|
||||
function lineByteSize(line: string, currentLineCount: number): number {
|
||||
return Buffer.byteLength(line, 'utf8') + (currentLineCount > 0 ? 1 : 0)
|
||||
}
|
||||
|
||||
function consumeLine(acc: PageAccumulator, rawLine: string, request: FsReadRequest): void {
|
||||
acc.totalLines += 1
|
||||
if (acc.totalLines < request.offset || acc.lines.length >= request.limit) return
|
||||
|
||||
const { text, truncated } = truncateReadLine(rawLine)
|
||||
if (truncated) acc.truncatedByLine = true
|
||||
const bytes = lineByteSize(text, acc.lines.length)
|
||||
if (acc.outputBytes + bytes > READ_MAX_BYTES) {
|
||||
acc.truncatedByBytes = true
|
||||
acc.done = true
|
||||
return
|
||||
}
|
||||
acc.outputBytes += bytes
|
||||
acc.lines.push({ number: acc.totalLines, text })
|
||||
}
|
||||
|
||||
function stripCarriageReturn(line: string): string {
|
||||
return line.endsWith('\r') ? line.slice(0, -1) : line
|
||||
}
|
||||
|
||||
/** The outcome shape `readTextPage` returns (minus the offset/limit echo, which the caller adds). */
|
||||
export interface ReadPageResult {
|
||||
lines: FsTextLine[]
|
||||
totalLines: number
|
||||
truncatedByBytes: boolean
|
||||
view: FsView
|
||||
version: string
|
||||
}
|
||||
|
||||
function buildResult(acc: PageAccumulator, request: FsReadRequest, version: string, displayPath: string): ReadPageResult {
|
||||
if (!acc.truncatedByBytes && request.offset > acc.totalLines && !(acc.totalLines === 0 && request.offset === 1)) {
|
||||
throw new FsError(`offset ${request.offset} is out of range for "${displayPath}" (${acc.totalLines} lines)`, 'FS_NOT_FOUND')
|
||||
}
|
||||
const endLine = acc.lines.at(-1)?.number ?? Math.max(0, request.offset - 1)
|
||||
const view: FsView = request.offset === 1 && !acc.truncatedByBytes && !acc.truncatedByLine && endLine >= acc.totalLines ? 'full' : 'partial'
|
||||
return { lines: acc.lines, totalLines: acc.totalLines, truncatedByBytes: acc.truncatedByBytes, view, version }
|
||||
}
|
||||
|
||||
function notTextError(verb: 'read' | 'edit', displayPath: string): FsError {
|
||||
return new FsError(`cannot ${verb} "${displayPath}": invalid UTF-8 text`, 'FS_NOT_TEXT')
|
||||
}
|
||||
@@ -209,8 +137,9 @@ function decodeUtf8(buffer: Uint8Array, verb: 'read' | 'edit', displayPath: stri
|
||||
try {
|
||||
return new TextDecoder('utf-8', { fatal: true }).decode(buffer)
|
||||
} catch (error: unknown) {
|
||||
if (error instanceof TypeError) throw notTextError(verb, displayPath)
|
||||
throw error
|
||||
/* v8 ignore next 2 -- TextDecoder({fatal}) only throws TypeError on invalid bytes; any other throw is an unreachable runtime fault. */
|
||||
if (!(error instanceof TypeError)) throw error
|
||||
throw notTextError(verb, displayPath)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -223,90 +152,50 @@ function decodeUtf8Stream(
|
||||
try {
|
||||
return chunk ? decoder.decode(chunk, { stream: true }) : decoder.decode()
|
||||
} catch (error: unknown) {
|
||||
if (error instanceof TypeError) throw notTextError(verb, displayPath)
|
||||
throw error
|
||||
/* v8 ignore next 2 -- TextDecoder({fatal}) only throws TypeError on invalid bytes; any other throw is an unreachable runtime fault. */
|
||||
if (!(error instanceof TypeError)) throw error
|
||||
throw notTextError(verb, displayPath)
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Read a bounded UTF-8 text-file page. Rejects non-regular files, invalid
|
||||
* UTF-8, and NUL-byte binary samples; dispatches to the fast or streaming path
|
||||
* by file size.
|
||||
*/
|
||||
export async function readTextPage(
|
||||
target: LocalTarget,
|
||||
request: FsReadRequest,
|
||||
signal?: AbortSignal,
|
||||
internals: FsIoInternals = {},
|
||||
): Promise<ReadPageResult> {
|
||||
throwIfAborted(signal, 'read')
|
||||
const absolutePath = target.targetKey
|
||||
async function statRegularFile(target: LocalTarget, verb: 'read', signal?: AbortSignal): Promise<Stats> {
|
||||
throwIfAborted(signal, verb)
|
||||
let info: Stats
|
||||
try {
|
||||
info = await stat(absolutePath)
|
||||
info = await stat(target.targetKey)
|
||||
} catch (error: unknown) {
|
||||
/* v8 ignore next 2 -- a non-ENOENT stat failure needs a permission/IO fault; only the not-found path is reachable in tests. */
|
||||
if (!isENOENT(error)) throw error
|
||||
throw new FsError(`cannot read "${target.displayPath}": not found`, 'FS_NOT_FOUND')
|
||||
throw new FsError(`cannot ${verb} "${target.displayPath}": not found`, 'FS_NOT_FOUND')
|
||||
}
|
||||
if (!info.isFile()) throw new FsError(`cannot read "${target.displayPath}": not a regular file`, 'FS_NOT_REGULAR_FILE')
|
||||
|
||||
const version = versionOf(info)
|
||||
const fastPathMax = internals.fastPathMaxSize ?? FAST_PATH_MAX_SIZE
|
||||
return info.size < fastPathMax
|
||||
? readTextPageFast(target, request, version, signal)
|
||||
: readTextPageStreaming(target, request, version, signal)
|
||||
if (!info.isFile()) throw new FsError(`cannot ${verb} "${target.displayPath}": not a regular file`, 'FS_NOT_REGULAR_FILE')
|
||||
return info
|
||||
}
|
||||
|
||||
async function readTextPageFast(
|
||||
target: LocalTarget,
|
||||
request: FsReadRequest,
|
||||
version: string,
|
||||
signal?: AbortSignal,
|
||||
): Promise<ReadPageResult> {
|
||||
/**
|
||||
* Read a whole regular UTF-8 text file into a single decoded string. Rejects
|
||||
* non-regular files, invalid UTF-8, and NUL-byte binary samples.
|
||||
*/
|
||||
export async function readWholeText(target: LocalTarget, signal?: AbortSignal): Promise<string> {
|
||||
await statRegularFile(target, 'read', signal)
|
||||
const raw = await readFile(target.targetKey, signal ? { signal } : {})
|
||||
throwIfAborted(signal, 'read')
|
||||
if (raw.subarray(0, BINARY_SAMPLE_BYTES).includes(0)) {
|
||||
throw new FsError(`cannot read "${target.displayPath}": binary file`, 'FS_NOT_TEXT')
|
||||
}
|
||||
|
||||
const text = decodeUtf8(raw, 'read', target.displayPath)
|
||||
const acc = newAccumulator()
|
||||
let startPos = 0
|
||||
let newlinePos: number
|
||||
while ((newlinePos = text.indexOf('\n', startPos)) !== -1) {
|
||||
consumeLine(acc, stripCarriageReturn(text.slice(startPos, newlinePos)), request)
|
||||
if (acc.done) break
|
||||
startPos = newlinePos + 1
|
||||
}
|
||||
if (!acc.done && startPos < text.length) {
|
||||
consumeLine(acc, stripCarriageReturn(text.slice(startPos)), request)
|
||||
}
|
||||
return buildResult(acc, request, version, target.displayPath)
|
||||
return decodeUtf8(raw, 'read', target.displayPath)
|
||||
}
|
||||
|
||||
async function readTextPageStreaming(
|
||||
target: LocalTarget,
|
||||
request: FsReadRequest,
|
||||
version: string,
|
||||
signal?: AbortSignal,
|
||||
): Promise<ReadPageResult> {
|
||||
/**
|
||||
* Stream a whole regular UTF-8 text file as decoded text chunks. Same text
|
||||
* semantics as {@link readWholeText} (regular-file check, binary/NUL rejection,
|
||||
* cross-chunk UTF-8 decoding), but never holds the whole file in memory.
|
||||
*/
|
||||
export async function* streamWholeText(target: LocalTarget, signal?: AbortSignal): AsyncIterable<string> {
|
||||
await statRegularFile(target, 'read', signal)
|
||||
const stream = createReadStream(target.targetKey, signal ? { signal } : {})
|
||||
const acc = newAccumulator()
|
||||
let lineBuffer = ''
|
||||
let sampledBytes = 0
|
||||
const decoder = new TextDecoder('utf-8', { fatal: true })
|
||||
|
||||
function appendToLineBuffer(segment: string): void {
|
||||
if (lineBuffer.length >= LINE_BUFFER_CAP) return
|
||||
lineBuffer += segment
|
||||
if (lineBuffer.length > LINE_BUFFER_CAP) lineBuffer = lineBuffer.slice(0, LINE_BUFFER_CAP)
|
||||
}
|
||||
|
||||
function flushLine(): void {
|
||||
consumeLine(acc, stripCarriageReturn(lineBuffer), request)
|
||||
lineBuffer = ''
|
||||
}
|
||||
let sampledBytes = 0
|
||||
|
||||
function scanBinarySample(chunk: Buffer): void {
|
||||
if (sampledBytes >= BINARY_SAMPLE_BYTES) return
|
||||
@@ -317,51 +206,17 @@ async function readTextPageStreaming(
|
||||
sampledBytes += sample.length
|
||||
}
|
||||
|
||||
function consumeChunk(chunk: string): ReadPageResult | undefined {
|
||||
let startPos = 0
|
||||
let newlinePos: number
|
||||
while ((newlinePos = chunk.indexOf('\n', startPos)) !== -1) {
|
||||
appendToLineBuffer(chunk.slice(startPos, newlinePos))
|
||||
flushLine()
|
||||
startPos = newlinePos + 1
|
||||
if (acc.done) return buildResult(acc, request, version, target.displayPath)
|
||||
}
|
||||
appendToLineBuffer(chunk.slice(startPos))
|
||||
return undefined
|
||||
}
|
||||
|
||||
try {
|
||||
for await (const chunk of stream as AsyncIterable<Buffer>) {
|
||||
scanBinarySample(chunk)
|
||||
const result = consumeChunk(decodeUtf8Stream(decoder, chunk, 'read', target.displayPath))
|
||||
if (result) return result
|
||||
yield decodeUtf8Stream(decoder, chunk, 'read', target.displayPath)
|
||||
}
|
||||
const finalResult = consumeChunk(decodeUtf8Stream(decoder, undefined, 'read', target.displayPath))
|
||||
if (finalResult) return finalResult
|
||||
yield decodeUtf8Stream(decoder, undefined, 'read', target.displayPath)
|
||||
} catch (error: unknown) {
|
||||
/* v8 ignore next 4 -- mid-stream errors need an abort/IO fault racing the loop; pre-abort is caught by throwIfAborted. */
|
||||
if (isAbortError(error)) throw new FsError('read aborted', 'FS_ABORTED')
|
||||
throw error
|
||||
}
|
||||
|
||||
if (lineBuffer.length > 0) flushLine()
|
||||
return buildResult(acc, request, version, target.displayPath)
|
||||
}
|
||||
|
||||
/** Format the line-numbered body + pagination footer for a read page. */
|
||||
export function formatReadBody(result: ReadPageResult, offset: number): string {
|
||||
const endLine = result.lines.at(-1)?.number ?? Math.max(0, offset - 1)
|
||||
let footer: string
|
||||
if (result.truncatedByBytes) {
|
||||
footer = `(Output capped at ${READ_MAX_BYTES_LABEL}. Showing lines ${offset}-${endLine}. Use offset=${endLine + 1} to continue.)`
|
||||
} else if (endLine < result.totalLines) {
|
||||
footer = `(Showing lines ${offset}-${endLine} of ${result.totalLines}. Use offset=${endLine + 1} to continue.)`
|
||||
} else {
|
||||
footer = `(End of file - total ${result.totalLines} lines)`
|
||||
}
|
||||
return result.lines.length > 0
|
||||
? `${result.lines.map(line => `${line.number}: ${line.text}`).join('\n')}\n\n${footer}`
|
||||
: footer
|
||||
}
|
||||
|
||||
// --- Writing ---
|
||||
|
||||
@@ -1,10 +1,11 @@
|
||||
/**
|
||||
* Local-filesystem implementation of the `ctx.fs` seam. {@link LocalFileSystem}
|
||||
* subclasses {@link FileSystem} and backs the four primitives with the host
|
||||
* filesystem via {@link module:@deepseek-ai/dsh-fs-local/fsio}. Path resolution
|
||||
* uses `realpath`, so the stable `targetKey` is the real file identity (two
|
||||
* input paths reaching the same file through symlinks share one key, and writes
|
||||
* land on the link target — preserving the link).
|
||||
* Local-filesystem implementation of the `ctx.fs` provider seam.
|
||||
* {@link LocalFileSystem} subclasses {@link FileSystem} and backs the six
|
||||
* text-storage primitives with the host filesystem via
|
||||
* {@link module:@deepseek-ai/dsh-fs-local/fsio}. Path resolution uses
|
||||
* `realpath`, so the stable `targetKey` is the real file identity (two input
|
||||
* paths reaching the same file through symlinks share one key, and writes land
|
||||
* on the link target — preserving the link).
|
||||
*
|
||||
* Future sandboxed/remote/virtual backends are sibling packages implementing
|
||||
* the same interface; loading this one populates `ctx.fs`.
|
||||
@@ -14,43 +15,39 @@
|
||||
|
||||
import { Context } from 'cordis'
|
||||
import z from 'schemastery'
|
||||
import { FileSystem, FsError } from '@deepseek-ai/dsh-fs'
|
||||
import { FileSystem, FsError, FsVersion } from '@deepseek-ai/dsh-fs'
|
||||
import type {
|
||||
FsEditOutcome,
|
||||
FsEditRequest,
|
||||
FsExpectation,
|
||||
FsReadOutcome,
|
||||
FsReadRequest,
|
||||
FsInfo,
|
||||
FsTarget,
|
||||
FsVersion,
|
||||
FsWriteExpectation,
|
||||
FsWriteOutcome,
|
||||
} from '@deepseek-ai/dsh-fs'
|
||||
import {
|
||||
applyLiteralEdit,
|
||||
probe,
|
||||
readForEdit,
|
||||
readTextPage,
|
||||
readWholeText,
|
||||
resolveLocalTarget,
|
||||
restoreLineEndings,
|
||||
streamWholeText,
|
||||
writeFileAtomic,
|
||||
} from './fsio.ts'
|
||||
import type { FsIoInternals } from './fsio.ts'
|
||||
|
||||
export {
|
||||
FAST_PATH_MAX_SIZE,
|
||||
READ_LIMIT,
|
||||
READ_MAX_BYTES,
|
||||
READ_MAX_LINE_LENGTH,
|
||||
STREAM_MIN_SIZE,
|
||||
applyLiteralEdit,
|
||||
formatReadBody,
|
||||
probe,
|
||||
readForEdit,
|
||||
readTextPage,
|
||||
readWholeText,
|
||||
resolveLocalTarget,
|
||||
restoreLineEndings,
|
||||
streamWholeText,
|
||||
writeFileAtomic,
|
||||
} from './fsio.ts'
|
||||
export type { FsIoInternals, LineEndings, LocalTarget, PathInfo, ReadPageResult } from './fsio.ts'
|
||||
export type { FsIoInternals, LineEndings, LocalTarget, PathInfo } from './fsio.ts'
|
||||
|
||||
/** Configuration for the local filesystem backend. */
|
||||
export interface Config {
|
||||
@@ -105,47 +102,41 @@ export class LocalFileSystem extends FileSystem {
|
||||
return { inputPath: path, targetKey: local.targetKey, displayPath: local.displayPath }
|
||||
}
|
||||
|
||||
override async readPage(target: FsTarget, request: FsReadRequest, signal?: AbortSignal): Promise<FsReadOutcome> {
|
||||
const result = await readTextPage(
|
||||
{ displayPath: target.displayPath, targetKey: target.targetKey },
|
||||
request,
|
||||
signal,
|
||||
this.internals,
|
||||
)
|
||||
return {
|
||||
offset: request.offset,
|
||||
limit: request.limit,
|
||||
lines: result.lines,
|
||||
totalLines: result.totalLines,
|
||||
version: result.version,
|
||||
view: result.view,
|
||||
...result.truncatedByBytes ? { truncatedByBytes: true } : {},
|
||||
}
|
||||
override async stat(target: FsTarget, signal?: AbortSignal): Promise<FsInfo | undefined> {
|
||||
if (signal?.aborted) throw new FsError('stat aborted', 'FS_ABORTED')
|
||||
const info = await probe(target.targetKey)
|
||||
if (!info) return undefined
|
||||
return { version: info.version, type: info.type, size: info.size }
|
||||
}
|
||||
|
||||
override async createOrReplace(
|
||||
override async readText(target: FsTarget, signal?: AbortSignal): Promise<string> {
|
||||
return readWholeText({ displayPath: target.displayPath, targetKey: target.targetKey }, signal)
|
||||
}
|
||||
|
||||
override streamText(target: FsTarget, signal?: AbortSignal): Promise<AsyncIterable<string>> {
|
||||
return Promise.resolve(streamWholeText({ displayPath: target.displayPath, targetKey: target.targetKey }, signal))
|
||||
}
|
||||
|
||||
override async writeText(
|
||||
target: FsTarget,
|
||||
content: string,
|
||||
expected: FsExpectation,
|
||||
expected: FsWriteExpectation,
|
||||
signal?: AbortSignal,
|
||||
): Promise<FsWriteOutcome> {
|
||||
return this.withLock(target.targetKey, async () => {
|
||||
const existing = await probe(target.targetKey)
|
||||
if (existing && !existing.isFile) {
|
||||
if (existing && existing.type !== 'file') {
|
||||
throw new FsError(`cannot write "${target.displayPath}": not a regular file`, 'FS_NOT_REGULAR_FILE')
|
||||
}
|
||||
|
||||
if (expected.kind === 'observed') {
|
||||
// Stale guard: the file must still be at the version the owner observed.
|
||||
if (expected.kind === 'replaceIfVersion') {
|
||||
// Stale guard: the file must still exist at the version the owner observed.
|
||||
if (!existing) throw new FsError(`cannot write "${target.displayPath}": file no longer exists`, 'FS_STALE_VERSION')
|
||||
if (existing.version !== expected.version) {
|
||||
throw new FsError(`cannot write "${target.displayPath}": file changed since it was read`, 'FS_STALE_VERSION')
|
||||
}
|
||||
} else if (expected.kind === 'partial') {
|
||||
if (!existing) throw new FsError(`cannot write "${target.displayPath}": file no longer exists`, 'FS_STALE_VERSION')
|
||||
throw new FsError(`cannot overwrite existing "${target.displayPath}" after only a partial read`, 'FS_PARTIAL_OBSERVATION')
|
||||
} else if (existing) {
|
||||
// Unobserved write onto an existing file: a blind overwrite — require a read first.
|
||||
// createIfAbsent onto an existing file: a blind overwrite — require a read first.
|
||||
throw new FsError(`cannot overwrite existing "${target.displayPath}" without reading it first`, 'FS_NOT_OBSERVED')
|
||||
}
|
||||
|
||||
@@ -158,7 +149,7 @@ export class LocalFileSystem extends FileSystem {
|
||||
})
|
||||
}
|
||||
|
||||
override async applyEdit(
|
||||
override async editText(
|
||||
target: FsTarget,
|
||||
edit: FsEditRequest,
|
||||
expected: { version: FsVersion },
|
||||
@@ -166,8 +157,10 @@ export class LocalFileSystem extends FileSystem {
|
||||
): Promise<FsEditOutcome> {
|
||||
return this.withLock(target.targetKey, async () => {
|
||||
const existing = await probe(target.targetKey)
|
||||
if (!existing) throw new FsError(`cannot edit "${target.displayPath}": not found`, 'FS_NOT_FOUND')
|
||||
if (!existing.isFile) throw new FsError(`cannot edit "${target.displayPath}": not a regular file`, 'FS_NOT_REGULAR_FILE')
|
||||
// Stale guard BEFORE literal matching: an edit based on an old read reports
|
||||
// FS_STALE_VERSION, not FS_EDIT_NOT_FOUND/FS_AMBIGUOUS_EDIT against newer content.
|
||||
if (!existing) throw new FsError(`cannot edit "${target.displayPath}": file changed since it was read`, 'FS_STALE_VERSION')
|
||||
if (existing.type !== 'file') throw new FsError(`cannot edit "${target.displayPath}": not a regular file`, 'FS_NOT_REGULAR_FILE')
|
||||
if (existing.version !== expected.version) {
|
||||
throw new FsError(`cannot edit "${target.displayPath}": file changed since it was read`, 'FS_STALE_VERSION')
|
||||
}
|
||||
@@ -188,9 +181,9 @@ export class LocalFileSystem extends FileSystem {
|
||||
|
||||
/* v8 ignore next 5 -- the post-write probe finding the file absent requires a
|
||||
* concurrent unlink between rename and stat; fall back to a sentinel version. */
|
||||
private versionAfterWrite(after: { version: string } | null, target: FsTarget): string {
|
||||
private versionAfterWrite(after: { version: FsVersion } | null, target: FsTarget): FsVersion {
|
||||
if (after) return after.version
|
||||
return `missing:${target.targetKey}`
|
||||
return FsVersion(`missing:${target.targetKey}`)
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -1,7 +1,9 @@
|
||||
/**
|
||||
* Tests for the local backend through the `ctx.fs` service: the full
|
||||
* read→write→edit lifecycle with the read-before-write policy, stale-version
|
||||
* guards, concurrency races, symlink identity, and HMR/disposal.
|
||||
* Tests for the local backend through the `ctx.fs` provider seam: stat, whole-
|
||||
* file/streamed text reads, atomic guarded writes (createIfAbsent /
|
||||
* replaceIfVersion), version-guarded literal edits, concurrency races, symlink
|
||||
* identity, and HMR/disposal. Read WINDOWING is policy and lives in
|
||||
* `dsh-file-context`, so it is not exercised here.
|
||||
*/
|
||||
|
||||
import { afterEach, beforeEach, describe, expect, it } from 'vitest'
|
||||
@@ -9,8 +11,9 @@ import { mkdtemp, readFile, rm, stat, symlink, writeFile, unlink } from 'node:fs
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
import { Context } from 'cordis'
|
||||
import { LocalFileSystem, probe } from '@deepseek-ai/dsh-fs-local'
|
||||
import type { FsExecContext } from '@deepseek-ai/dsh-fs'
|
||||
import { LocalFileSystem } from '@deepseek-ai/dsh-fs-local'
|
||||
import { FsVersion } from '@deepseek-ai/dsh-fs'
|
||||
import type { FsTarget } from '@deepseek-ai/dsh-fs'
|
||||
|
||||
let dir: string
|
||||
let ctx: Context
|
||||
@@ -28,12 +31,17 @@ afterEach(async () => {
|
||||
await rm(dir, { recursive: true, force: true })
|
||||
})
|
||||
|
||||
const READ_ALL = { offset: 1, limit: 2000 }
|
||||
const exec = (): FsExecContext => ({ agent: { session: {} } })
|
||||
function lockCount(localFs: LocalFileSystem): number {
|
||||
return (localFs as unknown as { locks: Map<string, Promise<unknown>> }).locks.size
|
||||
}
|
||||
|
||||
/** The version the backend currently reports for a resolved target. */
|
||||
async function versionOf(target: FsTarget): Promise<FsVersion> {
|
||||
const info = await fs.stat(target)
|
||||
if (!info) throw new Error('expected target to exist')
|
||||
return info.version
|
||||
}
|
||||
|
||||
describe('registration', () => {
|
||||
it('registers LocalFileSystem as ctx.fs with a default cwd', async () => {
|
||||
const bare = new Context()
|
||||
@@ -43,255 +51,219 @@ describe('registration', () => {
|
||||
})
|
||||
})
|
||||
|
||||
describe('read → write → edit lifecycle', () => {
|
||||
it('creates a new file without a prior read', async () => {
|
||||
describe('stat', () => {
|
||||
it('returns file metadata, directory type, and undefined for absent', async () => {
|
||||
await writeFile(join(dir, 'a.txt'), 'hello')
|
||||
const fileInfo = await fs.stat(await fs.resolve('a.txt'))
|
||||
expect(fileInfo?.type).toBe('file')
|
||||
expect(fileInfo?.size).toBe(5)
|
||||
expect(typeof fileInfo?.version).toBe('string')
|
||||
|
||||
expect((await fs.stat(await fs.resolve('.')))?.type).toBe('directory')
|
||||
expect(await fs.stat(await fs.resolve('missing.txt'))).toBeUndefined()
|
||||
})
|
||||
|
||||
it('honors a pre-aborted signal', async () => {
|
||||
await expect(fs.stat(await fs.resolve('a.txt'), AbortSignal.abort())).rejects.toMatchObject({ code: 'FS_ABORTED' })
|
||||
})
|
||||
})
|
||||
|
||||
describe('readText / streamText', () => {
|
||||
it('reads whole-file text', async () => {
|
||||
await writeFile(join(dir, 'a.txt'), 'one\ntwo\nthree')
|
||||
expect(await fs.readText(await fs.resolve('a.txt'))).toBe('one\ntwo\nthree')
|
||||
})
|
||||
|
||||
it('streams the same text', async () => {
|
||||
await writeFile(join(dir, 'a.txt'), 'one\ntwo\nthree')
|
||||
const target = await fs.resolve('a.txt')
|
||||
let streamed = ''
|
||||
for await (const chunk of await fs.streamText(target)) streamed += chunk
|
||||
expect(streamed).toBe('one\ntwo\nthree')
|
||||
})
|
||||
|
||||
it('rejects a missing file, a directory, binary, and invalid UTF-8', async () => {
|
||||
await expect(fs.readText(await fs.resolve('nope'))).rejects.toMatchObject({ code: 'FS_NOT_FOUND' })
|
||||
await expect(fs.readText(await fs.resolve('.'))).rejects.toMatchObject({ code: 'FS_NOT_REGULAR_FILE' })
|
||||
|
||||
await writeFile(join(dir, 'bin'), Buffer.from([0x68, 0x00, 0x69]))
|
||||
await expect(fs.readText(await fs.resolve('bin'))).rejects.toMatchObject({ code: 'FS_NOT_TEXT' })
|
||||
|
||||
await writeFile(join(dir, 'bad'), Buffer.from([0x68, 0xff, 0x69]))
|
||||
await expect(fs.readText(await fs.resolve('bad'))).rejects.toMatchObject({ code: 'FS_NOT_TEXT' })
|
||||
})
|
||||
})
|
||||
|
||||
describe('writeText', () => {
|
||||
it('createIfAbsent creates a new file', async () => {
|
||||
const target = await fs.resolve('new.txt')
|
||||
const outcome = await fs.write(target, 'fresh', exec())
|
||||
const outcome = await fs.writeText(target, 'fresh', { kind: 'createIfAbsent' })
|
||||
expect(outcome.operation).toBe('create')
|
||||
expect(await readFile(join(dir, 'new.txt'), 'utf8')).toBe('fresh')
|
||||
})
|
||||
|
||||
it('updates an existing file after reading it', async () => {
|
||||
it('createIfAbsent rejects an existing file as FS_NOT_OBSERVED', async () => {
|
||||
await writeFile(join(dir, 'a.txt'), 'old')
|
||||
const owner = exec()
|
||||
const target = await fs.resolve('a.txt')
|
||||
await fs.read(target, READ_ALL, owner)
|
||||
const outcome = await fs.write(target, 'new', owner)
|
||||
await expect(fs.writeText(target, 'new', { kind: 'createIfAbsent' }))
|
||||
.rejects.toMatchObject({ code: 'FS_NOT_OBSERVED' })
|
||||
expect(await readFile(join(dir, 'a.txt'), 'utf8')).toBe('old')
|
||||
})
|
||||
|
||||
it('replaceIfVersion replaces when the version matches', async () => {
|
||||
await writeFile(join(dir, 'a.txt'), 'old')
|
||||
const target = await fs.resolve('a.txt')
|
||||
const outcome = await fs.writeText(target, 'new', { kind: 'replaceIfVersion', version: await versionOf(target) })
|
||||
expect(outcome.operation).toBe('update')
|
||||
expect(await readFile(join(dir, 'a.txt'), 'utf8')).toBe('new')
|
||||
})
|
||||
|
||||
it('edits an existing file after reading it', async () => {
|
||||
await writeFile(join(dir, 'a.txt'), 'hello world')
|
||||
const owner = exec()
|
||||
it('replaceIfVersion rejects a stale version', async () => {
|
||||
await writeFile(join(dir, 'a.txt'), 'v1')
|
||||
const target = await fs.resolve('a.txt')
|
||||
await fs.read(target, READ_ALL, owner)
|
||||
const outcome = await fs.edit(target, { oldString: 'world', newString: 'there', replaceAll: false }, owner)
|
||||
expect(outcome.replacements).toBe(1)
|
||||
expect(await readFile(join(dir, 'a.txt'), 'utf8')).toBe('hello there')
|
||||
const stale = await versionOf(target)
|
||||
await writeFile(join(dir, 'a.txt'), 'changed-externally')
|
||||
await expect(fs.writeText(target, 'v2', { kind: 'replaceIfVersion', version: stale }))
|
||||
.rejects.toMatchObject({ code: 'FS_STALE_VERSION' })
|
||||
})
|
||||
|
||||
it('rejects an empty edit oldString through ctx.fs without hanging or changing the file', async () => {
|
||||
await writeFile(join(dir, 'a.txt'), 'hello world')
|
||||
const owner = exec()
|
||||
it('replaceIfVersion rejects a deleted target as stale, without recreating it', async () => {
|
||||
const path = join(dir, 'a.txt')
|
||||
await writeFile(path, 'v1')
|
||||
const target = await fs.resolve('a.txt')
|
||||
await fs.read(target, READ_ALL, owner)
|
||||
|
||||
await expect(fs.edit(target, { oldString: '', newString: 'boom', replaceAll: false }, owner))
|
||||
.rejects.toMatchObject({ code: 'FS_EDIT_NOT_FOUND' })
|
||||
expect(await readFile(join(dir, 'a.txt'), 'utf8')).toBe('hello world')
|
||||
const version = await versionOf(target)
|
||||
await unlink(path)
|
||||
await expect(fs.writeText(target, 'v2', { kind: 'replaceIfVersion', version }))
|
||||
.rejects.toMatchObject({ code: 'FS_STALE_VERSION' })
|
||||
await expect(stat(path)).rejects.toMatchObject({ code: 'ENOENT' })
|
||||
})
|
||||
|
||||
it('propagates truncatedByBytes from a byte-capped read', async () => {
|
||||
await writeFile(join(dir, 'big.txt'), Array.from({ length: 2000 }, () => 'y'.repeat(100)).join('\n'))
|
||||
const outcome = await fs.read(await fs.resolve('big.txt'), READ_ALL, exec())
|
||||
expect(outcome.truncatedByBytes).toBe(true)
|
||||
expect(outcome.view).toBe('partial')
|
||||
})
|
||||
|
||||
it('records an over-long-line read as partial, so write/edit stay blocked', async () => {
|
||||
await writeFile(join(dir, 'long.txt'), 'x'.repeat(3000))
|
||||
const owner = exec()
|
||||
const target = await fs.resolve('long.txt')
|
||||
const outcome = await fs.read(target, READ_ALL, owner)
|
||||
|
||||
expect(outcome.view).toBe('partial')
|
||||
await expect(fs.write(target, 'new', owner)).rejects.toMatchObject({ code: 'FS_PARTIAL_OBSERVATION' })
|
||||
await expect(
|
||||
fs.edit(target, { oldString: 'x', newString: 'y', replaceAll: false }, owner),
|
||||
).rejects.toMatchObject({ code: 'FS_PARTIAL_OBSERVATION' })
|
||||
})
|
||||
|
||||
it('allows a follow-up edit without re-reading (write/edit refresh state)', async () => {
|
||||
await writeFile(join(dir, 'a.txt'), 'a b')
|
||||
const owner = exec()
|
||||
const target = await fs.resolve('a.txt')
|
||||
await fs.read(target, READ_ALL, owner)
|
||||
await fs.edit(target, { oldString: 'a', newString: 'X', replaceAll: false }, owner)
|
||||
await fs.edit(target, { oldString: 'b', newString: 'Y', replaceAll: false }, owner)
|
||||
expect(await readFile(join(dir, 'a.txt'), 'utf8')).toBe('X Y')
|
||||
it('rejects writing onto a directory', async () => {
|
||||
const target = await fs.resolve('.')
|
||||
await expect(fs.writeText(target, 'x', { kind: 'createIfAbsent' }))
|
||||
.rejects.toMatchObject({ code: 'FS_NOT_REGULAR_FILE' })
|
||||
})
|
||||
|
||||
it('releases per-target mutation locks after success and failure', async () => {
|
||||
const target = await fs.resolve('a.txt')
|
||||
await fs.write(target, 'created', exec())
|
||||
await fs.writeText(target, 'created', { kind: 'createIfAbsent' })
|
||||
expect(lockCount(fs)).toBe(0)
|
||||
|
||||
await expect(fs.write(target, 'blind overwrite', exec())).rejects.toMatchObject({ code: 'FS_NOT_OBSERVED' })
|
||||
await expect(fs.writeText(target, 'again', { kind: 'createIfAbsent' }))
|
||||
.rejects.toMatchObject({ code: 'FS_NOT_OBSERVED' })
|
||||
expect(lockCount(fs)).toBe(0)
|
||||
})
|
||||
})
|
||||
|
||||
describe('read-before-write policy', () => {
|
||||
it('rejects a blind overwrite of an existing file (no prior read)', async () => {
|
||||
await writeFile(join(dir, 'a.txt'), 'old')
|
||||
describe('editText', () => {
|
||||
it('applies a literal edit at the matching version', async () => {
|
||||
await writeFile(join(dir, 'a.txt'), 'hello world')
|
||||
const target = await fs.resolve('a.txt')
|
||||
await expect(fs.write(target, 'new', exec())).rejects.toMatchObject({ code: 'FS_NOT_OBSERVED' })
|
||||
const outcome = await fs.editText(target, { oldString: 'world', newString: 'there', replaceAll: false }, { version: await versionOf(target) })
|
||||
expect(outcome.replacements).toBe(1)
|
||||
expect(await readFile(join(dir, 'a.txt'), 'utf8')).toBe('hello there')
|
||||
})
|
||||
|
||||
it('rejects a write after only a partial read', async () => {
|
||||
await writeFile(join(dir, 'a.txt'), 'one\ntwo')
|
||||
const owner = exec()
|
||||
it('checks the stale version BEFORE literal matching', async () => {
|
||||
await writeFile(join(dir, 'a.txt'), 'hello world')
|
||||
const target = await fs.resolve('a.txt')
|
||||
await fs.read(target, { offset: 1, limit: 1 }, owner)
|
||||
await expect(fs.write(target, 'new', owner)).rejects.toMatchObject({ code: 'FS_PARTIAL_OBSERVATION' })
|
||||
const stale = await versionOf(target)
|
||||
// Change the file so 'world' is gone — a stale edit must report STALE, not NOT_FOUND.
|
||||
await writeFile(join(dir, 'a.txt'), 'goodbye')
|
||||
await expect(fs.editText(target, { oldString: 'world', newString: 'there', replaceAll: false }, { version: stale }))
|
||||
.rejects.toMatchObject({ code: 'FS_STALE_VERSION' })
|
||||
})
|
||||
|
||||
it('rejects a write after a partial read when the file was deleted, without recreating it', async () => {
|
||||
const path = join(dir, 'a.txt')
|
||||
await writeFile(path, 'one\ntwo')
|
||||
const owner = exec()
|
||||
it('rejects a deleted target as stale (before matching)', async () => {
|
||||
await writeFile(join(dir, 'a.txt'), 'hello')
|
||||
const target = await fs.resolve('a.txt')
|
||||
await fs.read(target, { offset: 1, limit: 1 }, owner)
|
||||
await unlink(path)
|
||||
|
||||
await expect(fs.write(target, 'new', owner)).rejects.toMatchObject({ code: 'FS_STALE_VERSION' })
|
||||
await expect(stat(path)).rejects.toMatchObject({ code: 'ENOENT' })
|
||||
const version = await versionOf(target)
|
||||
await unlink(join(dir, 'a.txt'))
|
||||
await expect(fs.editText(target, { oldString: 'hello', newString: 'bye', replaceAll: false }, { version }))
|
||||
.rejects.toMatchObject({ code: 'FS_STALE_VERSION' })
|
||||
})
|
||||
|
||||
it('rejects an edit with no prior read (FS_NOT_OBSERVED)', async () => {
|
||||
await writeFile(join(dir, 'a.txt'), 'old')
|
||||
const target = await fs.resolve('a.txt')
|
||||
await expect(fs.edit(target, { oldString: 'old', newString: 'new', replaceAll: false }, exec()))
|
||||
.rejects.toMatchObject({ code: 'FS_NOT_OBSERVED' })
|
||||
it('rejects a non-regular target', async () => {
|
||||
const target = await fs.resolve('.')
|
||||
await expect(fs.editText(target, { oldString: 'a', newString: 'b', replaceAll: false }, { version: FsVersion('v') }))
|
||||
.rejects.toMatchObject({ code: 'FS_NOT_REGULAR_FILE' })
|
||||
})
|
||||
|
||||
it('rejects invalid UTF-8 reads and edits without rewriting the file', async () => {
|
||||
const path = join(dir, 'invalid-utf8.txt')
|
||||
it('rejects zero matches and ambiguous matches at the right version', async () => {
|
||||
await writeFile(join(dir, 'a.txt'), 'a a a')
|
||||
const target = await fs.resolve('a.txt')
|
||||
const version = await versionOf(target)
|
||||
await expect(fs.editText(target, { oldString: 'z', newString: 'X', replaceAll: false }, { version }))
|
||||
.rejects.toMatchObject({ code: 'FS_EDIT_NOT_FOUND' })
|
||||
await expect(fs.editText(target, { oldString: 'a', newString: 'X', replaceAll: false }, { version }))
|
||||
.rejects.toMatchObject({ code: 'FS_AMBIGUOUS_EDIT' })
|
||||
})
|
||||
|
||||
it('replaces all matches with replaceAll', async () => {
|
||||
await writeFile(join(dir, 'a.txt'), 'a a a')
|
||||
const target = await fs.resolve('a.txt')
|
||||
const outcome = await fs.editText(target, { oldString: 'a', newString: 'b', replaceAll: true }, { version: await versionOf(target) })
|
||||
expect(outcome.replacements).toBe(3)
|
||||
expect(await readFile(join(dir, 'a.txt'), 'utf8')).toBe('b b b')
|
||||
})
|
||||
|
||||
it('rejects invalid UTF-8 without rewriting the file', async () => {
|
||||
const path = join(dir, 'bad.txt')
|
||||
const bytes = Buffer.from([0x68, 0xff, 0x69])
|
||||
await writeFile(path, bytes)
|
||||
const owner = exec()
|
||||
const target = await fs.resolve('invalid-utf8.txt')
|
||||
|
||||
await expect(fs.read(target, READ_ALL, owner)).rejects.toMatchObject({ code: 'FS_NOT_TEXT' })
|
||||
const existing = await probe(target.targetKey)
|
||||
if (!existing) throw new Error('expected invalid UTF-8 fixture to exist')
|
||||
await expect(
|
||||
fs.applyEdit(target, { oldString: 'h', newString: 'H', replaceAll: false }, { version: existing.version }),
|
||||
).rejects.toMatchObject({ code: 'FS_NOT_TEXT' })
|
||||
const target = await fs.resolve('bad.txt')
|
||||
const version = await versionOf(target)
|
||||
await expect(fs.editText(target, { oldString: 'h', newString: 'H', replaceAll: false }, { version }))
|
||||
.rejects.toMatchObject({ code: 'FS_NOT_TEXT' })
|
||||
expect(await readFile(path)).toEqual(bytes)
|
||||
})
|
||||
})
|
||||
|
||||
describe('stale-version guard + concurrency (defensive class B)', () => {
|
||||
it('rejects a write when the file changed since it was read', async () => {
|
||||
await writeFile(join(dir, 'a.txt'), 'v1')
|
||||
const owner = exec()
|
||||
const target = await fs.resolve('a.txt')
|
||||
await fs.read(target, READ_ALL, owner)
|
||||
// An out-of-band change after the read.
|
||||
await writeFile(join(dir, 'a.txt'), 'changed-externally')
|
||||
await expect(fs.write(target, 'v2', owner)).rejects.toMatchObject({ code: 'FS_STALE_VERSION' })
|
||||
})
|
||||
|
||||
it('rejects an observed write when the file was deleted after the read', async () => {
|
||||
await writeFile(join(dir, 'a.txt'), 'v1')
|
||||
const owner = exec()
|
||||
const target = await fs.resolve('a.txt')
|
||||
await fs.read(target, READ_ALL, owner)
|
||||
await unlink(join(dir, 'a.txt')) // file vanishes; observed write must fail (not silently create)
|
||||
await expect(fs.write(target, 'v2', owner)).rejects.toMatchObject({ code: 'FS_STALE_VERSION' })
|
||||
})
|
||||
|
||||
it('two concurrent edits: one wins, the other is rejected as stale', async () => {
|
||||
await writeFile(join(dir, 'a.txt'), 'base')
|
||||
const owner = exec()
|
||||
const target = await fs.resolve('a.txt')
|
||||
await fs.read(target, READ_ALL, owner)
|
||||
// Both edits captured the same recorded version; only one rename can match it.
|
||||
const version = await versionOf(target)
|
||||
const results = await Promise.allSettled([
|
||||
fs.edit(target, { oldString: 'base', newString: 'one', replaceAll: false }, owner),
|
||||
fs.edit(target, { oldString: 'base', newString: 'two', replaceAll: false }, owner),
|
||||
fs.editText(target, { oldString: 'base', newString: 'one', replaceAll: false }, { version }),
|
||||
fs.editText(target, { oldString: 'base', newString: 'two', replaceAll: false }, { version }),
|
||||
])
|
||||
const fulfilled = results.filter(r => r.status === 'fulfilled')
|
||||
expect(results.filter(r => r.status === 'fulfilled')).toHaveLength(1)
|
||||
const rejected = results.filter(r => r.status === 'rejected')
|
||||
expect(fulfilled).toHaveLength(1)
|
||||
expect(rejected).toHaveLength(1)
|
||||
expect((rejected[0] as PromiseRejectedResult).reason).toMatchObject({ code: 'FS_STALE_VERSION' })
|
||||
expect(lockCount(fs)).toBe(0)
|
||||
})
|
||||
})
|
||||
|
||||
describe('symlink targetKey identity (defensive class F)', () => {
|
||||
it('a read via the real path authorizes an edit via the symlink path', async () => {
|
||||
describe('symlink targetKey identity', () => {
|
||||
it('two paths to the same file via a symlink share one version and write the real target', async () => {
|
||||
await writeFile(join(dir, 'real.txt'), 'hello')
|
||||
await symlink(join(dir, 'real.txt'), join(dir, 'link.txt'))
|
||||
const owner = exec()
|
||||
await fs.read(await fs.resolve('real.txt'), READ_ALL, owner)
|
||||
// Edit through the link: same realpath → same targetKey → prior read counts.
|
||||
const linkTarget = await fs.resolve('link.txt')
|
||||
const outcome = await fs.edit(linkTarget, { oldString: 'hello', newString: 'bye', replaceAll: false }, owner)
|
||||
expect(outcome.replacements).toBe(1)
|
||||
expect(await readFile(join(dir, 'real.txt'), 'utf8')).toBe('bye') // link preserved, target written
|
||||
})
|
||||
const viaReal = await fs.resolve('real.txt')
|
||||
const viaLink = await fs.resolve('link.txt')
|
||||
expect(viaLink.targetKey).toBe(viaReal.targetKey)
|
||||
|
||||
it('write through a symlink preserves the link and writes the real target', async () => {
|
||||
await writeFile(join(dir, 'real.txt'), 'hello')
|
||||
await symlink(join(dir, 'real.txt'), join(dir, 'link.txt'))
|
||||
const owner = exec()
|
||||
const linkTarget = await fs.resolve('link.txt')
|
||||
await fs.read(linkTarget, READ_ALL, owner)
|
||||
await fs.write(linkTarget, 'replaced', owner)
|
||||
expect(await readFile(join(dir, 'real.txt'), 'utf8')).toBe('replaced')
|
||||
const version = await versionOf(viaReal)
|
||||
await fs.editText(viaLink, { oldString: 'hello', newString: 'bye', replaceAll: false }, { version })
|
||||
expect(await readFile(join(dir, 'real.txt'), 'utf8')).toBe('bye') // link preserved
|
||||
})
|
||||
|
||||
it('a stale change is detected across both paths', async () => {
|
||||
await writeFile(join(dir, 'real.txt'), 'hello')
|
||||
await symlink(join(dir, 'real.txt'), join(dir, 'link.txt'))
|
||||
const owner = exec()
|
||||
await fs.read(await fs.resolve('real.txt'), READ_ALL, owner)
|
||||
await writeFile(join(dir, 'real.txt'), 'changed') // out-of-band via real path
|
||||
const linkTarget = await fs.resolve('link.txt')
|
||||
await expect(fs.edit(linkTarget, { oldString: 'hello', newString: 'bye', replaceAll: false }, owner))
|
||||
const viaReal = await fs.resolve('real.txt')
|
||||
const stale = await versionOf(viaReal)
|
||||
await writeFile(join(dir, 'real.txt'), 'changed')
|
||||
const viaLink = await fs.resolve('link.txt')
|
||||
await expect(fs.editText(viaLink, { oldString: 'hello', newString: 'bye', replaceAll: false }, { version: stale }))
|
||||
.rejects.toMatchObject({ code: 'FS_STALE_VERSION' })
|
||||
})
|
||||
})
|
||||
|
||||
describe('non-regular targets', () => {
|
||||
it('rejects writing onto a directory', async () => {
|
||||
const target = await fs.resolve('.') // the cwd dir
|
||||
await expect(fs.write(target, 'x', exec())).rejects.toMatchObject({ code: 'FS_NOT_REGULAR_FILE' })
|
||||
})
|
||||
|
||||
it('applyEdit rejects a target that vanished after the read', async () => {
|
||||
await writeFile(join(dir, 'a.txt'), 'hello')
|
||||
const owner = exec()
|
||||
const target = await fs.resolve('a.txt')
|
||||
const version = (await fs.read(target, READ_ALL, owner)).version
|
||||
await unlink(join(dir, 'a.txt'))
|
||||
await expect(fs.applyEdit(target, { oldString: 'hello', newString: 'bye', replaceAll: false }, { version }))
|
||||
.rejects.toMatchObject({ code: 'FS_NOT_FOUND' })
|
||||
})
|
||||
|
||||
it('applyEdit rejects a non-regular target', async () => {
|
||||
const target = await fs.resolve('.')
|
||||
await expect(fs.applyEdit(target, { oldString: 'a', newString: 'b', replaceAll: false }, { version: 'v' }))
|
||||
.rejects.toMatchObject({ code: 'FS_NOT_REGULAR_FILE' })
|
||||
})
|
||||
})
|
||||
|
||||
describe('HMR / disposal (defensive class D)', () => {
|
||||
describe('HMR / disposal', () => {
|
||||
it('disposing the fiber withdraws ctx.fs', async () => {
|
||||
const local = new Context()
|
||||
const fiber = await local.plugin(LocalFileSystem, { cwd: dir })
|
||||
const localFiber = await local.plugin(LocalFileSystem, { cwd: dir })
|
||||
expect(local.fs).toBeDefined()
|
||||
await fiber.dispose()
|
||||
await localFiber.dispose()
|
||||
expect(local.fs).toBeUndefined()
|
||||
})
|
||||
|
||||
it('a fresh provider does not inherit recorded file state', async () => {
|
||||
await writeFile(join(dir, 'a.txt'), 'hello')
|
||||
const local = new Context()
|
||||
const owner = exec()
|
||||
const fiber = await local.plugin(LocalFileSystem, { cwd: dir })
|
||||
await (local.fs as LocalFileSystem).read(await local.fs.resolve('a.txt'), READ_ALL, owner)
|
||||
await fiber.dispose()
|
||||
|
||||
await local.plugin(LocalFileSystem, { cwd: dir })
|
||||
const fs2 = local.fs as LocalFileSystem
|
||||
const target = await fs2.resolve('a.txt')
|
||||
// Same owner object, but state was released on disposal.
|
||||
await expect(fs2.edit(target, { oldString: 'hello', newString: 'bye', replaceAll: false }, owner))
|
||||
.rejects.toMatchObject({ code: 'FS_NOT_OBSERVED' })
|
||||
})
|
||||
})
|
||||
|
||||
@@ -1,24 +1,27 @@
|
||||
/**
|
||||
* Cordis-free tests for the raw local-filesystem I/O: path resolution,
|
||||
* fast/streaming reads, pagination/caps, binary rejection, atomic-write temp
|
||||
* safety, literal edit matching, and line-ending handling.
|
||||
* Cordis-free tests for the raw local-filesystem I/O: path resolution, probe,
|
||||
* whole-file/streamed text reads, binary/UTF-8 rejection, atomic-write temp
|
||||
* safety, literal edit matching, and line-ending handling. Line WINDOWING is
|
||||
* policy and lives in `dsh-file-context`, so it is not tested here.
|
||||
*/
|
||||
|
||||
import { afterEach, beforeEach, describe, expect, it } from 'vitest'
|
||||
import { mkdtemp, readFile, rm, stat, symlink, writeFile, mkdir, readdir } from 'node:fs/promises'
|
||||
import { mkdtemp, readFile, rm, stat, symlink, writeFile, mkdir, readdir, realpath } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
import { createServer } from 'node:net'
|
||||
import {
|
||||
applyLiteralEdit,
|
||||
formatReadBody,
|
||||
probe,
|
||||
readForEdit,
|
||||
readTextPage,
|
||||
readWholeText,
|
||||
resolveLocalTarget,
|
||||
restoreLineEndings,
|
||||
streamWholeText,
|
||||
writeFileAtomic,
|
||||
} from '@deepseek-ai/dsh-fs-local'
|
||||
import type { LocalTarget } from '@deepseek-ai/dsh-fs-local'
|
||||
import { FsTargetKey } from '@deepseek-ai/dsh-fs'
|
||||
|
||||
let dir: string
|
||||
beforeEach(async () => {
|
||||
@@ -28,8 +31,13 @@ afterEach(async () => {
|
||||
await rm(dir, { recursive: true, force: true })
|
||||
})
|
||||
|
||||
const READ_ALL = { offset: 1, limit: 2000 }
|
||||
const localTarget = (path: string): LocalTarget => ({ displayPath: path, targetKey: path })
|
||||
const localTarget = (path: string): LocalTarget => ({ displayPath: path, targetKey: FsTargetKey(path) })
|
||||
|
||||
async function collect(chunks: AsyncIterable<string>): Promise<string> {
|
||||
let out = ''
|
||||
for await (const chunk of chunks) out += chunk
|
||||
return out
|
||||
}
|
||||
|
||||
describe('resolveLocalTarget', () => {
|
||||
it('resolves a relative path from cwd and realpaths it', async () => {
|
||||
@@ -37,11 +45,10 @@ describe('resolveLocalTarget', () => {
|
||||
await writeFile(file, 'hi')
|
||||
const target = await resolveLocalTarget(dir, 'a.txt')
|
||||
expect(target.displayPath).toBe(file)
|
||||
expect(target.targetKey).toBe(await (await import('node:fs/promises')).realpath(file))
|
||||
expect(target.targetKey).toBe(await realpath(file))
|
||||
})
|
||||
|
||||
it('uses the realpathed parent + basename when the file does not exist (stable across create)', async () => {
|
||||
const { realpath } = await import('node:fs/promises')
|
||||
const target = await resolveLocalTarget(dir, 'missing.txt')
|
||||
expect(target.targetKey).toBe(join(await realpath(dir), 'missing.txt'))
|
||||
})
|
||||
@@ -67,186 +74,104 @@ describe('resolveLocalTarget', () => {
|
||||
})
|
||||
})
|
||||
|
||||
describe('readTextPage', () => {
|
||||
it('reads a small file with line numbers and full view', async () => {
|
||||
describe('probe', () => {
|
||||
it('returns null for a missing path and metadata for a file', async () => {
|
||||
expect(await probe(join(dir, 'nope'))).toBeNull()
|
||||
const file = join(dir, 'a.txt')
|
||||
await writeFile(file, 'hi')
|
||||
const info = await probe(file)
|
||||
expect(info?.type).toBe('file')
|
||||
expect(info?.size).toBe(2)
|
||||
expect(typeof info?.version).toBe('string')
|
||||
})
|
||||
|
||||
it('reports a directory and a non-regular type', async () => {
|
||||
const sub = join(dir, 'sub')
|
||||
await mkdir(sub)
|
||||
expect((await probe(sub))?.type).toBe('directory')
|
||||
})
|
||||
|
||||
it('reports a socket/special file as type "other"', async () => {
|
||||
const sockPath = join(dir, 'sock')
|
||||
const server = createServer()
|
||||
await new Promise<void>((resolve) => { server.listen(sockPath, () => { resolve() }) })
|
||||
try {
|
||||
expect((await probe(sockPath))?.type).toBe('other')
|
||||
} finally {
|
||||
await new Promise<void>((resolve) => { server.close(() => { resolve() }) })
|
||||
}
|
||||
})
|
||||
})
|
||||
|
||||
describe('readWholeText', () => {
|
||||
it('reads a small file', async () => {
|
||||
const file = join(dir, 'a.txt')
|
||||
await writeFile(file, 'one\ntwo\nthree')
|
||||
const result = await readTextPage(localTarget(file), READ_ALL)
|
||||
expect(result.lines).toEqual([
|
||||
{ number: 1, text: 'one' },
|
||||
{ number: 2, text: 'two' },
|
||||
{ number: 3, text: 'three' },
|
||||
])
|
||||
expect(result.totalLines).toBe(3)
|
||||
expect(result.view).toBe('full')
|
||||
})
|
||||
|
||||
it('paginates with offset/limit and reports a partial view', async () => {
|
||||
const file = join(dir, 'a.txt')
|
||||
await writeFile(file, 'one\ntwo\nthree\nfour')
|
||||
const result = await readTextPage(localTarget(file), { offset: 2, limit: 2 })
|
||||
expect(result.lines.map(l => l.number)).toEqual([2, 3])
|
||||
expect(result.view).toBe('partial')
|
||||
expect(formatReadBody(result, 2)).toContain('(Showing lines 2-3 of 4. Use offset=4 to continue.)')
|
||||
})
|
||||
|
||||
it('a whole-file read from offset 1 is a full view; offset>1 is partial', async () => {
|
||||
const file = join(dir, 'a.txt')
|
||||
await writeFile(file, 'one\ntwo')
|
||||
expect((await readTextPage(localTarget(file), { offset: 1, limit: 10 })).view).toBe('full')
|
||||
expect((await readTextPage(localTarget(file), { offset: 2, limit: 10 })).view).toBe('partial')
|
||||
})
|
||||
|
||||
it('truncates an over-long line', async () => {
|
||||
const file = join(dir, 'long.txt')
|
||||
await writeFile(file, 'x'.repeat(3000))
|
||||
const result = await readTextPage(localTarget(file), READ_ALL)
|
||||
expect(result.lines[0]?.text).toContain('... (line truncated to 2000 chars)')
|
||||
expect(result.view).toBe('partial')
|
||||
})
|
||||
|
||||
it('caps output bytes and reports truncatedByBytes', async () => {
|
||||
const file = join(dir, 'big.txt')
|
||||
const lines = Array.from({ length: 2000 }, () => 'y'.repeat(100))
|
||||
await writeFile(file, lines.join('\n'))
|
||||
const result = await readTextPage(localTarget(file), READ_ALL)
|
||||
expect(result.truncatedByBytes).toBe(true)
|
||||
expect(formatReadBody(result, 1)).toContain('Output capped at 50 KB')
|
||||
})
|
||||
|
||||
it('strips CRLF so a Windows file reads like LF', async () => {
|
||||
const file = join(dir, 'crlf.txt')
|
||||
await writeFile(file, 'one\r\ntwo\r\n')
|
||||
const result = await readTextPage(localTarget(file), READ_ALL)
|
||||
expect(result.lines.map(l => l.text)).toEqual(['one', 'two'])
|
||||
})
|
||||
|
||||
it('reads an empty file at offset 1', async () => {
|
||||
const file = join(dir, 'empty.txt')
|
||||
await writeFile(file, '')
|
||||
const result = await readTextPage(localTarget(file), READ_ALL)
|
||||
expect(result.lines).toEqual([])
|
||||
expect(result.totalLines).toBe(0)
|
||||
expect(formatReadBody(result, 1)).toBe('(End of file - total 0 lines)')
|
||||
})
|
||||
|
||||
it('rejects an offset past EOF', async () => {
|
||||
const file = join(dir, 'a.txt')
|
||||
await writeFile(file, 'one\ntwo')
|
||||
await expect(readTextPage(localTarget(file), { offset: 9, limit: 1 })).rejects.toMatchObject({ code: 'FS_NOT_FOUND' })
|
||||
})
|
||||
|
||||
it('rejects a binary file (fast path)', async () => {
|
||||
const file = join(dir, 'bin')
|
||||
await writeFile(file, Buffer.from([0x68, 0x00, 0x69]))
|
||||
await expect(readTextPage(localTarget(file), READ_ALL)).rejects.toMatchObject({ code: 'FS_NOT_TEXT' })
|
||||
})
|
||||
|
||||
it('rejects invalid UTF-8 bytes (fast path)', async () => {
|
||||
const file = join(dir, 'invalid-utf8.txt')
|
||||
await writeFile(file, Buffer.from([0x68, 0xff, 0x69]))
|
||||
await expect(readTextPage(localTarget(file), READ_ALL)).rejects.toMatchObject({ code: 'FS_NOT_TEXT' })
|
||||
expect(await readWholeText(localTarget(file))).toBe('one\ntwo\nthree')
|
||||
})
|
||||
|
||||
it('rejects a missing file and a directory', async () => {
|
||||
await expect(readTextPage(localTarget(join(dir, 'nope')), READ_ALL)).rejects.toMatchObject({ code: 'FS_NOT_FOUND' })
|
||||
await expect(readTextPage(localTarget(dir), READ_ALL)).rejects.toMatchObject({ code: 'FS_NOT_REGULAR_FILE' })
|
||||
await expect(readWholeText(localTarget(join(dir, 'nope')))).rejects.toMatchObject({ code: 'FS_NOT_FOUND' })
|
||||
await expect(readWholeText(localTarget(dir))).rejects.toMatchObject({ code: 'FS_NOT_REGULAR_FILE' })
|
||||
})
|
||||
|
||||
it('rejects binary and invalid UTF-8', async () => {
|
||||
await writeFile(join(dir, 'bin'), Buffer.from([0x68, 0x00, 0x69]))
|
||||
await expect(readWholeText(localTarget(join(dir, 'bin')))).rejects.toMatchObject({ code: 'FS_NOT_TEXT' })
|
||||
await writeFile(join(dir, 'bad'), Buffer.from([0x68, 0xff, 0x69]))
|
||||
await expect(readWholeText(localTarget(join(dir, 'bad')))).rejects.toMatchObject({ code: 'FS_NOT_TEXT' })
|
||||
})
|
||||
|
||||
it('honors a pre-aborted signal', async () => {
|
||||
const file = join(dir, 'a.txt')
|
||||
await writeFile(file, 'one')
|
||||
await expect(readTextPage(localTarget(file), READ_ALL, AbortSignal.abort())).rejects.toMatchObject({ code: 'FS_ABORTED' })
|
||||
await expect(readWholeText(localTarget(file), AbortSignal.abort())).rejects.toMatchObject({ code: 'FS_ABORTED' })
|
||||
})
|
||||
|
||||
it('passes a live (non-aborted) signal through the fast path', async () => {
|
||||
it('passes a live (non-aborted) signal through', async () => {
|
||||
const file = join(dir, 'a.txt')
|
||||
await writeFile(file, 'one\ntwo')
|
||||
const result = await readTextPage(localTarget(file), READ_ALL, new AbortController().signal)
|
||||
expect(result.totalLines).toBe(2)
|
||||
})
|
||||
|
||||
describe('streaming path (forced via a tiny fastPathMaxSize)', () => {
|
||||
const stream = { fastPathMaxSize: 1 }
|
||||
|
||||
it('reads and paginates large files the same way', async () => {
|
||||
const file = join(dir, 'a.txt')
|
||||
await writeFile(file, 'one\ntwo\nthree')
|
||||
const result = await readTextPage(localTarget(file), { offset: 2, limit: 1 }, undefined, stream)
|
||||
expect(result.lines).toEqual([{ number: 2, text: 'two' }])
|
||||
expect(result.totalLines).toBe(3)
|
||||
})
|
||||
|
||||
it('rejects a binary file on the streaming path', async () => {
|
||||
const file = join(dir, 'bin')
|
||||
await writeFile(file, Buffer.from([0x68, 0x00, 0x69]))
|
||||
await expect(readTextPage(localTarget(file), READ_ALL, undefined, stream)).rejects.toMatchObject({ code: 'FS_NOT_TEXT' })
|
||||
})
|
||||
|
||||
it('caps a newline-free giant line without unbounded buffering', async () => {
|
||||
const file = join(dir, 'one-line.txt')
|
||||
await writeFile(file, 'z'.repeat(5000))
|
||||
const result = await readTextPage(localTarget(file), READ_ALL, undefined, stream)
|
||||
expect(result.lines[0]?.text).toContain('... (line truncated to 2000 chars)')
|
||||
expect(result.view).toBe('partial')
|
||||
})
|
||||
|
||||
it('rejects invalid UTF-8 bytes on the streaming path', async () => {
|
||||
const file = join(dir, 'invalid-utf8.txt')
|
||||
await writeFile(file, Buffer.from([0x68, 0xff, 0x69]))
|
||||
await expect(readTextPage(localTarget(file), READ_ALL, undefined, stream)).rejects.toMatchObject({ code: 'FS_NOT_TEXT' })
|
||||
})
|
||||
|
||||
it('honors abort on the streaming path', async () => {
|
||||
const file = join(dir, 'a.txt')
|
||||
await writeFile(file, 'one\ntwo')
|
||||
await expect(readTextPage(localTarget(file), READ_ALL, AbortSignal.abort(), stream)).rejects.toMatchObject({ code: 'FS_ABORTED' })
|
||||
})
|
||||
|
||||
it('caps output bytes mid-stream', async () => {
|
||||
const file = join(dir, 'big.txt')
|
||||
await writeFile(file, Array.from({ length: 2000 }, () => 'y'.repeat(100)).join('\n'))
|
||||
const result = await readTextPage(localTarget(file), READ_ALL, undefined, stream)
|
||||
expect(result.truncatedByBytes).toBe(true)
|
||||
})
|
||||
|
||||
it('flushes a final line with no trailing newline', async () => {
|
||||
const file = join(dir, 'no-nl.txt')
|
||||
await writeFile(file, 'one\ntwo') // no trailing \n
|
||||
const result = await readTextPage(localTarget(file), READ_ALL, undefined, stream)
|
||||
expect(result.lines.map(l => l.text)).toEqual(['one', 'two'])
|
||||
})
|
||||
|
||||
it('handles a trailing newline (no dangling buffer at EOF)', async () => {
|
||||
const file = join(dir, 'nl.txt')
|
||||
await writeFile(file, 'one\ntwo\n') // trailing \n → empty buffer at end
|
||||
const result = await readTextPage(localTarget(file), READ_ALL, undefined, stream)
|
||||
expect(result.lines.map(l => l.text)).toEqual(['one', 'two'])
|
||||
expect(result.totalLines).toBe(2)
|
||||
})
|
||||
|
||||
it('passes a live (non-aborted) signal through to the stream', async () => {
|
||||
const file = join(dir, 'a.txt')
|
||||
await writeFile(file, 'one\ntwo')
|
||||
const result = await readTextPage(localTarget(file), READ_ALL, new AbortController().signal, stream)
|
||||
expect(result.totalLines).toBe(2)
|
||||
})
|
||||
|
||||
it('scans across multiple stream chunks', async () => {
|
||||
// A file well past the default 64 KB stream highWaterMark yields multiple chunks,
|
||||
// exercising the non-first-chunk branch and the line-buffer cap across appends.
|
||||
const file = join(dir, 'multi.txt')
|
||||
const lines = Array.from({ length: 50 }, (_, i) => `line ${i}: ${'x'.repeat(3000)}`)
|
||||
await writeFile(file, lines.join('\n'))
|
||||
const result = await readTextPage(localTarget(file), { offset: 1, limit: 3 }, undefined, stream)
|
||||
expect(result.lines[0]?.text.startsWith('line 0:')).toBe(true)
|
||||
expect(result.lines[0]?.text).toContain('... (line truncated to 2000 chars)')
|
||||
expect(result.totalLines).toBeGreaterThanOrEqual(3)
|
||||
})
|
||||
expect(await readWholeText(localTarget(file), new AbortController().signal)).toBe('one\ntwo')
|
||||
})
|
||||
})
|
||||
|
||||
describe('writeFileAtomic — temp-file safety (defensive class A)', () => {
|
||||
describe('streamWholeText', () => {
|
||||
it('streams the whole file as decoded text', async () => {
|
||||
const file = join(dir, 'a.txt')
|
||||
await writeFile(file, 'one\ntwo\nthree')
|
||||
expect(await collect(streamWholeText(localTarget(file)))).toBe('one\ntwo\nthree')
|
||||
})
|
||||
|
||||
it('streams a large multi-chunk file correctly', async () => {
|
||||
const file = join(dir, 'big.txt')
|
||||
const content = Array.from({ length: 50 }, (_, i) => `line ${i}: ${'x'.repeat(3000)}`).join('\n')
|
||||
await writeFile(file, content)
|
||||
expect(await collect(streamWholeText(localTarget(file)))).toBe(content)
|
||||
})
|
||||
|
||||
it('rejects a missing file, directory, binary, and invalid UTF-8', async () => {
|
||||
await expect(collect(streamWholeText(localTarget(join(dir, 'nope'))))).rejects.toMatchObject({ code: 'FS_NOT_FOUND' })
|
||||
await expect(collect(streamWholeText(localTarget(dir)))).rejects.toMatchObject({ code: 'FS_NOT_REGULAR_FILE' })
|
||||
await writeFile(join(dir, 'bin'), Buffer.from([0x68, 0x00, 0x69]))
|
||||
await expect(collect(streamWholeText(localTarget(join(dir, 'bin'))))).rejects.toMatchObject({ code: 'FS_NOT_TEXT' })
|
||||
await writeFile(join(dir, 'bad'), Buffer.from([0x68, 0xff, 0x69]))
|
||||
await expect(collect(streamWholeText(localTarget(join(dir, 'bad'))))).rejects.toMatchObject({ code: 'FS_NOT_TEXT' })
|
||||
})
|
||||
|
||||
it('honors a pre-aborted signal', async () => {
|
||||
const file = join(dir, 'a.txt')
|
||||
await writeFile(file, 'one')
|
||||
await expect(collect(streamWholeText(localTarget(file), AbortSignal.abort()))).rejects.toMatchObject({ code: 'FS_ABORTED' })
|
||||
})
|
||||
|
||||
it('passes a live (non-aborted) signal through the stream', async () => {
|
||||
const file = join(dir, 'a.txt')
|
||||
await writeFile(file, 'one\ntwo')
|
||||
expect(await collect(streamWholeText(localTarget(file), new AbortController().signal))).toBe('one\ntwo')
|
||||
})
|
||||
})
|
||||
|
||||
describe('writeFileAtomic — temp-file safety', () => {
|
||||
it('writes through a private staging dir and owner-only temp file', async () => {
|
||||
const file = join(dir, 'a.txt')
|
||||
let inspected = false
|
||||
@@ -259,8 +184,7 @@ describe('writeFileAtomic — temp-file safety (defensive class A)', () => {
|
||||
})
|
||||
expect(inspected).toBe(true)
|
||||
expect(await readFile(file, 'utf8')).toBe('hello')
|
||||
const info = await stat(file)
|
||||
expect(info.mode & 0o777).toBe(0o640)
|
||||
expect((await stat(file)).mode & 0o777).toBe(0o640)
|
||||
expect((await readdir(dir)).filter(n => n.includes('.tmp'))).toEqual([])
|
||||
})
|
||||
|
||||
@@ -278,7 +202,6 @@ describe('writeFileAtomic — temp-file safety (defensive class A)', () => {
|
||||
await expect(
|
||||
writeFileAtomic(file, 'hello', undefined, undefined, { tempDirName: () => tempDirName }),
|
||||
).rejects.toMatchObject({ code: 'EEXIST' })
|
||||
// The pre-existing staging dir is intact and the target was not created.
|
||||
expect(await readFile(join(dir, tempDirName, 'PRECIOUS'), 'utf8')).toBe('keep')
|
||||
await expect(stat(file)).rejects.toMatchObject({ code: 'ENOENT' })
|
||||
})
|
||||
@@ -303,9 +226,8 @@ describe('writeFileAtomic — temp-file safety (defensive class A)', () => {
|
||||
|
||||
it('cleans up the temp file when the final rename fails', async () => {
|
||||
const sub = join(dir, 'occupied')
|
||||
await mkdir(sub) // rename(temp, sub) fails because sub is a non-empty/dir target
|
||||
await mkdir(sub)
|
||||
await expect(writeFileAtomic(sub, 'hi', undefined, undefined)).rejects.toBeInstanceOf(Error)
|
||||
// No leftover staging dirs in the directory.
|
||||
expect((await readdir(dir)).filter(n => n.includes('.tmp'))).toEqual([])
|
||||
})
|
||||
})
|
||||
@@ -346,16 +268,11 @@ describe('readForEdit + restoreLineEndings', () => {
|
||||
expect(restoreLineEndings(edited.content, original.lineEndings)).toBe('one\r\nTWO\r\n')
|
||||
})
|
||||
|
||||
it('rejects a binary file', async () => {
|
||||
const file = join(dir, 'bin')
|
||||
await writeFile(file, Buffer.from([0x00, 0x01]))
|
||||
await expect(readForEdit(file, file)).rejects.toMatchObject({ code: 'FS_NOT_TEXT' })
|
||||
})
|
||||
|
||||
it('rejects invalid UTF-8 bytes', async () => {
|
||||
const file = join(dir, 'invalid-utf8.txt')
|
||||
await writeFile(file, Buffer.from([0x68, 0xff, 0x69]))
|
||||
await expect(readForEdit(file, file)).rejects.toMatchObject({ code: 'FS_NOT_TEXT' })
|
||||
it('rejects a binary file and invalid UTF-8', async () => {
|
||||
await writeFile(join(dir, 'bin'), Buffer.from([0x00, 0x01]))
|
||||
await expect(readForEdit(join(dir, 'bin'), join(dir, 'bin'))).rejects.toMatchObject({ code: 'FS_NOT_TEXT' })
|
||||
await writeFile(join(dir, 'bad'), Buffer.from([0x68, 0xff, 0x69]))
|
||||
await expect(readForEdit(join(dir, 'bad'), join(dir, 'bad'))).rejects.toMatchObject({ code: 'FS_NOT_TEXT' })
|
||||
})
|
||||
|
||||
it('passes a live (non-aborted) signal through the read', async () => {
|
||||
@@ -365,20 +282,3 @@ describe('readForEdit + restoreLineEndings', () => {
|
||||
expect(original.content).toBe('one\ntwo')
|
||||
})
|
||||
})
|
||||
|
||||
describe('probe', () => {
|
||||
it('returns null for a missing path and info for a file', async () => {
|
||||
expect(await probe(join(dir, 'nope'))).toBeNull()
|
||||
const file = join(dir, 'a.txt')
|
||||
await writeFile(file, 'hi')
|
||||
const info = await probe(file)
|
||||
expect(info?.isFile).toBe(true)
|
||||
expect(typeof info?.version).toBe('string')
|
||||
})
|
||||
|
||||
it('marks a directory as not a regular file', async () => {
|
||||
const sub = join(dir, 'sub')
|
||||
await mkdir(sub)
|
||||
expect((await probe(sub))?.isFile).toBe(false)
|
||||
})
|
||||
})
|
||||
|
||||
Reference in New Issue
Block a user