diff --git a/.agents/notes/implemented/feature/2026-08-15-desktop-harness-self-contained-links.md b/.agents/notes/implemented/feature/2026-08-15-desktop-harness-self-contained-links.md new file mode 100644 index 0000000000..e617d8ca60 --- /dev/null +++ b/.agents/notes/implemented/feature/2026-08-15-desktop-harness-self-contained-links.md @@ -0,0 +1,104 @@ +# Agent Note: Self-contained harness links for the desktop bundle + +Status: implemented + +English | [中文](2026-08-15-desktop-harness-self-contained-links.zh.md) + +## Problem + +The desktop app promises to run detached from any development environment: a +user installs the `.dmg`/`.nsis`, double-clicks, and gets a working `dsh`. That +promise silently breaks on **Windows**: `scripts/build-harness.mjs` copies the +repo's working `node_modules` (plus `vendor/packages/native/apps`) into +`build/harness` with `cpSync(..., { recursive: true })`, and Node's copy with the +default `dereference: false` recreates every symlink **verbatim** — target path +string and all. + +A pnpm workspace is held together by links inside `node_modules`. Without +Windows Developer Mode, pnpm records workspace links as **junctions**, which +store the **absolute path of the build machine's dev tree** (e.g. +`C:\Users\…\deepseek-harness\packages\core\dsh-core`). Copied verbatim, the +packaged harness keeps pointing at the dev tree: it reads/writes **development** +resources when run on the build machine (silently "working" — the reported +symptom) and its links dangle on any other machine, so the app cannot start. + +Why the build machine cannot catch this by running the app: the dev path exists +locally, so a junction always resolves. The only reliable check is a static link +walk that asserts every link target resolves inside the bundle, plus booting the +harness straight from the bundle. + +## Decision + +Make `build/harness` genuinely self-contained, in `apps/desktop`: + +- **`scripts/harness-relink.mjs`** (new, pure, unit-tested): walks the assembled + harness and rewrites every symlink/junction to a **relative target inside the + harness**, mirroring the dev layout (`repoRoot → build/harness`). Targets + inside the harness are left alone (idempotent); targets inside the repo are + mirrored and relinked; a target whose directory contains the bundle, or whose + in-tree target is absent (a workspace package the harness does not ship, e.g. + the desktop shell itself or a dependency-resolution member like + `python/sdk-runtime`), is **dropped** — it cannot be mirrored and is not part + of the runtime closure; genuinely external targets are dereferenced (copied in + as real files). `build-harness.mjs` re-runs the pass until no link changes + (rewriting one link can expose another). `findEscapingLinks`/ + `assertSelfContained` prove the invariant: **no link may resolve outside the + bundle or dangle** — the build fails otherwise. `probeSymlinkSupport` detects + whether the host can create directory symlinks. +- **`scripts/build-harness.mjs`**: refuses when a critical dir is missing + (`node_modules` etc.) or the CLI/web dist is unbuilt (previously a silent skip + shipped a broken harness); probes symlink capability and errors with guidance + (Windows Developer Mode / admin), with a `--dereference-ok` escape hatch for + locked-down hosts; after copying it runs the relink pass, `assertSelfContained`, + and a `dsh --version` smoke test. +- **`scripts/verify-harness.mjs`** (new): reusable verifier. Checks + `build/harness`, or with `--app ` a packed/unpacked app's + `resources/harness` (covering electron-builder's `extraResources` copy); + `--boot` spawns the web profile straight from the bundle (temp `DSH_HOME`, + telemetry off) and waits for its readiness line. +- **Wiring**: root `desktop:pack` now runs `build:harness` automatically (a stale + or missing runtime cannot be shipped); new `desktop:verify` = + `electron-builder --dir` + the verifier against the packed app. + +Requiring symlink capability (rather than silently dereferencing) keeps the +runtime compact: dereferencing the whole pnpm store would balloon the ≈2 GB +harness to many times its size, since every shared dependency would be copied +once per consumer. + +## Verification + +- `apps/desktop/tests/harness-relink.spec.ts` (vitest, cross-platform): relative + in-tree link untouched; absolute/junction link rewritten to a relative in-tree + link; `.pnpm` store mirror; external target dereferenced into real content; + dangling link reported; `assertSelfContained` throws on escaping links; + `probeSymlinkSupport` boolean. +- Standalone probe on Windows (admin): absolute junction → relative in-tree + symlink, external target → real files, `findEscapingLinks === []`. +- Per-platform release gate: `pnpm desktop:pack` (must print relink stats + + "self-contained"), then `pnpm desktop:verify --boot` on **both** Windows and + macOS (on macOS the relink pass is mostly a no-op because pnpm uses relative + symlinks; the check still proves the packaged copy is intact). The static link + walk is authoritative on the build machine, where the dev path exists. + +## Alternatives considered + +- **`cpSync(..., { dereference: true })` (dereference everything).** Rejected: + breaks hardlink dedup across the pnpm store, ballooning the ≈2 GB runtime to + many times its size. +- **Fix at install time (force relative symlinks).** Rejected: pnpm's + junction-vs-symlink choice is machine-dependent and not reliably controllable. +- **Runtime self-heal in the packaged app (re-point broken junctions to the + app's own resources).** Rejected: junctions are always absolute, so a portable + rewrite needs real relative symlinks on the target machine too — no simpler + than fixing the bundle at build time, and it ships complexity into the app. + +## Consequences + +- **Costs:** building on Windows requires Developer Mode or an admin shell; + `build:harness` takes longer (relink walk + smoke); release notes must mention + the capability requirement. +- **Buys:** a packaged app whose `resources/harness` provably references nothing + outside the bundle — the "detached from the development environment" guarantee + that is a hard requirement for production distribution. The existing + `asar: false` / `npmRebuild: false` and the manual-download update flow are + unchanged. diff --git a/.agents/notes/implemented/feature/2026-08-15-desktop-harness-self-contained-links.zh.md b/.agents/notes/implemented/feature/2026-08-15-desktop-harness-self-contained-links.zh.md new file mode 100644 index 0000000000..044bbd12d0 --- /dev/null +++ b/.agents/notes/implemented/feature/2026-08-15-desktop-harness-self-contained-links.zh.md @@ -0,0 +1,82 @@ +# Agent Note:桌面 bundle 的 harness 链接自包含化 + +Status: implemented + +English | [中文](2026-08-15-desktop-harness-self-contained-links.md) + +## 问题 + +桌面应用承诺脱离任何开发环境即可运行:用户装好 `.dmg`/`.nsis`,双击即得到可用的 +`dsh`。这一承诺在 **Windows** 上会静默失效:`scripts/build-harness.mjs` 用 +`cpSync(..., { recursive: true })` 把仓库工作树的 `node_modules`(连同 +`vendor/packages/native/apps`)复制进 `build/harness`,而 Node 复制在默认 +`dereference: false` 下会**逐字**重建每个符号链接——连同目标路径字符串。 + +pnpm workspace 靠 `node_modules` 内的链接维系。在未开启 Windows 开发者模式时, +pnpm 会把 workspace 链接记录为 **junction**,其中保存的是**构建机开发树的绝对路径** +(如 `C:\Users\…\deepseek-harness\packages\core\dsh-core`)。逐字复制后,打包出的 +harness 仍然指向开发树:在本机构建机上运行时会读写**开发**资源("看似正常"——即 +被报告的现场),换到任何其他机器则链接悬空、应用无法启动。 + +为什么构建机上"跑起来"证明不了正确性:本机开发路径存在,junction 总能解析。唯一 +可靠的检查是静态链接巡检——断言每个链接目标都解析到 bundle 内部——再加上从 +bundle 直接启动 harness 的冒烟测试。 + +## 决策 + +让 `build/harness` 真正做到自包含,改动全部在 `apps/desktop` 内: + +- **`scripts/harness-relink.mjs`**(新增、纯逻辑、单测覆盖):遍历组装好的 + harness,把每个符号链接/junction 改写为**指向 harness 内部的相对目标**,镜像 + 开发布局(`repoRoot → build/harness`)。目标已在 harness 内则保持不变(幂等); + 目标在仓库内则镜像后重链;目标所在目录包含 bundle 本身、或树内目标缺失(harness + 不随附的工作区包,如桌面外壳自身或 `python/sdk-runtime` 等依赖解析成员)则 + **丢弃**——无法镜像且不属于运行时闭包;真正外部目标则解引用(复制为真实文件)。 + `build-harness.mjs` 会重跑该过程直到不再有链接变化(改写一个链接可能暴露另一个)。 + `findEscapingLinks`/`assertSelfContained` 保证不变式:**任何链接都不得解析到 + bundle 之外或悬空**——否则构建失败。`probeSymlinkSupport` 探测宿主能否创建目录 + 符号链接。 +- **`scripts/build-harness.mjs`**:关键目录缺失(`node_modules` 等)或 CLI/web + dist 未构建时直接报错(此前静默跳过会打出坏包);探测符号链接能力,不具备时给出 + 指引(Windows 开发者模式 / 管理员)并以 `--dereference-ok` 作为锁死机器的逃生舱; + 复制完成后执行 relink、`assertSelfContained` 与 `dsh --version` 冒烟。 +- **`scripts/verify-harness.mjs`**(新增):可复用验证器。默认校验 `build/harness`, + 带 `--app ` 时校验打包/未打包 app 内的 `resources/harness`(覆盖 + electron-builder 的 `extraResources` 复制);`--boot` 直接从 bundle 拉起 web + profile(临时 `DSH_HOME`、关闭遥测)并等待就绪行。 +- **接线**:根 `desktop:pack` 现在自动执行 `build:harness`(杜绝漏打运行时); + 新增 `desktop:verify` = `electron-builder --dir` + 对打包产物的验证器。 + +要求具备符号链接能力(而非静默解引用)是为了保持运行时体积:把整个 pnpm store +解引用会把约 2GB 的 harness 放大数倍——每个共享依赖都会被按消费者数量复制一份。 + +## 验证 + +- `apps/desktop/tests/harness-relink.spec.ts`(vitest,跨平台):树内相对链接不动; + 绝对/junction 链接改写为树内相对链接;`.pnpm` store 镜像;外部目标解引用为真实 + 内容;悬空链接被报告;`assertSelfContained` 对越界链接抛错;`probeSymlinkSupport` + 返回布尔。 +- Windows(管理员)独立探针:绝对 junction → 树内相对符号链接、外部目标 → 真实 + 文件、`findEscapingLinks === []`。 +- 每平台发布门禁:`pnpm desktop:pack`(须打印 relink 统计与 "self-contained"), + 再在 **Windows 与 macOS 双端**跑 `pnpm desktop:verify --boot`(macOS 上 relink + 多为空操作,因为 pnpm 用相对符号链接,但检查仍证明打包副本完好)。在构建机上开发 + 路径存在,因此**静态链接巡检**是权威检查。 + +## 备选方案 + +- **`cpSync(..., { dereference: true })`(全量解引用)。** 否决:破坏 pnpm store + 的硬链接去重,约 2GB 运行时被放大数倍。 +- **安装期修复(强制相对符号链接)。** 否决:pnpm 在 junction 与符号链接之间的选择 + 依机器而定,无法可靠控制。 +- **打包后运行时自愈(把坏 junction 改指向 app 自身 resources)。** 否决:junction + 天生是绝对路径,可移植改写仍需目标机具备创建相对符号链接的能力——并不比在构建期 + 修 bundle 更简单,还会把复杂度送进应用内。 + +## 后果 + +- **代价:** Windows 上构建需要开发者模式或管理员 shell;`build:harness` 变慢 + (relink 巡检 + 冒烟);发布说明需提及这一能力要求。 +- **收益:** 打包应用的 `resources/harness` 可被证明不引用 bundle 之外任何路径—— + "脱离开发环境可用"这一生产分发的硬性前提得到保证。现有 `asar: false` / + `npmRebuild: false` 与手动下载更新流均不变。 diff --git a/README.md b/README.md index 5524c00daf..85196c36a5 100644 --- a/README.md +++ b/README.md @@ -108,8 +108,8 @@ pnpm desktop:dev # opens the desktop window ### Packaging a release ```sh -pnpm --filter @deepseek-ai/dsh-desktop run build:harness # assemble the self-contained runtime -pnpm desktop:pack # electron-builder: .dmg / .nsis +pnpm desktop:pack # build lib+web, assemble the self-contained harness, then electron-builder: .dmg / .nsis +pnpm desktop:verify # prove the packaged app's harness is self-contained and boots pnpm --filter @deepseek-ai/dsh-desktop run stage-release # copy installers + manifest into the publish site ``` diff --git a/README.zh.md b/README.zh.md index 18ea194e97..a1fb7ff23c 100644 --- a/README.zh.md +++ b/README.zh.md @@ -102,8 +102,8 @@ pnpm desktop:dev # 打开桌面窗口 ### 打包发布 ```sh -pnpm --filter @deepseek-ai/dsh-desktop run build:harness # 组装自包含运行时 -pnpm desktop:pack # electron-builder:.dmg / .nsis +pnpm desktop:pack # 构建 lib+web,组装自包含 harness,再 electron-builder:.dmg / .nsis +pnpm desktop:verify # 证明打包后 app 内的 harness 完全自包含且能启动 pnpm --filter @deepseek-ai/dsh-desktop run stage-release # 将安装包 + 清单拷入发布站 ``` diff --git a/apps/desktop/README.md b/apps/desktop/README.md index 7e375bc2c3..402e4ea254 100644 --- a/apps/desktop/README.md +++ b/apps/desktop/README.md @@ -101,11 +101,17 @@ override the spawned Node or `dsh` entry with `DSH_NODE` / `DSH_ENTRY`. ## Packaging ```sh -pnpm --filter @deepseek-ai/dsh-desktop run build:harness # (re)assemble the bundled runtime -pnpm desktop:pack # electron-builder: mac .dmg / win .nsis +pnpm desktop:pack # build lib+web, assemble the self-contained harness, then electron-builder: mac .dmg / win .nsis +pnpm desktop:verify # prove the packaged app's harness is self-contained and boots ``` -Artifacts land in `apps/desktop/dist`. +`desktop:pack` runs `build:harness` (assembling `build/harness`) automatically, +so a stale or missing runtime can never be shipped. Artifacts land in +`apps/desktop/dist`. `desktop:verify` rebuilds an unpacked app +(`electron-builder --dir`) and checks that every symlink inside the packaged +`resources/harness` resolves inside the bundle, then boots the web profile from +it — the check that the installer is genuinely detached from the development +machine. ### The self-contained harness (`build/harness`) @@ -145,7 +151,9 @@ Each target platform needs its own harness: the bundled `bin/node` and the native addons are OS/arch-specific. Regenerate `build/harness` on each target platform (or per-target in CI) before packaging that platform. The current configuration targets **macOS arm64** (`mac.target: dmg`); `win.target: nsis` -is declared but needs a Windows-built harness. +is declared but needs a Windows-built harness. On Windows the harness build +additionally needs Developer Mode enabled or an admin shell so workspace links +can be rewritten to relative in-tree symlinks (see *Link correctness*). ### Signing @@ -161,6 +169,38 @@ The bundled harness is multi-GB uncompressed (≈2 GB), dominated by ~1 GB. This is the inherent footprint of shipping the full harness runtime standalone, and is the accepted trade-off for a zero-external-dependency app. +### Link correctness + +The harness is held together by symlinks inside `node_modules` (pnpm workspace +links). On **Windows**, pnpm often records those as **junctions — absolute-path +reparse points pointing at the build machine's dev tree**. A plain directory +copy keeps them verbatim, so a packaged app built that way reads the +*development* tree when run on the build machine and dangles on any other +machine. + +`scripts/build-harness.mjs` therefore runs a relink pass +(`scripts/harness-relink.mjs`) after copying, re-running it until no link changes +(rewriting one link can expose another, so a single pass may not converge): +every symlink/junction is rewritten to a **relative target inside +`build/harness`** (mirroring the dev layout), any genuinely external target is +dereferenced (copied in as real files), and a link to a workspace package the +harness does not ship (e.g. the desktop shell itself, or a dependency-resolution +member like `python/sdk-runtime`) is **dropped** — it cannot be mirrored and is +not part of the runtime closure. The pass is followed by a self-containment +check that **fails the build if any link still escapes the bundle**, and by a +`dsh --version` smoke test. + +To build you must be able to create directory symlinks: **Windows — enable +Developer Mode or use an admin shell** (macOS/Linux need nothing). Without that +capability `build:harness` refuses, because the only fallback (dereferencing the +whole pnpm store) would balloon the ≈2 GB runtime to many times its size. A +locked-down machine can opt into that anyway with `--dereference-ok`. + +`pnpm desktop:verify` is the end-to-end check: it packs an unpacked app +(`electron-builder --dir`) and re-runs the same link check against the packaged +`resources/harness`, proving electron-builder's own `extraResources` copy kept +the relative links intact, then boots the web profile straight from the bundle. + ## Release process The app is **not code-signed**, so updates are a manual-download flow rather @@ -192,14 +232,18 @@ bump it too if you keep them in sync, but only the desktop one is user-visible. ### 2. Build and package ```sh -pnpm --filter @deepseek-ai/dsh-desktop run build:harness # assemble the self-contained runtime -pnpm desktop:pack # electron-builder: .dmg (mac) / .nsis .exe (win) +pnpm desktop:pack # build lib+web, assemble the self-contained harness, then electron-builder: .dmg (mac) / .nsis .exe (win) +pnpm desktop:verify # check the packaged harness is self-contained and boots ``` -Artifacts land in `apps/desktop/dist/`. Each target OS/arch needs its own -harness (`build/harness` bundles a platform Node + native addons), so regenerate -it on the target platform before packing that platform. Building the Windows -`.exe` on macOS needs wine (or build on a Windows host). +Artifacts land in `apps/desktop/dist/`. `desktop:pack` assembles `build/harness` +first (`build:harness`), so a stale or missing runtime cannot be shipped. Each +target OS/arch needs its own harness (`build/harness` bundles a platform Node + +native addons), so regenerate it on the target platform before packing that +platform. Building the Windows `.exe` on macOS needs wine (or build on a Windows +host). Run `desktop:verify` on each platform you package: it rebuilds an +unpacked app and proves the packaged `resources/harness` has no link escaping +the bundle and the web profile boots from it. ### 3. Generate the update manifest diff --git a/apps/desktop/package.json b/apps/desktop/package.json index 50f702f53a..a437983aee 100644 --- a/apps/desktop/package.json +++ b/apps/desktop/package.json @@ -16,6 +16,9 @@ "build": "tsc -b tsconfig.json && pnpm run build:preload", "build:preload": "esbuild src/preload.ts --bundle --platform=node --format=cjs --external:electron --outfile=lib/types/preload.cjs", "build:harness": "node scripts/build-harness.mjs", + "verify:harness": "node scripts/verify-harness.mjs", + "pack:dir": "electron-builder --dir", + "verify:packed": "node scripts/verify-harness.mjs --app dist", "generate-release-json": "node scripts/generate-release-json.mjs", "stage-release": "node scripts/stage-release.mjs", "typecheck": "tsc -b tsconfig.json", diff --git a/apps/desktop/scripts/build-harness.mjs b/apps/desktop/scripts/build-harness.mjs index 3bd4e34921..587dffa94f 100644 --- a/apps/desktop/scripts/build-harness.mjs +++ b/apps/desktop/scripts/build-harness.mjs @@ -10,13 +10,19 @@ * native, apps/cli, apps/web — plus the platform Node binary, into * `build/harness`, which electron-builder ships as `extraResources`. * - * Run from the repository root before `desktop:pack`. + * Run from the repository root (`pnpm --filter @deepseek-ai/dsh-desktop run + * build:harness`; the root `desktop:pack` script runs it automatically). The + * copy is followed by a relink pass (harness-relink.mjs) that rewrites every + * Windows junction / absolute link into a RELATIVE in-tree symlink, so the + * packaged harness no longer references the build machine's dev tree, and by a + * self-containment check plus a `dsh --version` smoke test. */ import { cpSync, chmodSync, existsSync, mkdirSync, rmSync, writeFileSync } from 'node:fs' import { dirname, join, resolve } from 'node:path' import { fileURLToPath } from 'node:url' import { execFileSync } from 'node:child_process' +import { assertSelfContained, probeSymlinkSupport, relinkHarness } from './harness-relink.mjs' const root = resolve(dirname(fileURLToPath(import.meta.url)), '../../..') const out = join(root, 'apps/desktop/build/harness') @@ -25,13 +31,41 @@ const out = join(root, 'apps/desktop/build/harness') const harnessDirs = ['node_modules', 'vendor', 'packages', 'native'] const extraDirs = [['apps/cli', 'apps/cli'], ['apps/web', 'apps/web']] +// A compact, portable harness depends on rewriting Windows junctions / absolute +// links into RELATIVE in-tree symlinks (see harness-relink.mjs). If this host +// cannot create directory symlinks, the only fallback is dereferencing the whole +// pnpm store — a multi-GB bloat — so refuse unless --dereference-ok opts in. +const dereferenceOk = process.argv.includes('--dereference-ok') +if (!probeSymlinkSupport() && !dereferenceOk) { + throw new Error( + 'cannot assemble a self-contained harness: this host cannot create directory symlinks.\n' + + 'Enable Windows Developer Mode or run as an administrator (macOS and Linux need no setup),\n' + + 'or pass --dereference-ok to accept a much larger dereferenced harness.', + ) +} + +// A missing critical dir silently shipped a broken (link-less) harness before. +// Refuse loudly instead: each one must exist, and lib/web must already be built. +for (const d of [...harnessDirs, ...extraDirs.map(([src]) => src)]) { + if (!existsSync(join(root, d))) { + throw new Error(`cannot build the harness: ${d}/ is missing — run \`pnpm install\` first`) + } +} +for (const [label, path] of [ + ['the dsh CLI (apps/cli/lib/bin.js)', join(root, 'apps/cli/lib/bin.js')], + ['the web frontend dist (apps/web/dist)', join(root, 'apps/web/dist')], +]) { + if (!existsSync(path)) { + throw new Error(`cannot build the harness: ${label} is missing — run \`pnpm desktop:build\` first`) + } +} + rmSync(out, { recursive: true, force: true }) mkdirSync(join(out, 'apps'), { recursive: true }) mkdirSync(join(out, 'bin'), { recursive: true }) for (const d of harnessDirs) { - const src = join(root, d) - if (existsSync(src)) cpSync(src, join(out, d), { recursive: true }) + cpSync(join(root, d), join(out, d), { recursive: true }) } for (const [src, dst] of extraDirs) { cpSync(join(root, src), join(out, dst), { recursive: true }) @@ -78,3 +112,32 @@ if (process.platform === 'win32') { } console.log(`assembled self-contained harness at ${out}`) + +// Normalize every link to a relative in-tree target, then prove the tree no +// longer references the build machine. This is the guarantee that the packaged +// app runs detached from the development environment. Re-run until a pass makes +// no further changes: rewriting one link can expose another (e.g. a package the +// copy dereferenced into a real dir whose internal links only become reachable +// on the next walk), so a single pass may not converge. +let report +for (let pass = 1; pass <= 5; pass += 1) { + report = relinkHarness(out, root) + console.log( + `relink pass ${pass}: ${report.normalized} rewritten, ${report.unchanged} in-tree, ` + + `${report.dereferenced} dereferenced, ${report.removed} dropped`, + ) + if (report.errors.length > 0) { + throw new Error(`harness relink failed:\n${report.errors.join('\n')}`) + } + if (report.normalized === 0 && report.removed === 0) break +} +assertSelfContained(out) +console.log('harness links: self-contained (no link escapes the bundle)') + +// Boot smoke: the bundled CLI must run straight from the bundle, proving the +// relinked node_modules resolves without the repository present. +const version = execFileSync(join(out, 'bin', nodeFile), [join(out, 'apps/cli/lib/bin.js'), '--version'], { + cwd: out, + encoding: 'utf8', +}) +console.log(`bundled dsh: ${version.trim()}`) diff --git a/apps/desktop/scripts/harness-relink.d.mts b/apps/desktop/scripts/harness-relink.d.mts new file mode 100644 index 0000000000..f476e01873 --- /dev/null +++ b/apps/desktop/scripts/harness-relink.d.mts @@ -0,0 +1,14 @@ +/** Type declarations for the pure relink/verify helpers in harness-relink.mjs. */ +export interface RelinkReport { + normalized: number + dereferenced: number + unchanged: number + removed: number + errors: string[] +} + +export function isWithin(root: string, p: string): boolean +export function relinkHarness(out: string, repoRoot: string): RelinkReport +export function findEscapingLinks(out: string): string[] +export function assertSelfContained(out: string): void +export function probeSymlinkSupport(): boolean diff --git a/apps/desktop/scripts/harness-relink.mjs b/apps/desktop/scripts/harness-relink.mjs new file mode 100644 index 0000000000..ba680c538f --- /dev/null +++ b/apps/desktop/scripts/harness-relink.mjs @@ -0,0 +1,209 @@ +#!/usr/bin/env node +/** + * Normalize every symlink/junction in a copied harness tree so the tree is + * self-contained: no link may resolve to a path outside the tree. + * + * A pnpm workspace is held together by links inside node_modules. On Windows + * those are often **junctions** (absolute-path reparse points) because real + * symlinks need Developer Mode or an admin shell; even plain symlinks can be + * absolute. `build-harness.mjs` copies the working tree with + * `cpSync(..., { recursive: true })`, which — with the default + * `dereference: false` — recreates every link verbatim, so the packaged harness + * keeps pointing at the BUILD machine's dev tree: it reads dev resources on the + * same machine and dangles on any other. This module rewrites each such link to + * a RELATIVE target inside the harness (mirroring the dev layout), and copies + * the real content of any genuinely external target in so nothing escapes. + * + * Pure Node, no Electron and no workspace TS program: imported by + * `scripts/build-harness.mjs` / `scripts/verify-harness.mjs` and unit-tested + * from `tests/harness-relink.spec.ts`. + * @module dsh-desktop/harness-relink + */ + +import { + cpSync, existsSync, lstatSync, mkdirSync, mkdtempSync, readdirSync, readlinkSync, rmSync, symlinkSync, +} from 'node:fs' +import { tmpdir } from 'node:os' +import { dirname, isAbsolute, join, relative, resolve, sep } from 'node:path' + +/** True when `p` resolves inside `root` (or is `root` itself). */ +export function isWithin(root, p) { + const rel = relative(resolve(root), resolve(p)) + return rel === '' || (rel !== '..' && !rel.startsWith(`..${sep}`) && !isAbsolute(rel)) +} + +/** The link's own kind so Windows creates the right reparse point ('dir'/'file'). */ +function linkKind(target) { + try { + return lstatSync(target).isDirectory() ? 'dir' : 'file' + } catch { + return 'dir' + } +} + +/** Remove a link and copy the target's real content into its place. */ +function dereferenceInto(link, target) { + rmSync(link, { force: true }) + cpSync(target, link, { recursive: true, dereference: true }) +} + +/** Rewrite `link` as a relative symlink to `mirrored`; dereference on failure. */ +function relinkTo(link, mirrored, report) { + const rel = relative(dirname(link), mirrored) + try { + rmSync(link, { force: true }) + symlinkSync(rel, link, linkKind(mirrored)) + report.normalized += 1 + } catch (error) { + // Symlink creation failed (e.g. no Developer Mode on Windows): a compact + // portable link is impossible, so copy the target content in instead. + try { + dereferenceInto(link, mirrored) + report.dereferenced += 1 + } catch (copyError) { + report.errors.push(`${link} -> ${mirrored} (${String(copyError)})`) + } + } +} + +/** + * Walk `out` and normalize every symlink/junction to a relative in-tree link. + * Idempotent: a link already pointing inside the tree is left alone. + * @param out - the harness root. + * @param repoRoot - the repository root whose layout `out` mirrors. + * @returns per-kind counts plus any unrecoverable errors. + */ +export function relinkHarness(out, repoRoot) { + const report = { normalized: 0, dereferenced: 0, unchanged: 0, removed: 0, errors: [] } + forEachLink(out, (link) => { + const target = readlinkSync(link) + const abs = resolve(dirname(link), target) + if (isWithin(out, abs)) { + if (!existsSync(abs)) { + // A link inside the tree whose target is absent points at a workspace + // package the harness does not ship (python/sdk-runtime, examples, …). + // It cannot resolve and is not part of the runtime: drop it. + rmSync(link, { force: true }) + report.removed += 1 + } else if (relative(dirname(link), abs) === target) { + // Already inside the tree: make sure the link is a relative in-tree path. + report.unchanged += 1 + } else { + relinkTo(link, abs, report) + } + } else if (isWithin(repoRoot, abs)) { + // Points at the dev tree: mirror the target into the harness and relink. + const mirrored = join(out, relative(repoRoot, abs)) + if (existsSync(mirrored)) { + relinkTo(link, mirrored, report) + } else if (isWithin(abs, out)) { + // The target contains the harness itself (e.g. the desktop shell + // package, which the harness deliberately does not ship). Copying it in + // would recurse into the bundle and it is not part of the runtime + // closure, so the link cannot be made self-contained — drop it. + rmSync(link, { force: true }) + report.removed += 1 + } else if (existsSync(abs)) { + try { + dereferenceInto(link, abs) + report.dereferenced += 1 + } catch (error) { + report.errors.push(`${link} -> ${abs} (${String(error)})`) + } + } else { + report.errors.push(`${link} -> ${abs} (dangling)`) + } + } else if (existsSync(abs)) { + // Genuinely external target (e.g. a global store): copy content in. + try { + dereferenceInto(link, abs) + report.dereferenced += 1 + } catch (error) { + report.errors.push(`${link} -> ${abs} (${String(error)})`) + } + } else { + report.errors.push(`${link} -> ${abs} (dangling)`) + } + }) + return report +} + +/** + * Every symlink/junction whose target escapes `out` or no longer exists. A + * non-empty result means the harness is NOT detached from its build machine. + */ +export function findEscapingLinks(out) { + const escaping = [] + forEachLink(out, (link) => { + const abs = resolve(dirname(link), readlinkSync(link)) + if (!isWithin(out, abs) || !existsSync(abs)) escaping.push(link) + }) + return escaping +} + +/** Throw unless the tree is self-contained (no escaping or dangling links). */ +export function assertSelfContained(out) { + const escaping = findEscapingLinks(out) + if (escaping.length === 0) return + const listed = escaping.slice(0, 20).join('\n ') + throw new Error( + `harness is NOT self-contained: ${escaping.length} link(s) resolve outside it or are dangling:\n ${listed}`, + ) +} + +/** + * Whether the current host can create (relative) directory symlinks. Windows + * without Developer Mode (and without an admin shell) cannot; without this + * capability the packaged harness could not be relinked compactly. + */ +export function probeSymlinkSupport() { + const base = mkdtempSync(join(tmpdir(), 'dsh-symlink-probe-')) + const target = join(base, 'target') + const link = join(base, 'link') + try { + // The target must exist: Windows refuses a directory link to a missing + // target, and on POSIX existsSync() would report a dangling link as absent. + mkdirSync(target, { recursive: true }) + symlinkSync(target, link, 'dir') + return existsSync(link) + } catch { + return false + } finally { + rmSync(base, { recursive: true, force: true }) + } +} + +/** + * Visit every symlink/junction under `root` without following into any link. + * Junctions are reported as symlinks by `lstat` on Windows; because the walk + * never recurses through a link, it cannot loop or escape the tree. + */ +function forEachLink(root, fn) { + const stack = [resolve(root)] + const seen = new Set() + while (stack.length > 0) { + const dir = stack.pop() + if (dir === undefined || seen.has(dir)) continue + seen.add(dir) + let entries + try { + entries = readdirSync(dir, { withFileTypes: true }) + } catch { + continue + } + for (const entry of entries) { + const path = join(dir, entry.name) + let stat + try { + stat = lstatSync(path) + } catch { + continue + } + if (stat.isSymbolicLink()) { + fn(path) + } else if (stat.isDirectory()) { + stack.push(path) + } + } + } +} diff --git a/apps/desktop/scripts/verify-harness.mjs b/apps/desktop/scripts/verify-harness.mjs new file mode 100644 index 0000000000..06e7198fd6 --- /dev/null +++ b/apps/desktop/scripts/verify-harness.mjs @@ -0,0 +1,189 @@ +#!/usr/bin/env node +/** + * Verify a harness tree is self-contained and actually boots from the bundle. + * + * Default: check the assembled `build/harness`. With `--app `, locate the + * `resources/harness` inside a packed/unpacked app instead — this additionally + * proves electron-builder's `extraResources` copy preserved the relative + * in-tree links. `--boot` also spawns the web profile straight from the bundle + * (temp DSH_HOME, telemetry off) and waits for its readiness line, proving the + * relinked node_modules resolves without the repository present. + * + * Usage: + * node scripts/verify-harness.mjs # build/harness + * node scripts/verify-harness.mjs --boot # + web boot smoke + * node scripts/verify-harness.mjs --app dist # a packed app's harness + * node scripts/verify-harness.mjs --app dist --boot + */ + +import { spawn, spawnSync } from 'node:child_process' +import { existsSync, mkdtempSync, readdirSync, rmSync } from 'node:fs' +import { tmpdir } from 'node:os' +import { basename, dirname, isAbsolute, join, resolve } from 'node:path' +import { fileURLToPath } from 'node:url' +import { assertSelfContained } from './harness-relink.mjs' + +const root = resolve(dirname(fileURLToPath(import.meta.url)), '../../..') +const desktopDir = join(root, 'apps/desktop') +const defaultHarness = join(desktopDir, 'build/harness') + +const args = process.argv.slice(2) +const boot = args.includes('--boot') +const appIndex = args.indexOf('--app') +const appValue = args.find((a) => a.startsWith('--app='))?.slice('--app='.length) ?? (appIndex !== -1 ? args[appIndex + 1] : undefined) +const appRoot = appValue === undefined + ? undefined + : isAbsolute(appValue) + ? resolve(appValue) + : resolve(desktopDir, appValue) + +const timeoutMs = 60_000 +const READY_RE = /http:\/\/127\.0\.0\.1:(\d+)/ + +function fail(message) { + console.error(`verify-harness: ${message}`) + process.exit(1) +} + +/** Find every `resources/harness` under an unpacked electron-builder app. */ +function findHarnessUnder(appDir) { + const found = [] + // electron-builder unpacked layout: + // win: /win-unpacked/resources/harness + // mac: /mac(-arch)/.app/Contents/Resources/harness + const visit = (dir, depth) => { + if (depth > 5) return + let entries + try { + entries = readdirSync(dir, { withFileTypes: true }) + } catch { + return + } + for (const entry of entries) { + const p = join(dir, entry.name) + if (basename(p) === 'harness' && basename(dirname(p)) === 'resources') found.push(p) + if (entry.isDirectory()) visit(p, depth + 1) + } + } + visit(appDir, 0) + return found +} + +function nodeBinary(harness) { + return join(harness, 'bin', process.platform === 'win32' ? 'node.exe' : 'node') +} + +function cliEntry(harness) { + return join(harness, 'apps/cli/lib/bin.js') +} + +/** Run the bundled CLI's `--version` straight from the bundle. */ +function checkVersion(harness) { + const result = spawnSync(nodeBinary(harness), [cliEntry(harness), '--version'], { + cwd: harness, + encoding: 'utf8', + timeout: timeoutMs, + }) + if (result.error !== undefined || result.status !== 0) { + fail(`the bundled CLI failed to run: ${result.error?.message ?? result.stderr?.trim() ?? `exit ${String(result.status)}`}`) + } + console.log(` dsh --version: ${result.stdout?.trim()}`) +} + +/** + * Spawn the web profile from the bundle and resolve once it prints its + * readiness URL. Hermetic: a temp DSH_HOME and telemetry off, so the check + * neither touches the real profile nor the repository. + */ +function bootWeb(harness) { + const home = mkdtempSync(join(tmpdir(), 'dsh-verify-home-')) + const child = spawn( + nodeBinary(harness), + [cliEntry(harness), '--profile', 'web', '--host', '127.0.0.1', '--port', '0'], + { + cwd: harness, + stdio: ['ignore', 'pipe', 'inherit'], + env: { ...process.env, DSH_HOME: home, DSH_TELEMETRY_DISABLED: '1', DSH_ALLOW_PLUGIN_INSTALL: '1' }, + }, + ) + return new Promise((resolvePromise, reject) => { + let settled = false + let buffered = '' + const cleanup = () => { + try { + rmSync(home, { recursive: true, force: true }) + } catch { + // Best effort; a locked file on Windows is not fatal for a verify tool. + } + } + const timer = setTimeout(() => { + if (settled) return + settled = true + child.kill() + cleanup() + reject(new Error(`web boot timed out after ${timeoutMs}ms`)) + }, timeoutMs) + child.stdout?.setEncoding('utf8') + child.stdout?.on('data', (chunk) => { + if (settled) return + buffered += chunk + if (READY_RE.test(buffered)) { + settled = true + clearTimeout(timer) + child.kill() + cleanup() + resolvePromise() + } + }) + child.on('error', (error) => { + if (settled) return + settled = true + clearTimeout(timer) + cleanup() + reject(error) + }) + child.on('exit', (code) => { + if (settled) return + settled = true + clearTimeout(timer) + cleanup() + reject(new Error(`harness exited before serving (code ${String(code)})`)) + }) + }) +} + +async function main() { + const harness = (() => { + if (appRoot !== undefined) { + const found = findHarnessUnder(appRoot) + if (found.length === 0) fail(`no resources/harness found under ${appRoot}`) + if (found.length > 1) { + console.warn(`verify-harness: ${found.length} harness dirs found; checking the first:\n ${found.join('\n ')}`) + } + return found[0] + } + if (!existsSync(defaultHarness)) fail(`build/harness not found at ${defaultHarness} — run build:harness first`) + return defaultHarness + })() + + console.log(`verify-harness: ${harness}`) + if (!existsSync(nodeBinary(harness))) fail(`bundled node not found at ${nodeBinary(harness)}`) + try { + assertSelfContained(harness) + } catch (error) { + fail(error instanceof Error ? error.message : String(error)) + } + console.log(' links: self-contained (no link escapes the bundle)') + checkVersion(harness) + if (boot) { + try { + await bootWeb(harness) + } catch (error) { + fail(error instanceof Error ? error.message : String(error)) + } + console.log(' web boot: ok') + } + console.log('verify-harness: ok') +} + +main().catch((error) => fail(error instanceof Error ? error.message : String(error))) diff --git a/apps/desktop/tests/harness-relink.spec.ts b/apps/desktop/tests/harness-relink.spec.ts new file mode 100644 index 0000000000..77cc8e3583 --- /dev/null +++ b/apps/desktop/tests/harness-relink.spec.ts @@ -0,0 +1,165 @@ +import { + existsSync, lstatSync, mkdirSync, mkdtempSync, readFileSync, readlinkSync, rmSync, symlinkSync, writeFileSync, +} from 'node:fs' +import { tmpdir } from 'node:os' +import { dirname, join, resolve } from 'node:path' +import { afterEach, describe, expect, it } from 'vitest' +import { + assertSelfContained, findEscapingLinks, isWithin, probeSymlinkSupport, relinkHarness, +} from '../scripts/harness-relink.mjs' + +const dirs: string[] = [] + +afterEach(() => { + for (const dir of dirs.splice(0)) rmSync(dir, { recursive: true, force: true }) +}) + +/** + * A repo whose layout mirrors the real one: `out` is the harness copy of the + * dev tree at `/apps/desktop/build/harness`, and the repo has a package + * the harness mirrors. + */ +function makeTree() { + const root = mkdtempSync(join(tmpdir(), 'dsh-relink-')) + dirs.push(root) + const repo = join(root, 'repo') + const out = join(repo, 'apps/desktop/build/harness') + mkdirSync(join(repo, 'packages/core/dsh-core'), { recursive: true }) + writeFileSync(join(repo, 'packages/core/dsh-core/index.js'), 'export const x = 1\n') + mkdirSync(join(out, 'packages/core/dsh-core'), { recursive: true }) + writeFileSync(join(out, 'packages/core/dsh-core/index.js'), 'export const x = 1\n') + mkdirSync(join(out, 'node_modules/@deepseek-ai'), { recursive: true }) + return { root, repo, out } +} + +/** The dir-symlink type to use when pre-creating a fixture link on this host. */ +const dirLinkType = () => (process.platform === 'win32' ? 'junction' : 'dir') + +/** Whether tests that create symlinks can run on this host. */ +const canSymlink = probeSymlinkSupport() + +describe('isWithin', () => { + it('accepts a path inside the root and rejects an escape', () => { + expect(isWithin('C:\\a\\b', 'C:\\a\\b\\c')).toBe(true) + expect(isWithin('C:\\a\\b', 'C:\\a\\other')).toBe(false) + expect(isWithin('C:\\a\\b', 'C:\\a\\b')).toBe(true) + }) + + it('rejects an absolute path on another drive', () => { + expect(isWithin('C:\\a\\b', 'D:\\x')).toBe(false) + }) +}) + +describe.skipIf(!canSymlink)('relinkHarness', () => { + it('leaves a relative in-tree link alone', () => { + const { out } = makeTree() + const link = join(out, 'node_modules/@deepseek-ai/dsh-core') + symlinkSync('../../packages/core/dsh-core', link, 'dir') + const report = relinkHarness(out, dirname(out)) + expect(report.unchanged).toBeGreaterThanOrEqual(1) + expect(report.errors).toEqual([]) + expect(findEscapingLinks(out)).toEqual([]) + }) + + it('rewrites an absolute link to the dev tree into a relative in-tree link', () => { + const { repo, out } = makeTree() + const link = join(out, 'node_modules/@deepseek-ai/dsh-core') + const devTarget = join(repo, 'packages/core/dsh-core') + symlinkSync(devTarget, link, dirLinkType()) + const report = relinkHarness(out, repo) + expect(report.normalized).toBeGreaterThanOrEqual(1) + expect(report.errors).toEqual([]) + expect(lstatSync(link).isSymbolicLink()).toBe(true) + const raw = readlinkSync(link) + expect(resolve(dirname(link), raw)).toBe(resolve(join(out, 'packages/core/dsh-core'))) + expect(findEscapingLinks(out)).toEqual([]) + }) + + it('mirrors a node_modules/.pnpm store link into the harness', () => { + const { repo, out } = makeTree() + // The store entry the workspace link would point at, in both trees. + for (const base of [repo, out]) { + mkdirSync(join(base, 'node_modules/.pnpm/b@1/node_modules/b'), { recursive: true }) + writeFileSync(join(base, 'node_modules/.pnpm/b@1/node_modules/b/index.js'), 'export const b = 2\n') + } + const link = join(out, 'node_modules/@deepseek-ai/a') + symlinkSync(join(repo, 'node_modules/.pnpm/b@1/node_modules/b'), link, dirLinkType()) + const report = relinkHarness(out, repo) + expect(report.errors).toEqual([]) + expect(resolve(dirname(link), readlinkSync(link))) + .toBe(resolve(join(out, 'node_modules/.pnpm/b@1/node_modules/b'))) + expect(findEscapingLinks(out)).toEqual([]) + }) + + it('dereferences a genuinely external target into real content', () => { + const { root, out } = makeTree() + const external = join(root, 'external') + mkdirSync(external, { recursive: true }) + writeFileSync(join(external, 'index.js'), 'export const ext = 3\n') + const link = join(out, 'node_modules/external') + symlinkSync(external, link, dirLinkType()) + const report = relinkHarness(out, dirname(out)) + expect(report.dereferenced).toBeGreaterThanOrEqual(1) + expect(report.errors).toEqual([]) + expect(lstatSync(link).isSymbolicLink()).toBe(false) + expect(readFileSync(join(link, 'index.js'), 'utf8')).toContain('ext') + expect(findEscapingLinks(out)).toEqual([]) + }) + + it('drops a link whose target contains the harness (e.g. the desktop shell)', () => { + const { repo, out } = makeTree() + // `repo/apps/desktop` contains `out` (the harness): it is a workspace + // package the harness deliberately does not ship, so the link cannot be + // made self-contained and must be dropped, not copied. + const link = join(out, 'node_modules/desktop-shell') + symlinkSync(join(repo, 'apps/desktop'), link, dirLinkType()) + const report = relinkHarness(out, repo) + expect(report.removed).toBeGreaterThanOrEqual(1) + expect(report.errors).toEqual([]) + expect(existsSync(link)).toBe(false) + expect(findEscapingLinks(out)).toEqual([]) + }) + + it('drops a link that points at a missing in-tree target (unshipped package)', () => { + const { out } = makeTree() + // A link to `out/python/sdk-runtime` — inside the tree lexically but absent + // (python is not part of the harness): it cannot resolve, so it is dropped. + const link = join(out, 'node_modules/dsh-sdk-runtime') + symlinkSync('../python/sdk-runtime', link, dirLinkType()) + const report = relinkHarness(out, dirname(out)) + expect(report.removed).toBeGreaterThanOrEqual(1) + expect(report.errors).toEqual([]) + expect(existsSync(link)).toBe(false) + expect(findEscapingLinks(out)).toEqual([]) + }) + + it('reports a dangling link as an error and leaves it detectable', () => { + const { root, out } = makeTree() + const link = join(out, 'node_modules/ghost') + symlinkSync(join(root, 'does-not-exist'), link, dirLinkType()) + const report = relinkHarness(out, dirname(out)) + expect(report.errors.some(e => e.includes('dangling'))).toBe(true) + expect(findEscapingLinks(out)).toContain(link) + }) +}) + +describe.skipIf(!canSymlink)('findEscapingLinks / assertSelfContained', () => { + it('flags an escaping link and assertSelfContained throws', () => { + const { repo, out } = makeTree() + const link = join(out, 'node_modules/outside') + symlinkSync(join(repo, 'packages/core/dsh-core'), link, 'dir') + expect(findEscapingLinks(out)).toContain(link) + expect(() => assertSelfContained(out)).toThrow(/self-contained/) + }) + + it('assertSelfContained passes on a clean tree', () => { + const { out } = makeTree() + expect(() => assertSelfContained(out)).not.toThrow() + }) +}) + +describe('probeSymlinkSupport', () => { + it('reports a boolean without throwing', () => { + expect(typeof probeSymlinkSupport()).toBe('boolean') + }) +}) diff --git a/package.json b/package.json index 5d50121fbf..751b674f2c 100644 --- a/package.json +++ b/package.json @@ -142,7 +142,8 @@ "desktop:build": "pnpm run build:lib:host && pnpm --filter @deepseek-ai/dsh-web-frontend run build && pnpm --filter @deepseek-ai/dsh-desktop run build", "desktop:dev": "pnpm run desktop:build && pnpm --filter @deepseek-ai/dsh-desktop exec electron .", "desktop:dev:fast": "pnpm --filter @deepseek-ai/dsh-web-frontend run build && pnpm --filter @deepseek-ai/dsh-desktop run build && pnpm --filter @deepseek-ai/dsh-desktop exec electron .", - "desktop:pack": "pnpm run desktop:build && pnpm --filter @deepseek-ai/dsh-desktop run pack", + "desktop:pack": "pnpm run desktop:build && pnpm --filter @deepseek-ai/dsh-desktop run build:harness && pnpm --filter @deepseek-ai/dsh-desktop run pack", + "desktop:verify": "pnpm --filter @deepseek-ai/dsh-desktop run pack:dir && pnpm --filter @deepseek-ai/dsh-desktop run verify:packed", "postinstall": "node scripts/install-lefthook.mjs" }, "devDependencies": {