fix: address ds-review-bot v6/v7 findings on the image-input assembly
- resolveLlmRoute: reuse the yml pi-ai row for providers it already routes (DUPLICATE_ADAPTER boot failure) and detect an unset model by origin, not by comparison against one deployment default; covered by a new spec. - LlmService.resolveModelInfoFor preserves (and validates) modality metadata, arming the host image preflight for exact-route resolution. - session.selectModel refuses a text-only target once the session log carries an image on any replayed route; an accepted switch would strand every later turn with no in-product recovery. - The composer no longer gates image intake on the handshake activeModel snapshot (wrong authority for a per-session decision); the host preflight plus the error strip own capability, deployment limits stay client-side. - InputHub shell teardown releases the scope's draft images (File objects and object URLs leaked for the page lifetime). - session.prompt image parts carry optional alt into the durable block; ImageBlock documents assistant-side rendering as forward compatibility. - Assembled built-client lane apps/web/tests/image-display.snapshot.ts pins the history galleries over the authorized attachment route, the lightbox, and the composer paste rail; the attachment rail is an accessible group. - Docs: validateImage on the seam page, fixture byte metadata matches its PNG, and the Agent Note claims now match the shipped coverage.
This commit is contained in:
creatixchu
+2
-2
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-22-web-multimodal-image-input-and-durable-attachments.md
|
||||
2026-07-22-web-multimodal-image-input-and-durable-attachments.md: 746ae8a5ffb111866c99c6aad5cb0906f252481e
|
||||
2026-07-22-web-multimodal-image-input-and-durable-attachments.zh.md: 5061e8e29a14790d22456a44e44fa162d244f116
|
||||
2026-07-22-web-multimodal-image-input-and-durable-attachments.md: 2b676c1965dbef0bdc24ba6d5b4af5bf66840780
|
||||
2026-07-22-web-multimodal-image-input-and-durable-attachments.zh.md: f13648051bcf4c81c8b032897645c9b6aeb0d32a
|
||||
|
||||
+3
-3
@@ -120,7 +120,7 @@ Base64 crosses JSON-RPC once and is discarded after persistence. The host valida
|
||||
|
||||
Model catalog entries gain optional merge-extensible input and output modality declarations. A missing declaration means unknown; a present list without `image` is an explicit negative capability.
|
||||
|
||||
The host is the authoritative preflight boundary. It resolves the session's latest routed provider/model, falling back through agent options to host defaults; if that model explicitly excludes image input, it rejects the prompt before writing any attachment or event, and the client restores the draft. Unknown capability proceeds to the adapter guard so uncatalogued model identifiers remain usable. `host.describe` projects the active model and image limits into `SessionsService`; the resident `InputBar` uses them before allocating object URLs or base64 for fast feedback. Decoded-pixel validation and the session's actual route remain authoritative on the host.
|
||||
The host is the authoritative preflight boundary. It resolves the session's latest routed provider/model, falling back through agent options to host defaults; if that model explicitly excludes image input, it rejects the prompt before writing any attachment or event, and the client restores the draft. Unknown capability proceeds to the adapter guard so uncatalogued model identifiers remain usable. `host.describe` projects the host-default active model and the image limits into `SessionsService`; the composer applies only the deployment limits before allocating object URLs or base64. Model capability is deliberately not gated client-side: the handshake snapshot cannot represent a session's current target after `session.selectModel`, so the host preflight is the sole capability authority and its rejection renders through the composer error strip. Decoded-pixel validation and the session's actual route remain authoritative on the host.
|
||||
|
||||
The Pi-AI adapter is the first visual-input route: it resolves `ctx.attachments` at request time, then resolves each durable reference and emits native image content only for models that declare image input. The shipped Web assembly reaches it through `dsh web --provider <name> --model <id>`, which mounts that pi-ai catalog route with the provider's ambient credentials; the DeepSeek-only default remains text-only. Request-time service resolution keeps Cordis load order from freezing optional attachment availability. The hand-written DeepSeek adapter throws typed `UNSUPPORTED_CONTENT` for an image anywhere in the request, including nested tool results. No adapter may flatten or skip an image.
|
||||
|
||||
@@ -194,8 +194,8 @@ UI state can be stale and does not protect direct SDK, ACP, replay, or uncatalog
|
||||
## Testing
|
||||
|
||||
- Storage tests cover content-addressed deduplication, private permissions, admission failures, corruption/missing-object failures, and reading history after deployment limits are lowered.
|
||||
- Host and protocol tests cover persist-before-event ordering, absence of base64 in logs, session-scoped authorization, capability rejection, upload limits, and bounded HTTP request bodies.
|
||||
- Client unit and assembled Chromium tests cover paste and drop, mixed clipboard text, image-only send, draft restoration, historical user and assistant images, original preview, ordering, and draft/session/application object-URL cleanup.
|
||||
- Host and protocol tests cover persist-before-event ordering, absence of base64 in logs, session-scoped authorization, capability rejection, upload limits, bounded HTTP request bodies, and refusal of a text-only `session.selectModel` once the session log carries an image (an accepted switch would strand every later turn).
|
||||
- Client unit tests cover paste and drop, mixed clipboard text, image-only send, draft restoration, ordering, and draft/session-scope/application object-URL cleanup; the keyless assembled built-client lane (`apps/web/tests/image-display.snapshot.ts`, `DSH_EXAMPLE_MODE=lib pnpm run test:snapshot`) covers the historical user and assistant galleries over the authorized attachment route, the original-size lightbox, and the composer paste rail.
|
||||
- Adapter and compaction tests cover native Pi-AI image conversion, late attachment-service composition, text-only rejection, nested tool-result images, preserved summary input, and explicit image-output rejection.
|
||||
- A credentialed real-API test sends a PNG through the Anthropic `claude-opus-4-8` route and requires the model to identify its QR code.
|
||||
- The current production adapter set declares text-only output; output-provider certification remains outside version one.
|
||||
|
||||
+3
-3
@@ -120,7 +120,7 @@ Base64 只跨越一次 JSON-RPC，并在持久化后丢弃。宿主会校验规
|
||||
|
||||
模型目录项增加可选且可合并扩展的输入与输出模态声明。缺少声明表示未知；声明存在但不含 `image`，则明确表示不支持图片。
|
||||
|
||||
宿主是权威的前置检查边界。它会解析会话最新路由到的提供方和模型，并在缺失时依次回退到 agent 选项和宿主默认值；如果该模型明确排除图片输入，宿主会在写入任何附件或事件前拒绝提示词，客户端则恢复草稿。能力未知时继续进入适配器强制检查，使未收录的模型标识符仍然可用。`host.describe` 把当前模型和图片限制投影到 `SessionsService`；常驻 `InputBar` 会在分配对象 URL 或 base64 前使用这些信息提供快速反馈。解码像素校验与会话的实际路由仍由宿主作出权威判定。
|
||||
宿主是权威的前置检查边界。它会解析会话最新路由到的提供方和模型，并在缺失时依次回退到 agent 选项和宿主默认值；如果该模型明确排除图片输入，宿主会在写入任何附件或事件前拒绝提示词，客户端则恢复草稿。能力未知时继续进入适配器强制检查，使未收录的模型标识符仍然可用。`host.describe` 把宿主默认的当前模型和图片限制投影到 `SessionsService`；composer 在分配对象 URL 或 base64 前只应用部署级限制。模型能力刻意不在客户端把关：握手快照无法表达 `session.selectModel` 之后会话的当前目标，因此宿主前置检查是唯一的能力权威，其拒绝通过 composer 错误条呈现。解码像素校验与会话的实际路由仍由宿主作出权威判定。
|
||||
|
||||
Pi-AI 适配器是首条视觉输入路径：它在请求时解析 `ctx.attachments`，再解析每个持久引用，并且仅为声明支持图片输入的模型生成提供方原生图片内容。交付的 Web 组装通过 `dsh web --provider <name> --model <id>` 到达这条路径——该命令用提供方的环境凭据挂载对应的 pi-ai 目录路由；仅含 DeepSeek 的默认组装仍是纯文本。在请求时解析服务，可避免 Cordis 加载顺序将可选附件服务的可用性固化。手写 DeepSeek 适配器遇到请求中任何位置的图片时都会抛出类型化的 `UNSUPPORTED_CONTENT` 错误，包括嵌套工具结果中的图片。任何适配器都不得将图片展平或跳过。
|
||||
|
||||
@@ -194,8 +194,8 @@ UI 状态可能陈旧，也无法保护直接 SDK、ACP、回放或未收录模
|
||||
## 测试
|
||||
|
||||
- 存储测试覆盖内容寻址去重、私有权限、准入失败、对象损坏或缺失时的失败，以及收紧部署限制后读取历史数据。
|
||||
- 宿主与协议测试覆盖先持久化再追加事件的顺序、日志中不含 base64、会话作用域授权、能力拒绝、上传限制，以及大小受限的 HTTP 请求体。
|
||||
- 客户端单元测试和组装后的 Chromium 测试覆盖粘贴与拖放、混合剪贴板文本、仅图片发送、草稿恢复、历史用户与助手图片、原图预览、顺序，以及草稿、会话和应用层级的对象 URL 清理。
|
||||
- 宿主与协议测试覆盖先持久化再追加事件的顺序、日志中不含 base64、会话作用域授权、能力拒绝、上传限制、大小受限的 HTTP 请求体，以及在会话日志已含图片时拒绝切换到纯文本模型的 `session.selectModel`（接受该切换会让此后每一轮都失败）。
|
||||
- 客户端单元测试覆盖粘贴与拖放、混合剪贴板文本、仅图片发送、草稿恢复、顺序，以及草稿、会话作用域和应用层级的对象 URL 清理；keyless 的组装后构建产物通道（`apps/web/tests/image-display.snapshot.ts`，`DSH_EXAMPLE_MODE=lib pnpm run test:snapshot`）覆盖经授权附件路由渲染的历史用户与助手图片画廊、原图 lightbox，以及 composer 粘贴缩略图条。
|
||||
- 适配器与压缩测试覆盖 Pi-AI 原生图片转换、后置附件服务组合、仅文本拒绝、嵌套工具结果中的图片、保留摘要输入，以及明确拒绝图片输出。
|
||||
- 需要凭据的实际 API 测试会通过 Anthropic `claude-opus-4-8` 路径发送一张 PNG，并要求模型识别其中的二维码。
|
||||
- 当前生产适配器集合声明仅支持文本输出；输出提供方认证不在第一版范围内。
|
||||
|
||||
Reference in New Issue
Block a user