Compare commits
12 Commits
1e63da95ec
..
main
| Author | SHA1 | Date | |
|---|---|---|---|
 Pine
|
4e73a6bba8 | ||
|
Pine
|
9f2b90aeca | ||
|
PineWin
|
c43dc7f888 | ||
|
Pine
|
3688d5d0b1 | ||
|
Pine
|
e72442dae0 | ||
|
PineWin
|
93b27cf227 | ||
|
Pine
|
fd6493f582 | ||
|
PineWin
|
c9460c8d6e | ||
|
Pine
|
a5abf7bae4 | ||
|
Pine
|
e36840d12b | ||
|
Pine
|
91e20c3a81 | ||
|
Pine
|
5ed396aea4 |
@@ -1,6 +0,0 @@
|
||||
# 前端构建配置（已 gitignore，勿提交）
|
||||
# 前端 MQTT 账号：dpm / 123456
|
||||
VITE_API_BASE=http://192.168.1.9:10085
|
||||
VITE_MQTT_URL=ws://192.168.1.3:8083/mqtt
|
||||
VITE_MQTT_USERNAME=dpm
|
||||
VITE_MQTT_PASSWORD=123456
|
||||
@@ -13,6 +13,15 @@ dist-ssr
|
||||
|
||||
# 后端密钥 / 前端本地配置（含阿里云密钥、MQTT 凭据，不入库）
|
||||
backend/.env
|
||||
.env.local
|
||||
|
||||
# Windows 安装包产物（可随时由 source + tauri build 重新生成，不入库）
|
||||
*.exe
|
||||
|
||||
# 误放入的 @appica 前端组件文档（非项目源码）
|
||||
backend/installation.md
|
||||
backend/autocomplete.md
|
||||
backend/icons.md
|
||||
|
||||
# 后端运行时数据（媒体上传、持久化配置）
|
||||
backend/media/
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
# 工作规则（永远遵守）
|
||||
|
||||
> 📍 工作区位置：`code/dpm/`（园区大屏系统，独立仓库）。整体目录规范见根 `CLAUDE.md`；进程管理铁律适用于本仓库全部前后端服务。
|
||||
|
||||
## 进程管理铁律
|
||||
- **绝对禁止 AI 启动、重启、停止任何前后端服务进程**（后端 10085 / s2s 8765 / vite 1420 / Tauri 等）。
|
||||
- 需要启动或重启时，**必须先通知用户**，由用户亲自操作。
|
||||
|
||||
@@ -1,199 +1,93 @@
|
||||
# 大屏媒体轮播系统 (DPM)
|
||||
|
||||
**Digital Playback Media** — 基于 Tauri + React 的全屏媒体轮播控制系统，专为昆明市大学生创业园展播大屏设计。
|
||||
> 📍 工作区位置：`code/dpm/`（独立仓库）。系统/运维文档见 `design/dpm/`；权威后端说明见 `backend/README.md`；整体目录规范见根 `CLAUDE.md`。
|
||||
|
||||
## 概述
|
||||
**Digital Playback Media** — 园区大屏展播控制系统，服务昆明市大学生创业园：数据大屏、数字孪生 3D、AI 助手、语音助手、媒体轮播、企业展示墙，浏览器后台可远程控制。
|
||||
|
||||
DPM 是一套完整的数字标牌（Digital Signage）解决方案，包含两大界面：
|
||||
## 架构总览
|
||||
|
||||
- **大屏展示页** (`/screen`) — 全屏自动轮播，支持视频与图片，具备沉浸式空间视觉背景
|
||||
- **管理后台** (`/admin`) — 登录保护的管理面板，用于媒体上传、播放列表编排、播放控制与系统设置
|
||||
```
|
||||
┌──────────────┐ REST（全部数据/媒体/设置） ┌────────────────┐
|
||||
│ Tauri 大屏 │ ◄────────────────────────── │ FastAPI :10085 │
|
||||
│ (React 前端) │ └───────┬────────┘
|
||||
│ │ MQTT 订阅 opc/display/command │ MQTT 发布
|
||||
│ 账号: dpm │ ◄────────────────────────────────────│ 账号: dpmserver
|
||||
└──────┬───────┘ MQTT Broker (EMQX) │
|
||||
│ 192.168.1.3:1883 │
|
||||
└───────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
系统通过 **SSE（Server-Sent Events）** 实现管理端与展示端的实时状态同步，支持局域网内远程管理。
|
||||
- **数据 / 媒体 / 设置** → REST API（FastAPI，JSON 文件持久化）
|
||||
- **页面与媒体控制** → MQTT（EMQX broker），按 `clientId` 定向下发；SSE `/api/events` 作回退
|
||||
- **桌面壳** → Tauri 2（Rust），仅负责窗口生命周期与 3 个命令（读运行配置、双屏、安装更新）
|
||||
- **管理后台** → FastAPI + Jinja 渲染的 `/admin` 页面（非 React 路由）
|
||||
|
||||
## 技术栈
|
||||
> **权威说明以 `backend/README.md` 为准**，含全部接口、MQTT 主题、环境变量与 Docker 部署。
|
||||
|
||||
| 层级 | 技术 |
|
||||
## 目录
|
||||
|
||||
| 路径 | 说明 |
|
||||
|------|------|
|
||||
| 前端框架 | React 19 + React Router 7 |
|
||||
| 构建工具 | Vite 7 |
|
||||
| 桌面框架 | Tauri 2 (Rust) |
|
||||
| HTTP 服务 | Axum 0.8 (嵌入式 HTTP 服务器) |
|
||||
| 数据持久化 | JSON 文件存储 |
|
||||
| 实时通信 | Server-Sent Events (SSE) |
|
||||
| 内网穿透 | nps 客户端嵌入式集成 |
|
||||
|
||||
## 功能特性
|
||||
|
||||
### 媒体管理
|
||||
- **本地上传** — 支持 .mp4、.mkv、.avi、.jpg、.jpeg、.png 格式
|
||||
- **远程 URL** — 添加网络媒体资源链接
|
||||
- **媒体预览** — 图片与视频的模态框预览
|
||||
- **文件删除** — 从媒体库移除文件
|
||||
|
||||
### 播放控制
|
||||
- **播放/暂停** — 实时控制展示端的播放状态
|
||||
- **上一曲/下一曲** — 切换播放内容
|
||||
- **顺序/随机播放** — 两种播放模式
|
||||
- **图片时长可调** — 1~300 秒
|
||||
- **音量控制** — 实时同步调节
|
||||
|
||||
### 展示端体验
|
||||
- **沉浸式空间背景** — 深邃星空背景 + 透视网格 + 动态环境光晕
|
||||
- **自动播放** — 启动后自动加载播放列表
|
||||
- **视频/图片自适应** — 全屏覆盖显示
|
||||
- **声音管理** — 浏览器自动播放策略处理
|
||||
- **点击暂停/恢复** — 现场快速控制
|
||||
|
||||
### 系统功能
|
||||
- **全屏/窗口模式切换** — 自由切换展示形态
|
||||
- **最小化窗口** — 后台运行
|
||||
- **开机自启动** — 系统级自启（Tauri autostart 插件）
|
||||
- **关闭 → 最小化** — 防止误关闭
|
||||
- **内网穿透** — 内置 nps 客户端，支持外网远程访问
|
||||
|
||||
### 管理后台
|
||||
- **登录保护** — 账号密码验证
|
||||
- **实时状态显示** — 当前播放内容、状态
|
||||
- **上传进度条** — 文件上传实时反馈
|
||||
- **响应式设计** — 桌面/移动端自适应
|
||||
- **Toast 通知** — 操作即时反馈
|
||||
| `backend/` | **后端**：FastAPI（uv 管理），`:10085`；`app/` 内含全部 REST、MQTT、模拟引擎、AI/语音/RAG、手势识别。 |
|
||||
| `src/` | **前端**：React 19 + Vite + Tailwind + three.js + MQTT + Tauri API。路由 `/`（数据）、`/twin`（孪生）、`/ai`、`/screen`（媒体）、`/voice`、`/wall`。 |
|
||||
| `src-tauri/` | Tauri 2 桌面壳（Rust）。 |
|
||||
| `docs/` | 部署 / 更新 / 语音 / MQTT 手册 / ESP32。 |
|
||||
| `public/` | 静态资源。 |
|
||||
| `数据资料/` | 园区本地数据资料（不入库）。 |
|
||||
|
||||
## 快速开始
|
||||
|
||||
### 环境要求
|
||||
|
||||
- **Rust** (edition 2021)
|
||||
- **Node.js** >= 18
|
||||
- **Yarn** 或 npm
|
||||
- **Tauri CLI** (`cargo install tauri-cli`)
|
||||
|
||||
### 安装
|
||||
### 前端（vite）
|
||||
|
||||
```bash
|
||||
# 安装前端依赖
|
||||
yarn install
|
||||
|
||||
# 开发模式运行（同时启动 Vite 开发服务器和 Tauri 应用）
|
||||
yarn tauri dev
|
||||
|
||||
# 仅启动 Web 开发服务器（浏览器中预览）
|
||||
yarn dev
|
||||
yarn dev # Vite 开发服务器 :1420
|
||||
yarn build # 构建 dist/
|
||||
```
|
||||
|
||||
### 构建生产版本
|
||||
### 后端（uv）
|
||||
|
||||
```bash
|
||||
yarn tauri build
|
||||
cd backend
|
||||
uv sync
|
||||
cp .env.example .env # 按需填 DashScope Key、MQTT 账号
|
||||
uv run python main.py # 监听 0.0.0.0:10085
|
||||
```
|
||||
|
||||
构建产物位于 `src-tauri/target/release/` 目录。
|
||||
### Docker 部署（后端 + EMQX，推荐）
|
||||
|
||||
## 项目结构
|
||||
```bash
|
||||
export MQTT_PUBLIC_HOST=192.168.1.50 # 展播端可访问的服务器局域网 IP
|
||||
export DASHSCOPE_API_KEY=sk-xxx # 未配置时 AI 走本地规则引擎
|
||||
docker compose up -d --build
|
||||
curl http://<服务器IP>:10085/api/health
|
||||
```
|
||||
|
||||
端口：`10085`（API/后台/媒体）、`8765`（s2s 语音）、`1883/8083/18083`（EMQX）。数据卷 `dpm_media` / `dpm_knowledge` / `dpm_data`。
|
||||
|
||||
## 关键约定
|
||||
|
||||
- **进程管理**：见 `AGENTS.md` —— AI 禁止启动/重启/停止任何服务（后端 :10085 / s2s :8765 / vite :1420 / Tauri）。
|
||||
- **凭据**：`backend/.env` 与根 `.env.local`（`VITE_*` 前缀）含阿里云密钥与 MQTT 凭据，不入库。
|
||||
- **前端连接配置**：`.env.local` 中 `VITE_API_BASE` / `VITE_MQTT_URL` / `VITE_MQTT_USERNAME` / `VITE_MQTT_PASSWORD`；运行时优先级：exe 旁 `config.json` > 后端 `/api/config` > 内置默认 IP。
|
||||
|
||||
## 项目结构（源码）
|
||||
|
||||
```
|
||||
dpm/
|
||||
├── index.html # HTML 入口
|
||||
├── package.json # 前端依赖配置
|
||||
├── vite.config.js # Vite 构建配置（API 代理到 :10801）
|
||||
├── public/
|
||||
│ └── npc.zip # 内网穿透客户端（嵌入二进制）
|
||||
├── src/ # 前端源代码
|
||||
│ ├── main.jsx # React 入口
|
||||
│ ├── App.jsx # 路由配置（/admin, /screen）
|
||||
│ ├── pages/
|
||||
│ │ ├── Admin.jsx # 管理后台页面
|
||||
│ │ └── Screen.jsx # 大屏展示页面
|
||||
│ ├── utils/
|
||||
│ │ └── api.js # API 封装（REST + SSE）
|
||||
│ └── styles/
|
||||
│ ├── tokens.css # CSS 基础重置
|
||||
│ ├── admin.css # 管理后台样式
|
||||
│ └── screen.css # 大屏展示样式
|
||||
└── src-tauri/ # Rust 后端源代码
|
||||
├── Cargo.toml # Rust 依赖配置
|
||||
├── tauri.conf.json # Tauri 应用配置
|
||||
├── capabilities/
|
||||
│ └── default.json # Tauri 权限配置
|
||||
└── src/
|
||||
├── main.rs # 程序入口
|
||||
├── lib.rs # 模块导出 + 启动逻辑
|
||||
├── server.rs # Axum HTTP 服务器（路由 + 处理器）
|
||||
├── models.rs # 数据模型（Settings, PlaybackState, etc.）
|
||||
├── storage.rs # JSON 文件持久化存储
|
||||
└── events.rs # SSE 事件广播管理器
|
||||
├── package.json / vite.config.js
|
||||
├── src/ # 前端（React 19）
|
||||
│ ├── main.jsx # 入口：读 Tauri 运行配置 + /api/config
|
||||
│ ├── App.jsx # 路由（/ /twin /ai /screen /voice /wall）
|
||||
│ ├── pages/ # 六大屏页面
|
||||
│ ├── components/ # Building3D、AiChatPanel、GlobalVision 等
|
||||
│ ├── voice/ # 实时语音栈（s2s-ws-client、worklets）
|
||||
│ └── utils/ # mqtt、useMqttControl、updateCheck、statusBus
|
||||
├── backend/
|
||||
│ ├── main.py # 启动入口（prepare_env → uvicorn）
|
||||
│ └── app/ # config/routers/mqtt/sim_engine/storage/llm/...
|
||||
└── src-tauri/ # Tauri 2（Rust）
|
||||
```
|
||||
|
||||
## API 参考
|
||||
|
||||
系统内置 HTTP 服务器监听 `0.0.0.0:10801`，提供以下 REST API：
|
||||
|
||||
| 方法 | 路径 | 说明 |
|
||||
|------|------|------|
|
||||
| POST | `/api/login` | 管理员登录 |
|
||||
| GET | `/api/settings` | 获取系统设置 |
|
||||
| POST | `/api/settings` | 更新系统设置 |
|
||||
| GET | `/api/playlist` | 获取播放列表（含音量/模式/时长） |
|
||||
| POST | `/api/playlist/add` | 添加到播放列表 |
|
||||
| POST | `/api/playlist/remove` | 从播放列表移除 |
|
||||
| GET | `/media` | 获取所有媒体文件列表 |
|
||||
| POST | `/upload` | 上传媒体文件（multipart） |
|
||||
| POST | `/api/media/add-url` | 添加 URL 媒体 |
|
||||
| POST | `/api/delete` | 删除媒体文件 |
|
||||
| POST | `/api/control` | 播放控制（play/pause/next/prev） |
|
||||
| GET | `/api/state` | 获取当前播放状态 |
|
||||
| POST | `/api/state` | 更新播放状态 |
|
||||
| GET | `/api/events` | SSE 实时事件流 |
|
||||
| POST | `/api/display-command` | 显示命令（minimize 等） |
|
||||
| GET | `/file/*` | 静态媒体文件服务 |
|
||||
|
||||
### 默认登录凭据
|
||||
|
||||
- **账号**: `admin`
|
||||
- **密码**: `**********`
|
||||
|
||||
> 可通过编辑系统数据目录下的 `dpm/data.json` 修改。
|
||||
|
||||
### 数据存储位置
|
||||
|
||||
系统数据存储在操作系统标准数据目录下的 `dpm/` 文件夹中：
|
||||
|
||||
- **macOS**: `~/Library/Application Support/dpm/data.json`
|
||||
- **Linux**: `~/.local/share/dpm/data.json`
|
||||
- **Windows**: `C:\Users\<用户>\AppData\Roaming\dpm/data.json`
|
||||
|
||||
媒体文件存储在用户 `~/Downloads/Media/` 目录。
|
||||
|
||||
## 内网穿透
|
||||
|
||||
系统内置 [nps](https://github.com/ehang-io/nps) 客户端，自动从嵌入的 `public/npc.zip` 解压并启动，实现外网远程访问管理后台。
|
||||
|
||||
- 启动参数在 `server.rs` 中配置
|
||||
- *仅支持 Windows*
|
||||
- 自动设置 Unix 可执行权限
|
||||
- 解压到系统数据目录 `dpm/npc/`
|
||||
|
||||
## 开发说明
|
||||
|
||||
### 端口
|
||||
|
||||
| 端口 | 用途 |
|
||||
|------|------|
|
||||
| 10801 | 后端 HTTP API 服务器 |
|
||||
| 1420 | Vite 前端开发服务器 |
|
||||
|
||||
### Vite 开发代理
|
||||
|
||||
开发模式下，Vite 自动将 `/api`、`/file`、`/media`、`/upload` 路径代理到 `localhost:10801`，前端无需关心跨域问题。
|
||||
|
||||
### 构建
|
||||
|
||||
前端构建产物（`dist/`）通过 `RustEmbed` 编译到 Rust 二进制中，发布时只需分发单个可执行文件。
|
||||
|
||||
## 许可
|
||||
|
||||
本项目为昆明市大学生创业园公益定制开发。
|
||||
|
||||
---
|
||||
|
||||
*云南派音人工智能科技提供技术支持*
|
||||
昆明市大学生创业园公益定制开发，*云南派音人工智能科技提供技术支持*。
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
# OPC 智能园区后端（FastAPI + MQTT）
|
||||
|
||||
> 📍 工作区位置：`code/dpm/backend/`（DPM 园区大屏后端，独立仓库）。权威说明以本文档为准；系统/运维文档见 `design/dpm/`，前端见 `code/dpm/src/`。
|
||||
|
||||
独立后端服务：**所有数据通过 REST API 提供**，**页面/媒体控制通过 MQTT 下发**，媒体资源由后端统一存储与返回。
|
||||
|
||||
## 架构
|
||||
|
||||
+12
-12
@@ -14,34 +14,34 @@ log = logging.getLogger("dpm.ai_tools")
|
||||
|
||||
# ---------------- 工具执行 ----------------
|
||||
|
||||
def execute_tool(tool):
|
||||
"""执行一个工具调用：发布 MQTT 控制指令，返回是否发布成功"""
|
||||
def execute_tool(tool, screen_id=""):
|
||||
"""执行一个工具调用：发布 MQTT 控制指令（带发起屏幕 screen_id，只发到该屏），返回是否发布成功"""
|
||||
t = tool.get("type")
|
||||
params = tool.get("params") or {}
|
||||
if t == "navigate":
|
||||
return hub.publish_command("navigate", {"page": params.get("page", "home")}).get("published", False)
|
||||
return hub.publish_command("navigate", {"page": params.get("page", "home")}, screen_id=screen_id).get("published", False)
|
||||
if t == "control":
|
||||
action = params.get("action")
|
||||
if action in ("play", "pause", "next", "prev"):
|
||||
return hub.publish_command(action).get("published", False)
|
||||
return hub.publish_command(action, screen_id=screen_id).get("published", False)
|
||||
if action == "set_mode":
|
||||
return hub.publish_command("set_mode", {"mode": params.get("mode", "auto")}).get("published", False)
|
||||
return hub.publish_command("set_mode", {"mode": params.get("mode", "auto")}, screen_id=screen_id).get("published", False)
|
||||
return False
|
||||
if t == "alert":
|
||||
return hub.publish_command("alert", {"title": params.get("title", "提示"), "content": params.get("content", "")}).get("published", False)
|
||||
return hub.publish_command("alert", {"title": params.get("title", "提示"), "content": params.get("content", "")}, screen_id=screen_id).get("published", False)
|
||||
if t == "show_card":
|
||||
return hub.publish_command("show_card", params).get("published", False)
|
||||
return hub.publish_command("show_card", params, screen_id=screen_id).get("published", False)
|
||||
return False
|
||||
|
||||
|
||||
def run_tools(tools):
|
||||
"""执行一组工具，返回 [{type, ok}] 并汇总是否全部发布成功"""
|
||||
def run_tools(tools, screen_id=""):
|
||||
"""执行一组工具（带发起屏幕 screen_id），返回 [{type, ok}] 并汇总是否全部发布成功"""
|
||||
results = []
|
||||
all_ok = True
|
||||
for tool in tools or []:
|
||||
ok = bool(execute_tool(tool))
|
||||
log.info("ai_tools: 执行工具 %s params=%s -> %s", tool.get("type"),
|
||||
json.dumps(tool.get("params") or {}, ensure_ascii=False), ok)
|
||||
ok = bool(execute_tool(tool, screen_id))
|
||||
log.info("ai_tools: 执行工具 %s params=%s screen=%s -> %s", tool.get("type"),
|
||||
json.dumps(tool.get("params") or {}, ensure_ascii=False), screen_id or "all", ok)
|
||||
results.append({"type": tool.get("type"), "ok": ok})
|
||||
all_ok = all_ok and ok
|
||||
return results, all_ok
|
||||
|
||||
@@ -39,6 +39,10 @@ class Settings:
|
||||
DATA_FILE = Path(_env("DPM_DATA_FILE", str(BASE_DIR / "data.json")))
|
||||
DIST_DIR = Path(_env("DPM_DIST_DIR", str(BASE_DIR.parent / "dist"))) # 前端构建产物（可选托管）
|
||||
|
||||
# ---- 视频帧保存（摄像头识别帧，仅检测到人脸/手势时落盘，便于排查） ----
|
||||
SAVE_VISION_FRAMES = _env("DPM_SAVE_VISION_FRAMES", "1") == "1"
|
||||
VISION_FRAME_SAVE_DIR = Path(_env("DPM_VISION_SAVE_DIR", str(MEDIA_DIR / "video")))
|
||||
|
||||
# ---- MQTT（优先使用 .env 中的局域网 Broker 配置） ----
|
||||
MQTT_ENABLED = _env("DPM_MQTT_ENABLED", "1") == "1"
|
||||
MQTT_HOST = _env("DPM_MQTT_HOST", _env("MQTT_BROKER_HOST", "192.168.1.9"))
|
||||
|
||||
+10
-10
@@ -135,32 +135,32 @@ def _chat_once(messages, with_tools=True):
|
||||
return json.loads(resp.read().decode("utf-8"))
|
||||
|
||||
|
||||
def _fallback(messages):
|
||||
def _fallback(messages, screen_id=""):
|
||||
"""DashScope 不可用时回退本地规则引擎"""
|
||||
from .ai_tools import run_chat as rule_run
|
||||
|
||||
result = rule_run(messages)
|
||||
# 规则引擎产出的工具同样执行（MQTT 广播），并汇总发布结果
|
||||
_, all_ok = _exec_all(result.get("tools", []))
|
||||
# 规则引擎产出的工具同样执行（MQTT 广播，带发起屏幕），并汇总发布结果
|
||||
_, all_ok = _exec_all(result.get("tools", []), screen_id)
|
||||
result["model"] = "rule-engine"
|
||||
result["mqtt_published"] = all_ok
|
||||
return result
|
||||
|
||||
|
||||
def _exec_all(tools):
|
||||
"""执行工具列表，返回 (results, all_ok)"""
|
||||
def _exec_all(tools, screen_id=""):
|
||||
"""执行工具列表（带发起屏幕 screen_id，工具结果只发到该屏），返回 (results, all_ok)"""
|
||||
from .ai_tools import run_tools
|
||||
|
||||
results, all_ok = run_tools(tools)
|
||||
results, all_ok = run_tools(tools, screen_id)
|
||||
return results, all_ok
|
||||
|
||||
|
||||
def run_chat(messages):
|
||||
def run_chat(messages, screen_id=""):
|
||||
"""入口：{reply, tools, model}
|
||||
tools 已在后端执行（MQTT 广播），返回值供请求端本地同步执行"""
|
||||
tools 已在后端执行（MQTT 广播，带发起屏幕 screen_id），返回值供请求端本地同步执行"""
|
||||
if not settings.DASHSCOPE_API_KEY:
|
||||
log.warning("未配置 DASHSCOPE_API_KEY，使用本地规则引擎")
|
||||
return _fallback(messages)
|
||||
return _fallback(messages, screen_id)
|
||||
|
||||
# 注入园区知识库（标准资料口径）：按用户 query 检索命中段落动态注入，
|
||||
# 替代原 1200 字截断全量注入（避免尾部知识丢失）。无命中时回退 brief()。
|
||||
@@ -226,7 +226,7 @@ def run_chat(messages):
|
||||
"content": json.dumps({"ok": True}, ensure_ascii=False),
|
||||
})
|
||||
# 工具执行（MQTT 广播），并汇总发布结果
|
||||
results, all_ok = _exec_all(executed)
|
||||
results, all_ok = _exec_all(executed, screen_id)
|
||||
log.info("llm: 工具执行结果 %s", results)
|
||||
for t in executed:
|
||||
log.info("llm: 执行 %s %s", t["type"], json.dumps(t["params"], ensure_ascii=False))
|
||||
|
||||
+55
-17
@@ -6,6 +6,7 @@
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
import re
|
||||
import threading
|
||||
import time
|
||||
from contextlib import asynccontextmanager
|
||||
@@ -13,8 +14,9 @@ from pathlib import Path
|
||||
|
||||
from fastapi import FastAPI, Request
|
||||
from fastapi.middleware.cors import CORSMiddleware
|
||||
from fastapi.responses import FileResponse, JSONResponse
|
||||
from fastapi.responses import FileResponse, JSONResponse, RedirectResponse
|
||||
from fastapi.staticfiles import StaticFiles
|
||||
from pydantic import BaseModel
|
||||
|
||||
from .admin import admin_login, admin_logout, admin_page
|
||||
from .config import settings
|
||||
@@ -106,8 +108,8 @@ app.add_api_route("/admin/logout", admin_logout, methods=["GET"], include_in_sch
|
||||
_DOWNLOAD_ROOT = Path(__file__).resolve().parent.parent # backend/
|
||||
|
||||
|
||||
@app.get("/download", include_in_schema=False)
|
||||
@app.get("/11", include_in_schema=False)
|
||||
@app.get("/download")
|
||||
@app.get("/11")
|
||||
async def download_installer():
|
||||
"""下载 Windows 客户端安装包（backend/ 下 *.setup.exe，如 云超服..._x64-setup.exe）。"""
|
||||
for f in sorted(_DOWNLOAD_ROOT.glob("*setup.exe")):
|
||||
@@ -120,20 +122,56 @@ async def download_installer():
|
||||
return JSONResponse({"detail": "安装包不存在"}, status_code=404)
|
||||
|
||||
|
||||
# ---------- 客户端更新检测（仅 Windows：比对版本 → 下发安装包下载地址） ----------
|
||||
|
||||
def _latest_installer():
|
||||
"""返回 backend/ 下最新（按文件名排序取最大）的 *.setup.exe。"""
|
||||
for f in sorted(_DOWNLOAD_ROOT.glob("*setup.exe"), reverse=True):
|
||||
if f.is_file():
|
||||
return f
|
||||
return None
|
||||
|
||||
|
||||
def _version_from_name(name: str) -> str:
|
||||
"""从安装包文件名解析版本，如 `..._0.1.0_x64-setup.exe` → `0.1.0`。"""
|
||||
m = re.search(r"_(\d+\.\d+(?:\.\d+)?)", name or "")
|
||||
return m.group(1) if m else ""
|
||||
|
||||
|
||||
def _version_key(v: str):
|
||||
return tuple(int(x) for x in v.split(".") if x.isdigit())
|
||||
|
||||
|
||||
class UpdateCheckBody(BaseModel):
|
||||
current: str = "" # 前端上报的当前版本，如 0.1.0
|
||||
|
||||
|
||||
@app.post("/api/update/check")
|
||||
async def update_check(body: UpdateCheckBody):
|
||||
"""检测更新：返回最新版本号、安装包下载地址、是否需要更新（仅 Windows）。"""
|
||||
inst = _latest_installer()
|
||||
if not inst:
|
||||
return {"ok": False, "error": "installer not found"}
|
||||
latest = _version_from_name(inst.name)
|
||||
current = (body.current or "").strip()
|
||||
update_available = False
|
||||
if latest and current and current != latest:
|
||||
update_available = _version_key(latest) > _version_key(current)
|
||||
elif latest and not current:
|
||||
update_available = True
|
||||
return {
|
||||
"ok": True,
|
||||
"latest_version": latest,
|
||||
"installer_name": inst.name,
|
||||
"download_url": "/download",
|
||||
"update_available": update_available,
|
||||
}
|
||||
|
||||
|
||||
# ---------- 前端静态托管 + SPA 回退（浏览器直接访问 :8000 即可） ----------
|
||||
|
||||
@app.get("/{path:path}", include_in_schema=False)
|
||||
async def spa_fallback(path: str):
|
||||
if not DIST.exists():
|
||||
return JSONResponse({"detail": "前端未构建（dist 目录不存在）"}, status_code=404)
|
||||
dist = DIST.resolve()
|
||||
if not path:
|
||||
target = dist / "index.html"
|
||||
else:
|
||||
target = (dist / path).resolve()
|
||||
if target.is_file() and target.is_relative_to(dist):
|
||||
return FileResponse(target)
|
||||
index = dist / "index.html"
|
||||
if index.exists():
|
||||
return FileResponse(index)
|
||||
return JSONResponse({"detail": "Not Found"}, status_code=404)
|
||||
async def spa_fallback(request: Request):
|
||||
"""默认导向管理后台：屏蔽通过后端地址直接访问大屏前端（含首页 / 及 /twin /ai /screen 等）。
|
||||
/api、/admin、/static、/file 等已在更早路由/挂载处理，不会走到这里。"""
|
||||
return RedirectResponse(url="/admin", status_code=307)
|
||||
|
||||
+106
-14
@@ -33,7 +33,7 @@ class MqttHub:
|
||||
self.client = None
|
||||
self.connected = False
|
||||
self._lock = threading.RLock()
|
||||
self._screens = {} # client_id -> last_seen_ts
|
||||
self.devices = {} # device_id(client_id) -> {role, parent_id, last_seen}
|
||||
self.last_command = None # {action, params, ts, published}
|
||||
|
||||
# ---------- 生命周期 ----------
|
||||
@@ -81,30 +81,45 @@ class MqttHub:
|
||||
log.warning("MQTT 断开（rc=%s），自动重连中...", rc)
|
||||
|
||||
def _on_message(self, client, userdata, msg):
|
||||
"""接收大屏心跳：记录 client_id 与时间"""
|
||||
"""接收大屏心跳：更新设备表在线时间（设备由注册表登记 role/parent）"""
|
||||
if msg.topic == settings.TOPIC_HEARTBEAT:
|
||||
try:
|
||||
payload = json.loads(msg.payload.decode("utf-8"))
|
||||
cid = payload.get("client_id") or msg.topic
|
||||
with self._lock:
|
||||
self._screens[cid] = time.time()
|
||||
# 清理过期
|
||||
cutoff = time.time() - settings.SCREEN_TTL
|
||||
self._screens = {k: v for k, v in self._screens.items() if v > cutoff}
|
||||
cur = self.devices.get(cid, {"role": "", "parent_id": ""})
|
||||
cur["last_seen"] = time.time()
|
||||
self.devices[cid] = cur
|
||||
self._prune()
|
||||
except Exception: # noqa: BLE001
|
||||
pass
|
||||
|
||||
def _prune(self):
|
||||
"""剔除超时未上报的设备"""
|
||||
cutoff = time.time() - settings.SCREEN_TTL
|
||||
self.devices = {k: v for k, v in self.devices.items() if v.get("last_seen", 0) > cutoff}
|
||||
|
||||
# ---------- 状态查询 ----------
|
||||
def screens_online(self):
|
||||
with self._lock:
|
||||
cutoff = time.time() - settings.SCREEN_TTL
|
||||
return sum(1 for v in self._screens.values() if v > cutoff)
|
||||
return sum(1 for v in self.devices.values() if v.get("last_seen", 0) > time.time() - settings.SCREEN_TTL)
|
||||
|
||||
def status(self):
|
||||
with self._lock:
|
||||
self._prune()
|
||||
devices = [
|
||||
{
|
||||
"device_id": cid,
|
||||
"role": v.get("role", ""),
|
||||
"parent_id": v.get("parent_id", ""),
|
||||
}
|
||||
for cid, v in self.devices.items()
|
||||
]
|
||||
return {
|
||||
"mqtt_connected": self.connected,
|
||||
"mqtt_host": f"{settings.MQTT_HOST}:{settings.MQTT_PORT}",
|
||||
"screens_online": self.screens_online(),
|
||||
"screens_online": len(devices),
|
||||
"devices": devices,
|
||||
"last_command": self.last_command,
|
||||
}
|
||||
|
||||
@@ -120,26 +135,103 @@ class MqttHub:
|
||||
log.warning("MQTT 发布失败: %s", e)
|
||||
return False
|
||||
|
||||
def publish_command(self, action, params=None):
|
||||
"""统一指令信封：{cmd_id, ts, action, params, published}
|
||||
MQTT 广播 + SSE 兼容推送；published 标记是否真正发布成功"""
|
||||
def register_device(self, device_id, role="", parent_id=""):
|
||||
"""前端上报设备：唯一 device_id + 角色(main/secondary) + 从属主屏 parent_id。"""
|
||||
device_id = (device_id or "").strip()
|
||||
if not device_id:
|
||||
return
|
||||
with self._lock:
|
||||
cur = self.devices.get(device_id, {})
|
||||
if role:
|
||||
cur["role"] = role
|
||||
if parent_id:
|
||||
cur["parent_id"] = parent_id
|
||||
cur["last_seen"] = time.time()
|
||||
self.devices[device_id] = cur
|
||||
self._prune()
|
||||
log.info("mqtt: 设备注册 id=%s role=%s parent=%s", device_id, role or cur.get("role"), parent_id or cur.get("parent_id"))
|
||||
|
||||
def publish_command(self, action, params=None, screen_id=None, screen_role=None):
|
||||
"""按客户端(设备)路由下发指令：
|
||||
· screen_id 精确到某台设备 → 只发到该设备独有 topic
|
||||
· screen_role(main/secondary) → 解析到该角色的设备，发到其独有 topic
|
||||
· 两者皆空（全部）→ 遍历在线设备列表，依次下发到各自独有 topic
|
||||
客户端只订阅自己的独有频道，因此定向指令绝不会被其它屏幕收到。"""
|
||||
payload = {
|
||||
"cmd_id": new_cmd_id(),
|
||||
"ts": int(time.time() * 1000),
|
||||
"action": action,
|
||||
"params": params or {},
|
||||
}
|
||||
ok = self.publish(settings.TOPIC_COMMAND, payload)
|
||||
target = None
|
||||
requested = bool(screen_id or screen_role in ("main", "secondary"))
|
||||
if screen_id:
|
||||
target = (screen_id or "").strip()
|
||||
elif screen_role in ("main", "secondary"):
|
||||
with self._lock:
|
||||
for cid, v in self.devices.items():
|
||||
if v.get("role") == screen_role:
|
||||
target = cid
|
||||
break
|
||||
if target:
|
||||
# 定向：只发到目标设备独有频道
|
||||
return self._publish_to(payload, target, screen_role)
|
||||
if requested:
|
||||
# 指定了目标但未找到对应设备：不广播，避免误发到全部
|
||||
log.warning("mqtt: 目标设备未找到（id=%s role=%s），未下发", screen_id, screen_role)
|
||||
payload["published"] = False
|
||||
with self._lock:
|
||||
self.last_command = {"action": action, "params": params or {}, "ts": payload["ts"], "published": False, "topic": "none"}
|
||||
return payload
|
||||
|
||||
# 全部：遍历在线设备列表，依次下发到各自独有频道
|
||||
with self._lock:
|
||||
cutoff = time.time() - settings.SCREEN_TTL
|
||||
online = [cid for cid, v in self.devices.items() if v.get("last_seen", 0) > cutoff]
|
||||
ok = True
|
||||
published_topics = []
|
||||
for cid in online:
|
||||
p = dict(payload)
|
||||
p["screen_id"] = cid
|
||||
with self._lock:
|
||||
p["screen_role"] = self.devices.get(cid, {}).get("role", "")
|
||||
if self.publish(f"{settings.TOPIC_COMMAND}/{cid}", p):
|
||||
bus.emit(p)
|
||||
published_topics.append(cid)
|
||||
else:
|
||||
ok = False
|
||||
payload["published"] = ok
|
||||
payload["screen_id"] = ""
|
||||
payload["screen_role"] = "both"
|
||||
with self._lock:
|
||||
self.last_command = {
|
||||
"action": action,
|
||||
"params": params or {},
|
||||
"ts": payload["ts"],
|
||||
"published": ok,
|
||||
"topic": "all",
|
||||
}
|
||||
log.info("mqtt: 全部下发 %s 到 %d 台设备", action, len(published_topics))
|
||||
return payload
|
||||
|
||||
def _publish_to(self, payload, target, screen_role=""):
|
||||
"""发到指定设备独有频道，并记录 last_command。"""
|
||||
topic = f"{settings.TOPIC_COMMAND}/{target}"
|
||||
payload["screen_id"] = target
|
||||
with self._lock:
|
||||
payload["screen_role"] = self.devices.get(target, {}).get("role", screen_role or "")
|
||||
ok = self.publish(topic, payload)
|
||||
payload["published"] = ok
|
||||
with self._lock:
|
||||
self.last_command = {
|
||||
"action": payload["action"],
|
||||
"params": payload.get("params") or {},
|
||||
"ts": payload["ts"],
|
||||
"published": ok,
|
||||
"topic": topic,
|
||||
}
|
||||
if ok:
|
||||
bus.emit(payload) # SSE 兼容通道（仅发布成功时推送）
|
||||
bus.emit(payload)
|
||||
return payload
|
||||
|
||||
def publish_tick(self, snapshot):
|
||||
|
||||
+28
-9
@@ -73,6 +73,8 @@ class StateBody(BaseModel):
|
||||
class DisplayCommandBody(BaseModel):
|
||||
action: str
|
||||
params: dict = Field(default_factory=dict)
|
||||
screen_id: str = "" # 目标屏幕唯一 client_id（精确到某台）
|
||||
screen_role: str = "" # main | secondary（按角色定位）
|
||||
|
||||
|
||||
class ChatMessage(BaseModel):
|
||||
@@ -82,6 +84,7 @@ class ChatMessage(BaseModel):
|
||||
|
||||
class ChatBody(BaseModel):
|
||||
messages: list[ChatMessage]
|
||||
screen_id: str = "" # 发起对话的屏幕设备 id（工具结果只发到该屏）
|
||||
|
||||
|
||||
# ==================== 认证 / 设置 ====================
|
||||
@@ -449,7 +452,7 @@ async def ai_chat(body: ChatBody):
|
||||
"""大模型对话：工具调用（切页/控制/卡片/通知）经 MQTT 广播到所有大屏
|
||||
返回 {reply, tools, model}；tools 供请求端本地同步执行"""
|
||||
messages = [m.model_dump() for m in body.messages]
|
||||
result = llm_run_chat(messages)
|
||||
result = llm_run_chat(messages, body.screen_id or "")
|
||||
return result
|
||||
|
||||
|
||||
@@ -480,18 +483,35 @@ _VALID_DISPLAY_ACTIONS = {
|
||||
"voice_start", "voice_stop", "voice_refresh",
|
||||
# 媒体
|
||||
"play", "pause", "next", "prev", "set_mode", "play_target",
|
||||
# 双屏
|
||||
"dual_screen",
|
||||
# 全局
|
||||
"alert", "show_card", "minimize", "settings_changed", "playlist_changed",
|
||||
}
|
||||
|
||||
|
||||
class RegisterBody(BaseModel):
|
||||
device_id: str = "" # 唯一设备标识（MQTT client_id）
|
||||
role: str = "" # main | secondary
|
||||
parent_id: str = "" # 副屏从属的主屏 device_id；主屏为空
|
||||
|
||||
|
||||
@router.post("/api/display/register")
|
||||
async def display_register(body: RegisterBody):
|
||||
"""设备注册：前端上报唯一 device_id + 角色 + 从属主屏，后端维护设备表供精准控制。"""
|
||||
hub.register_device(body.device_id, body.role, body.parent_id)
|
||||
return {"ok": True, "device_id": body.device_id, "role": body.role, "parent_id": body.parent_id}
|
||||
|
||||
|
||||
@router.post("/api/display/command")
|
||||
async def display_command_publish(body: DisplayCommandBody):
|
||||
"""管理端/任意客户端通过 REST 发控制指令 → 后端转 MQTT 广播给所有大屏"""
|
||||
if body.action not in _VALID_DISPLAY_ACTIONS:
|
||||
return JSONResponse({"ok": False, "error": f"不支持的指令: {body.action}"}, status_code=400)
|
||||
cmd = hub.publish_command(body.action, body.params)
|
||||
# 按屏幕终端路由：screen_id 精确到某客户端；screen_role 按主/副定位；皆空 → 全局广播
|
||||
cmd = hub.publish_command(body.action, body.params, body.screen_id or None, body.screen_role or None)
|
||||
return {"ok": cmd.get("published", False), "cmd_id": cmd["cmd_id"], "action": body.action,
|
||||
"screen_id": body.screen_id or "", "screen_role": body.screen_role or "both",
|
||||
"mqtt_connected": hub.connected}
|
||||
|
||||
|
||||
@@ -531,14 +551,12 @@ class VisionEventBody(BaseModel):
|
||||
|
||||
@router.post("/api/vision/event")
|
||||
async def vision_event(body: VisionEventBody):
|
||||
"""前端摄像头识别状态/触发上报；后端记录详细日志，triggered 时广播 alert 到全屏。"""
|
||||
"""前端摄像头识别状态/触发上报；后端记录详细日志。
|
||||
主动问候已关闭，故不再对 triggered 广播 alert（避免右上角弹「已主动问候」误导提示）。"""
|
||||
log.info(
|
||||
"vision event=%s faces=%d dwell_ms=%d detail=%s",
|
||||
body.event, body.faces, body.dwell_ms, body.detail,
|
||||
)
|
||||
if body.event == "triggered":
|
||||
hub.publish_command("alert", {"text": "有访客正对屏幕，语音助手已主动问候", "faces": body.faces})
|
||||
log.info("vision triggered -> alert 已广播（faces=%d）", body.faces)
|
||||
return {"ok": True}
|
||||
|
||||
|
||||
@@ -587,6 +605,7 @@ def vision_llm(body: VisionLlmBody):
|
||||
class ToolsExecBody(BaseModel):
|
||||
name: str = ""
|
||||
args: dict = {}
|
||||
screen_id: str = "" # 发起语音对话的屏幕设备 id（工具结果只发到该屏）
|
||||
|
||||
@router.post("/api/tools/exec")
|
||||
def tools_exec(body: ToolsExecBody):
|
||||
@@ -595,10 +614,10 @@ def tools_exec(body: ToolsExecBody):
|
||||
if not name:
|
||||
return {"ok": False, "error": "missing tool name"}
|
||||
from .tools import exec_tool
|
||||
result = exec_tool(name, body.args or {})
|
||||
log.info("api: /api/tools/exec name=%s args=%s result=%s", name,
|
||||
result = exec_tool(name, body.args or {}, body.screen_id or "")
|
||||
log.info("api: /api/tools/exec name=%s args=%s screen=%s result=%s", name,
|
||||
json.dumps(body.args or {}, ensure_ascii=False)[:300],
|
||||
str(result)[:400])
|
||||
body.screen_id or "all", str(result)[:400])
|
||||
return {"ok": True, "result": result}
|
||||
|
||||
@router.get("/api/kb/brief")
|
||||
|
||||
+11
-7
@@ -60,15 +60,15 @@ def _query_companies(args):
|
||||
}, ensure_ascii=False)
|
||||
|
||||
|
||||
def _control_display(args):
|
||||
"""大屏显示控制：切页 / 弹通知 / 媒体播放暂停（经 MQTT 广播到大屏）"""
|
||||
def _control_display(args, screen_id=""):
|
||||
"""大屏显示控制：切页 / 弹通知 / 媒体播放暂停（经 MQTT 按发起屏幕下发）"""
|
||||
from .mqtt import hub
|
||||
action = (args.get("action") or "").strip()
|
||||
target = (args.get("target") or "").strip()
|
||||
if action not in ("switch_page", "alert", "media_play", "media_pause"):
|
||||
return json.dumps({"error": f"不支持的 action: {action}"}, ensure_ascii=False)
|
||||
params = {"target": target} if target else {}
|
||||
ok = hub.publish_command(action, params).get("published", False)
|
||||
ok = hub.publish_command(action, params, screen_id=screen_id).get("published", False)
|
||||
return json.dumps({"ok": ok, "action": action, "target": target}, ensure_ascii=False)
|
||||
|
||||
|
||||
@@ -125,15 +125,19 @@ def tool_schemas():
|
||||
]
|
||||
|
||||
|
||||
def exec_tool(name: str, args: dict):
|
||||
"""执行工具 → 返回给 LLM 的字符串结果（JSON）"""
|
||||
def exec_tool(name: str, args: dict, screen_id: str = ""):
|
||||
"""执行工具 → 返回给 LLM 的字符串结果（JSON）。
|
||||
控制类工具（control_display）按发起屏幕 screen_id 只下发到该屏。"""
|
||||
t = TOOLS.get(name)
|
||||
if not t:
|
||||
log.warning("tools: 未知工具 %s", name)
|
||||
return json.dumps({"error": f"未知工具: {name}"}, ensure_ascii=False)
|
||||
try:
|
||||
result = t["fn"](args or {})
|
||||
log.info("tools: 执行工具 %s args=%s", name, json.dumps(args, ensure_ascii=False))
|
||||
if name == "control_display":
|
||||
result = _control_display(args or {}, screen_id)
|
||||
else:
|
||||
result = t["fn"](args or {})
|
||||
log.info("tools: 执行工具 %s args=%s screen=%s", name, json.dumps(args, ensure_ascii=False), screen_id or "all")
|
||||
log.info("tools: 结果 -> %s", _trunc(result, 1000))
|
||||
return result
|
||||
except Exception as e: # noqa: BLE001
|
||||
|
||||
@@ -19,8 +19,12 @@ from pathlib import Path
|
||||
|
||||
from PIL import Image
|
||||
|
||||
from .config import settings
|
||||
|
||||
log = logging.getLogger("dpm.vision")
|
||||
|
||||
_frame_seq = 0
|
||||
|
||||
MODEL_DIR = Path(__file__).resolve().parent.parent / "models"
|
||||
FACE_MODEL_PATH = MODEL_DIR / "yolov8n-face.pt"
|
||||
POSE_MODEL_PATH = MODEL_DIR / "yolov8n-pose.pt"
|
||||
@@ -188,6 +192,40 @@ def predict_jpeg(jpeg_bytes: bytes):
|
||||
}
|
||||
|
||||
|
||||
def _should_save(data: dict) -> bool:
|
||||
"""仅当画面检测到人脸/手势时才值得落盘保存。"""
|
||||
return (
|
||||
int(data.get("faces") or 0) > 0
|
||||
or bool(data.get("raised"))
|
||||
or bool(data.get("fists"))
|
||||
or bool(data.get("both_up"))
|
||||
or bool(data.get("pointing"))
|
||||
or bool(data.get("hands_close"))
|
||||
)
|
||||
|
||||
|
||||
def maybe_save_frame(jpeg: bytes, data: dict):
|
||||
"""仅将命中（有人/有动作）的 JPEG 帧落盘到 MEDIA_DIR/video/<小时目录>/，便于排查识别链路。
|
||||
目录按小时分割（YYYYMMDD_HH），不自动清理。
|
||||
可通过 DPM_SAVE_VISION_FRAMES=0 关闭，或 DPM_VISION_SAVE_DIR 改目录。"""
|
||||
global _frame_seq
|
||||
if not settings.SAVE_VISION_FRAMES:
|
||||
return
|
||||
if not _should_save(data):
|
||||
return
|
||||
_frame_seq += 1
|
||||
ts = time.strftime("%Y%m%d_%H%M%S")
|
||||
hour = time.strftime("%Y%m%d_%H")
|
||||
try:
|
||||
save_dir = settings.VISION_FRAME_SAVE_DIR / hour
|
||||
save_dir.mkdir(parents=True, exist_ok=True)
|
||||
path = save_dir / f"{ts}_{_frame_seq:05d}.jpg"
|
||||
with open(path, "wb") as f:
|
||||
f.write(jpeg)
|
||||
except Exception as e: # noqa: BLE001
|
||||
log.warning("vision: 保存帧失败 %s", e)
|
||||
|
||||
|
||||
def predict_base64(b64: str):
|
||||
"""入口：base64 JPEG → 检测结果 dict（含 ok 标记，失败时带 error）"""
|
||||
try:
|
||||
@@ -198,6 +236,7 @@ def predict_base64(b64: str):
|
||||
try:
|
||||
data = predict_jpeg(jpeg)
|
||||
data["ok"] = True
|
||||
maybe_save_frame(jpeg, data)
|
||||
return data
|
||||
except Exception as e: # noqa: BLE001
|
||||
log.error("vision: 推理失败 %s", e, exc_info=True)
|
||||
|
||||
@@ -1147,6 +1147,13 @@ button { font-family: "Poppins", "PingFang SC", sans-serif; cursor: pointer; tra
|
||||
background: rgba(0, 189, 125, 0.14);
|
||||
transform: translateY(-1px);
|
||||
}
|
||||
.screen-target.active {
|
||||
background: #1D5DCE;
|
||||
border-color: #1D5DCE;
|
||||
color: #fff;
|
||||
box-shadow: 0 4px 12px rgba(29, 93, 206, 0.3);
|
||||
}
|
||||
.screen-target.active:hover { background: #174fb0; }
|
||||
.media-url {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
@@ -1258,3 +1265,128 @@ button { font-family: "Poppins", "PingFang SC", sans-serif; cursor: pointer; tra
|
||||
font-size: 13px; color: #1d2c44; outline: none;
|
||||
}
|
||||
.mqtt-input:focus { border-color: #2f6bff; }
|
||||
|
||||
/* ============ 弱化连接状态（沉底、紧凑、低调） ============ */
|
||||
.conn-card { opacity: 0.88; }
|
||||
.conn-card .card-header { padding-bottom: 6px; }
|
||||
.conn-card .conn-status { display: flex; flex-wrap: wrap; gap: 6px 22px; }
|
||||
.conn-card .conn-row { min-width: 150px; }
|
||||
.conn-card .conn-label { font-size: 11px; }
|
||||
|
||||
/* ============ 大屏控制：mqtt-group 按操作优先级排序 ============ */
|
||||
.mqtt-card { display: flex; flex-direction: column; }
|
||||
.mqtt-card .mqtt-group { order: 50; } /* 未显式设置 order 的组默认靠后 */
|
||||
|
||||
/* ============ 手机适配（响应式） ============ */
|
||||
@media (max-width: 900px) {
|
||||
.main-grid { grid-template-columns: 1fr; padding: 16px; gap: 16px; }
|
||||
}
|
||||
@media (max-width: 600px) {
|
||||
.main-grid { padding: 10px; gap: 12px; }
|
||||
.topbar { flex-wrap: wrap; row-gap: 8px; }
|
||||
.topbar-center { order: 3; width: 100%; justify-content: flex-start; }
|
||||
.page-switch-grid { grid-template-columns: repeat(2, 1fr); gap: 8px; }
|
||||
.mqtt-row { flex-wrap: wrap; gap: 8px; }
|
||||
.page-switch-btn { flex: 1 1 45%; }
|
||||
.form-inline { flex-direction: column; }
|
||||
.form-row { flex-wrap: wrap; }
|
||||
.conn-card .conn-status { flex-direction: column; gap: 6px; }
|
||||
.card { padding: 16px; }
|
||||
}
|
||||
|
||||
/* ============ 遥控器式快捷控制条 ============ */
|
||||
.remote-bar {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
flex-wrap: wrap;
|
||||
gap: 10px 18px;
|
||||
max-width: 1400px;
|
||||
margin: 16px auto 0;
|
||||
padding: 12px 16px;
|
||||
background: var(--surface);
|
||||
border: 1px solid var(--border-subtle);
|
||||
border-radius: var(--radius-md);
|
||||
box-shadow: var(--shadow-card);
|
||||
}
|
||||
.remote-group { display: flex; gap: 6px; }
|
||||
.remote-btn {
|
||||
width: 42px; height: 42px;
|
||||
display: inline-flex; align-items: center; justify-content: center;
|
||||
font-size: 24px; line-height: 1;
|
||||
border: 1px solid var(--border-default);
|
||||
border-radius: var(--radius-sm);
|
||||
background: #fff; color: var(--text);
|
||||
cursor: pointer;
|
||||
transition: var(--transition);
|
||||
}
|
||||
.remote-btn:hover { background: var(--primary-subtle); border-color: var(--primary); color: var(--primary); }
|
||||
.remote-target {
|
||||
display: flex; align-items: center; gap: 8px;
|
||||
padding: 5px 12px;
|
||||
background: var(--primary-subtle);
|
||||
border: 1px solid rgba(0,189,125,.2);
|
||||
border-radius: var(--radius-md);
|
||||
}
|
||||
.remote-label { font-size: 12px; font-weight: 600; color: var(--text-secondary); white-space: nowrap; }
|
||||
.remote-count { font-size: 11px; color: var(--text-dim); white-space: nowrap; }
|
||||
.remote-target select {
|
||||
min-width: 160px;
|
||||
padding: 6px 10px;
|
||||
border: 1px solid var(--border-default);
|
||||
border-radius: var(--radius-sm);
|
||||
background: #fff; color: var(--text); font-size: 13px;
|
||||
}
|
||||
.remote-pages { display: flex; gap: 6px; flex-wrap: wrap; }
|
||||
|
||||
@media (max-width: 600px) {
|
||||
.remote-bar { margin: 10px; padding: 10px; }
|
||||
.remote-target { flex: 1 1 100%; }
|
||||
.remote-target select { flex: 1; min-width: 0; }
|
||||
}
|
||||
|
||||
/* 遥控条：页面切换合并模块 + 分隔线 */
|
||||
.remote-nav { display: flex; align-items: center; flex-wrap: wrap; gap: 6px; }
|
||||
.remote-sep { width: 1px; height: 26px; background: var(--border-default); margin: 0 4px; }
|
||||
@media (max-width: 600px) {
|
||||
.remote-nav { width: 100%; }
|
||||
.remote-sep { display: none; }
|
||||
}
|
||||
|
||||
/* ============ 顶部控制模块（选择屏幕 / 页面切换，独立上下排列） ============ */
|
||||
.control-stack {
|
||||
max-width: 1400px;
|
||||
margin: 16px auto 0;
|
||||
padding: 0 32px;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 24px;
|
||||
}
|
||||
.control-card { margin: 0; }
|
||||
@media (max-width: 600px) {
|
||||
.control-stack { padding: 0 10px; }
|
||||
}
|
||||
|
||||
/* 页面切换：左右箭头独立一行，与下方页面按钮分隔开 */
|
||||
.page-nav-prev {
|
||||
display: flex;
|
||||
gap: 8px;
|
||||
margin-bottom: 12px;
|
||||
padding-bottom: 12px;
|
||||
border-bottom: 1px solid var(--border-subtle);
|
||||
}
|
||||
.page-nav-links { padding-top: 2px; }
|
||||
@media (max-width: 600px) {
|
||||
.page-nav-prev { justify-content: center; }
|
||||
.page-nav-links { justify-content: flex-start; }
|
||||
}
|
||||
|
||||
/* 卡片图标：继承主题主色（appica color -> currentColor） */
|
||||
.card-header .card-icon { color: var(--primary); }
|
||||
|
||||
/* 页面按钮：当前激活页高亮 */
|
||||
.page-switch-btn.active {
|
||||
background: var(--primary);
|
||||
border-color: var(--primary);
|
||||
color: #fff;
|
||||
box-shadow: 0 2px 8px rgba(0,189,125,0.35);
|
||||
}
|
||||
|
||||
+80
-7
@@ -10,6 +10,61 @@
|
||||
var fullscreen = !!init.fullscreen;
|
||||
var autostart = !!init.autostart;
|
||||
|
||||
/* 目标屏幕（按 MQTT 客户端精准控制）：client_id + role */
|
||||
var screenId = '';
|
||||
var screenRole = '';
|
||||
|
||||
/* 状态识别：点击某页面后，把该页操作模块前置到页面控制下方，并高亮当前页面按钮 */
|
||||
function activatePage(page) {
|
||||
if (!page) return;
|
||||
var cards = document.querySelectorAll('.main-grid .card[data-page]');
|
||||
for (var i = 0; i < cards.length; i++) cards[i].style.order = '';
|
||||
var active = null;
|
||||
for (var j = 0; j < cards.length; j++) {
|
||||
if (cards[j].getAttribute('data-page') === page) { active = cards[j]; break; }
|
||||
}
|
||||
if (active) active.style.order = -10; // 前置到 main-grid 顶部（页面控制下方）
|
||||
var navs = document.querySelectorAll('[data-nav]');
|
||||
for (var k = 0; k < navs.length; k++) navs[k].classList.toggle('active', navs[k].getAttribute('data-nav') === page);
|
||||
}
|
||||
|
||||
function applyScreenTarget(sel) {
|
||||
sel = sel || document.getElementById('screenTarget');
|
||||
if (!sel) return;
|
||||
var v = sel.value || '';
|
||||
screenId = v;
|
||||
var opt = null;
|
||||
if (v) { for (var i = 0; i < sel.options.length; i++) { if (sel.options[i].value === v) { opt = sel.options[i]; break; } } }
|
||||
screenRole = opt ? (opt.getAttribute('data-role') || '') : '';
|
||||
}
|
||||
|
||||
function loadScreens() {
|
||||
fetch('/api/display/state', { cache: 'no-store' })
|
||||
.then(function (r) { return r.json(); })
|
||||
.then(function (d) {
|
||||
var sel = document.getElementById('screenTarget');
|
||||
var cnt = document.getElementById('screenCount');
|
||||
if (!sel) return;
|
||||
var devices = (d && d.devices) || [];
|
||||
if (cnt) cnt.textContent = '在线设备：' + (devices.length || 0) + ' 台';
|
||||
var cur = sel.value;
|
||||
sel.innerHTML = '<option value="">全部屏幕</option>';
|
||||
devices.forEach(function (dev) {
|
||||
var role = dev.role || '';
|
||||
var name = role === 'main' ? '主屏' : role === 'secondary' ? '副屏' : (role || '屏幕');
|
||||
var parent = (dev.role === 'secondary' && dev.parent_id) ? '·属于:' + String(dev.parent_id).slice(0, 8) : '';
|
||||
var opt = document.createElement('option');
|
||||
opt.value = dev.device_id;
|
||||
opt.setAttribute('data-role', role);
|
||||
opt.textContent = name + parent + ' · ' + String(dev.device_id).slice(0, 16);
|
||||
sel.appendChild(opt);
|
||||
});
|
||||
if (cur) sel.value = cur;
|
||||
applyScreenTarget(sel);
|
||||
})
|
||||
.catch(function () {});
|
||||
}
|
||||
|
||||
/* ---------- Toast ---------- */
|
||||
var toastContainer = document.getElementById('toastContainer');
|
||||
function toast(msg, type) {
|
||||
@@ -79,6 +134,7 @@
|
||||
setConn('api', true, '在线');
|
||||
setConn('mqtt', !!d.mqtt_connected, d.mqtt_connected ? '已连接（' + (d.mqtt_host || '') + '）' : '未连接');
|
||||
setConn('screens', (d.screens_online || 0) > 0, (d.screens_online || 0) + ' 台');
|
||||
loadScreens(); // 刷新目标屏幕下拉（按 MQTT 客户端）
|
||||
var lc = d.last_command;
|
||||
if (lc) {
|
||||
var names = { navigate: '页面切换', play: '播放', pause: '暂停', next: '下一项', prev: '上一项',
|
||||
@@ -95,6 +151,12 @@
|
||||
});
|
||||
}
|
||||
if (connEls.api) { pollStatus(); setInterval(pollStatus, 3000); }
|
||||
// 目标屏幕下拉：变更时记录目标 client_id + role；初始加载一次
|
||||
var screenSel = document.getElementById('screenTarget');
|
||||
if (screenSel) {
|
||||
screenSel.addEventListener('change', function () { applyScreenTarget(screenSel); });
|
||||
loadScreens();
|
||||
}
|
||||
|
||||
/* =========================================================
|
||||
事件委托：页面切换 / 播放控制 / 媒体操作
|
||||
@@ -129,8 +191,15 @@
|
||||
if (btn.hasAttribute('data-mqtt')) {
|
||||
var cmd = null;
|
||||
try { cmd = JSON.parse(btn.getAttribute('data-mqtt')); } catch (err) { toast('指令 JSON 错误', 'error'); return; }
|
||||
// 屏幕终端精准控制：除「双屏开关」外，注入目标 client_id + role；全空=全局广播
|
||||
if (cmd.action !== 'dual_screen') { cmd.screen_id = screenId; cmd.screen_role = screenRole; }
|
||||
// 状态识别：切页命令 → 前置对应页面操作模块
|
||||
if (cmd.action === 'navigate' && cmd.params && cmd.params.page) activatePage(cmd.params.page);
|
||||
var tgtTxt = screenId
|
||||
? (cmd.screen_role === 'main' ? '主屏' : cmd.screen_role === 'secondary' ? '副屏' : '屏幕') + '·' + String(screenId).slice(0, 10)
|
||||
: '全部';
|
||||
post('/api/display/command', cmd, function (d) {
|
||||
toast(d && d.ok !== false ? '指令已发送：' + (cmd.action || '') : '发送失败（MQTT 未连接？）',
|
||||
toast(d && d.ok !== false ? ('指令已发送：' + (cmd.action || '') + ' → ' + tgtTxt) : '发送失败（MQTT 未连接？）',
|
||||
d && d.ok !== false ? 'ok' : 'error');
|
||||
});
|
||||
return;
|
||||
@@ -140,7 +209,7 @@
|
||||
if (btn.id === 'mqttAiSend') {
|
||||
var text = (document.getElementById('mqttAiInput').value || '').trim();
|
||||
if (!text) { toast('请输入问题', 'error'); return; }
|
||||
post('/api/display/command', { action: 'ai_input', params: { text: text } }, function () {
|
||||
post('/api/display/command', { action: 'ai_input', params: { text: text }, screen_id: screenId, screen_role: screenRole }, function () {
|
||||
toast('已发送问题到大屏 AI 页');
|
||||
});
|
||||
return;
|
||||
@@ -150,17 +219,21 @@
|
||||
if (btn.id === 'mqttAlertSend') {
|
||||
var alertText = (document.getElementById('mqttAlertText').value || '').trim();
|
||||
if (!alertText) { toast('请输入通知内容', 'error'); return; }
|
||||
post('/api/display/command', { action: 'alert', params: { text: alertText } }, function () {
|
||||
toast('通知已广播');
|
||||
post('/api/display/command', { action: 'alert', params: { text: alertText }, screen_id: screenId, screen_role: screenRole }, function () {
|
||||
toast('通知已发送');
|
||||
});
|
||||
return;
|
||||
}
|
||||
|
||||
// 页面切换
|
||||
// 页面切换（绑定目标屏幕：screen_id 精确某台；空=全部）
|
||||
if (btn.hasAttribute('data-nav')) {
|
||||
var page = btn.getAttribute('data-nav');
|
||||
post('/api/display/command', { action: 'navigate', params: { page: page } }, function () {
|
||||
toast('已切换大屏至「' + btn.textContent.trim() + '」');
|
||||
activatePage(page); // 状态识别：前置该页操作模块
|
||||
var tgtTxt = screenId
|
||||
? (screenRole === 'main' ? '主屏' : screenRole === 'secondary' ? '副屏' : '屏幕') + '·' + String(screenId).slice(0, 10)
|
||||
: '全部';
|
||||
post('/api/display/command', { action: 'navigate', params: { page: page }, screen_id: screenId, screen_role: screenRole }, function () {
|
||||
toast('已切换大屏至「' + btn.textContent.trim() + '」 → ' + tgtTxt);
|
||||
});
|
||||
return;
|
||||
}
|
||||
|
||||
+440
-309
@@ -1,144 +1,246 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="zh-CN">
|
||||
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>管理后台 · 昆明市大学生创业园</title>
|
||||
<link rel="stylesheet" href="/static/admin.css">
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>管理后台 · 昆明市大学生创业园</title>
|
||||
<link rel="stylesheet" href="/static/admin.css">
|
||||
</head>
|
||||
|
||||
<body>
|
||||
{% if not logged_in %}
|
||||
<!-- ==================== 登录页 ==================== -->
|
||||
<div class="admin-page">
|
||||
<div class="login-wrapper">
|
||||
<div class="login-card">
|
||||
<div class="login-brand">
|
||||
<div class="icon">
|
||||
<div class="iso-top"></div>
|
||||
<div class="iso-left"></div>
|
||||
<div class="iso-right"></div>
|
||||
<div class="iso-dot"></div>
|
||||
{% if not logged_in %}
|
||||
<!-- ==================== 登录页 ==================== -->
|
||||
<div class="admin-page">
|
||||
<div class="login-wrapper">
|
||||
<div class="login-card">
|
||||
<div class="login-brand">
|
||||
<div class="icon">
|
||||
<div class="iso-top"></div>
|
||||
<div class="iso-left"></div>
|
||||
<div class="iso-right"></div>
|
||||
<div class="iso-dot"></div>
|
||||
</div>
|
||||
<h2>昆明市大学生创业园</h2>
|
||||
<p>大屏幕轮播控制系统</p>
|
||||
</div>
|
||||
<h2>昆明市大学生创业园</h2>
|
||||
<p>大屏幕轮播控制系统</p>
|
||||
{% if error %}
|
||||
<div class="toast error" id="loginError" style="position:static;margin-bottom:12px;">账号或密码错误，请重试</div>
|
||||
{% endif %}
|
||||
<form method="post" action="/admin/login">
|
||||
<div class="field">
|
||||
<label>账号</label>
|
||||
<input name="username" placeholder="请输入账号" autocomplete="username" required>
|
||||
</div>
|
||||
<div class="field">
|
||||
<label>密码</label>
|
||||
<input type="password" name="password" placeholder="请输入密码" autocomplete="current-password" required>
|
||||
</div>
|
||||
<button type="submit" class="btn-login">登 录</button>
|
||||
</form>
|
||||
</div>
|
||||
{% if error %}
|
||||
<div class="toast error" id="loginError" style="position:static;margin-bottom:12px;">账号或密码错误，请重试</div>
|
||||
{% endif %}
|
||||
<form method="post" action="/admin/login">
|
||||
<div class="field">
|
||||
<label>账号</label>
|
||||
<input name="username" placeholder="请输入账号" autocomplete="username" required>
|
||||
</div>
|
||||
<div class="field">
|
||||
<label>密码</label>
|
||||
<input type="password" name="password" placeholder="请输入密码" autocomplete="current-password" required>
|
||||
</div>
|
||||
<button type="submit" class="btn-login">登 录</button>
|
||||
</form>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
{% else %}
|
||||
<!-- ==================== 管理台 ==================== -->
|
||||
<div class="admin-page">
|
||||
<div class="dashboard active">
|
||||
{% else %}
|
||||
<!-- ==================== 管理台 ==================== -->
|
||||
<div class="admin-page">
|
||||
<div class="dashboard active">
|
||||
|
||||
<!-- 顶栏 -->
|
||||
<div class="topbar">
|
||||
<div class="topbar-left">
|
||||
<div class="brand-dot"></div>
|
||||
<h1>昆明市大学生创业园</h1>
|
||||
</div>
|
||||
<div class="topbar-center">
|
||||
<span class="status-bar-text" id="statusText">等待大屏连接...</span>
|
||||
</div>
|
||||
<div class="topbar-right">
|
||||
<button class="btn-control" data-control="prev" title="上一个">
|
||||
<svg width="16" height="16" viewBox="0 0 16 16" fill="currentColor"><path d="M11 3L6 8L11 13" stroke="currentColor" stroke-width="2" fill="none" stroke-linecap="round" stroke-linejoin="round"/></svg>
|
||||
</button>
|
||||
<button class="btn-control play-btn" data-control="play" title="播放">
|
||||
<svg width="18" height="18" viewBox="0 0 18 18" fill="currentColor"><polygon points="5,3 15,9 5,15"/></svg>
|
||||
</button>
|
||||
<button class="btn-control play-btn" data-control="pause" title="暂停">
|
||||
<svg width="18" height="18" viewBox="0 0 18 18" fill="currentColor"><rect x="4" y="3" width="3" height="12" rx="1"/><rect x="11" y="3" width="3" height="12" rx="1"/></svg>
|
||||
</button>
|
||||
<button class="btn-control" data-control="next" title="下一个">
|
||||
<svg width="16" height="16" viewBox="0 0 16 16" fill="currentColor"><path d="M5 3L10 8L5 13" stroke="currentColor" stroke-width="2" fill="none" stroke-linecap="round" stroke-linejoin="round"/></svg>
|
||||
</button>
|
||||
<span class="topbar-divider"></span>
|
||||
{% if from_screen %}<a class="btn-return" href="/screen">返回展播</a>{% endif %}
|
||||
<a class="btn-return" href="/">数据大屏</a>
|
||||
<a class="btn-logout" href="/admin/logout">退出</a>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="main-grid">
|
||||
<!-- 连接状态 -->
|
||||
<div class="card">
|
||||
<div class="card-header">
|
||||
<div class="card-icon"><svg viewBox="0 0 24 24" width="20" height="20" fill="none" stroke="#1D5DCE" stroke-width="1.6"><path d="M5 12.5a10 10 0 0 1 14 0"/><path d="M8 15.5a6 6 0 0 1 8 0"/><path d="M12 19h.01"/></svg></div>
|
||||
<h3>连接状态</h3>
|
||||
<!-- 顶栏 -->
|
||||
<div class="topbar">
|
||||
<div class="topbar-left">
|
||||
<div class="brand-dot"></div>
|
||||
<h1>昆明市大学生创业园</h1>
|
||||
</div>
|
||||
<div class="conn-status">
|
||||
<div class="conn-row" id="connApi">
|
||||
<span class="conn-dot"></span>
|
||||
<span class="conn-label">后端服务</span>
|
||||
<span class="conn-value">检测中...</span>
|
||||
<div class="topbar-center">
|
||||
<span class="status-bar-text" id="statusText">等待大屏连接...</span>
|
||||
</div>
|
||||
<div class="topbar-right">
|
||||
<button class="btn-control" data-control="prev" title="上一个">
|
||||
<svg width="16" height="16" viewBox="0 0 16 16" fill="currentColor">
|
||||
<path d="M11 3L6 8L11 13" stroke="currentColor" stroke-width="2" fill="none" stroke-linecap="round"
|
||||
stroke-linejoin="round" />
|
||||
</svg>
|
||||
</button>
|
||||
<button class="btn-control play-btn" data-control="play" title="播放">
|
||||
<svg width="18" height="18" viewBox="0 0 18 18" fill="currentColor">
|
||||
<polygon points="5,3 15,9 5,15" />
|
||||
</svg>
|
||||
</button>
|
||||
<button class="btn-control play-btn" data-control="pause" title="暂停">
|
||||
<svg width="18" height="18" viewBox="0 0 18 18" fill="currentColor">
|
||||
<rect x="4" y="3" width="3" height="12" rx="1" />
|
||||
<rect x="11" y="3" width="3" height="12" rx="1" />
|
||||
</svg>
|
||||
</button>
|
||||
<button class="btn-control" data-control="next" title="下一个">
|
||||
<svg width="16" height="16" viewBox="0 0 16 16" fill="currentColor">
|
||||
<path d="M5 3L10 8L5 13" stroke="currentColor" stroke-width="2" fill="none" stroke-linecap="round"
|
||||
stroke-linejoin="round" />
|
||||
</svg>
|
||||
</button>
|
||||
<span class="topbar-divider"></span>
|
||||
{% if from_screen %}<a class="btn-return" href="/screen">返回展播</a>{% endif %}
|
||||
<a class="btn-return" href="/">数据大屏</a>
|
||||
<a class="btn-logout" href="/admin/logout">退出</a>
|
||||
</div>
|
||||
</div>
|
||||
<!-- 选择屏幕（独立模块） -->
|
||||
<div class="control-stack">
|
||||
<div class="remote-target">
|
||||
<span id="screenCount" class="remote-count">在线 0 台</span>
|
||||
<select id="screenTarget">
|
||||
<option value="">全部控制</option>
|
||||
</select>
|
||||
</div>
|
||||
|
||||
|
||||
<!-- 页面切换（独立模块） -->
|
||||
<div class="card control-card">
|
||||
<div class="page-nav-prev">
|
||||
<button class="remote-btn" data-mqtt='{"action":"navigate_rel","params":{"delta":-1}}' title="向左切页"
|
||||
aria-label="向左">‹</button>
|
||||
<button class="remote-btn" data-mqtt='{"action":"navigate_rel","params":{"delta":1}}' title="向右切页"
|
||||
aria-label="向右">›</button>
|
||||
</div>
|
||||
<div class="conn-row" id="connMqtt">
|
||||
<span class="conn-dot"></span>
|
||||
<span class="conn-label">MQTT Broker</span>
|
||||
<span class="conn-value">检测中...</span>
|
||||
</div>
|
||||
<div class="conn-row" id="connScreens">
|
||||
<span class="conn-dot"></span>
|
||||
<span class="conn-label">在线大屏</span>
|
||||
<span class="conn-value">检测中...</span>
|
||||
</div>
|
||||
<div class="conn-row" id="connLastCmd">
|
||||
<span class="conn-dot"></span>
|
||||
<span class="conn-label">最近指令</span>
|
||||
<span class="conn-value">—</span>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- 页面切换 -->
|
||||
<div class="card">
|
||||
<div class="card-header">
|
||||
<div class="card-icon"><svg viewBox="0 0 24 24" width="20" height="20" fill="none" stroke="#1D5DCE" stroke-width="1.6"><rect x="3" y="3" width="8" height="8" rx="1.5"/><rect x="13" y="3" width="8" height="8" rx="1.5"/><rect x="3" y="13" width="8" height="8" rx="1.5"/><rect x="13" y="13" width="8" height="8" rx="1.5"/></svg></div>
|
||||
<h3>页面切换（控制大屏展示）</h3>
|
||||
</div>
|
||||
<div class="page-switch-grid">
|
||||
<button class="page-switch-btn" data-nav="/">数据大屏</button>
|
||||
<button class="page-switch-btn" data-nav="/twin">数字孪生</button>
|
||||
<button class="page-switch-btn" data-nav="/ai">AI 助手</button>
|
||||
<button class="page-switch-btn" data-nav="/screen">媒体轮播</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- 大屏 MQTT 控制（全部前端命令） -->
|
||||
<div class="card">
|
||||
<div class="card-header">
|
||||
<div class="card-icon"><svg viewBox="0 0 24 24" width="20" height="20" fill="none" stroke="#1D5DCE" stroke-width="1.6"><path d="M2 6a2 2 0 0 1 2-2h16a2 2 0 0 1 2 2v12a2 2 0 0 1-2 2H4a2 2 0 0 1-2-2z"/><path d="M8 10h8M8 14h4"/></svg></div>
|
||||
<h3>大屏控制（MQTT 指令）</h3>
|
||||
</div>
|
||||
|
||||
<!-- 页面导航 -->
|
||||
<div class="mqtt-group">
|
||||
<div class="mqtt-group-title">页面导航</div>
|
||||
<div class="mqtt-row">
|
||||
<div class="mqtt-row page-nav-links">
|
||||
<button class="page-switch-btn" data-nav="/">数据大屏</button>
|
||||
<button class="page-switch-btn" data-nav="/twin">数字孪生</button>
|
||||
<button class="page-switch-btn" data-nav="/ai">AI 助手</button>
|
||||
<button class="page-switch-btn" data-nav="/screen">媒体轮播</button>
|
||||
<button class="page-switch-btn" data-mqtt='{"action":"navigate","params":{"page":"/voice"}}'>语音对话</button>
|
||||
<button class="page-switch-btn" data-mqtt='{"action":"navigate","params":{"page":"/wall"}}'>企业展示墙</button>
|
||||
<button class="page-switch-btn" data-mqtt='{"action":"navigate_rel","params":{"delta":-1}}'>← 向左</button>
|
||||
<button class="page-switch-btn" data-mqtt='{"action":"navigate_rel","params":{"delta":1}}'>向右 →</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="main-grid">
|
||||
<!-- 连接状态（弱化：沉到底部、紧凑展示） -->
|
||||
<div class="card conn-card" style="order:99">
|
||||
<div class="card-header">
|
||||
<div class="card-icon"><svg viewBox="0 0 24 24" width="20" height="20" fill="none" stroke="currentColor"
|
||||
stroke-width="1.5">
|
||||
<path d="M3 12h4l3 8 4-16 3 8h4" />
|
||||
</svg></div>
|
||||
<h3>连接状态</h3>
|
||||
</div>
|
||||
<div class="conn-status">
|
||||
<div class="conn-row" id="connApi">
|
||||
<span class="conn-dot"></span>
|
||||
<span class="conn-label">后端服务</span>
|
||||
<span class="conn-value">检测中...</span>
|
||||
</div>
|
||||
<div class="conn-row" id="connMqtt">
|
||||
<span class="conn-dot"></span>
|
||||
<span class="conn-label">MQTT Broker</span>
|
||||
<span class="conn-value">检测中...</span>
|
||||
</div>
|
||||
<div class="conn-row" id="connScreens">
|
||||
<span class="conn-dot"></span>
|
||||
<span class="conn-label">在线大屏</span>
|
||||
<span class="conn-value">检测中...</span>
|
||||
</div>
|
||||
<div class="conn-row" id="connLastCmd">
|
||||
<span class="conn-dot"></span>
|
||||
<span class="conn-label">最近指令</span>
|
||||
<span class="conn-value">—</span>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- 企业展示墙 -->
|
||||
<div class="mqtt-group">
|
||||
<div class="mqtt-group-title">企业展示墙（/wall）</div>
|
||||
<!-- 媒体轮播（独立模块） -->
|
||||
<div class="card" data-page="/screen">
|
||||
<div class="card-header">
|
||||
<div class="card-icon"><svg viewBox="0 0 24 24" width="20" height="20" fill="none" stroke="currentColor"
|
||||
stroke-width="1.5">
|
||||
<path d="M6 4v16l13-8z" />
|
||||
</svg></div>
|
||||
<h3>媒体轮播</h3>
|
||||
</div>
|
||||
<div class="mqtt-row">
|
||||
<button class="page-switch-btn" data-mqtt='{"action":"prev","params":{}}'>上一段</button>
|
||||
<button class="page-switch-btn" data-mqtt='{"action":"play","params":{}}'>播放</button>
|
||||
<button class="page-switch-btn" data-mqtt='{"action":"pause","params":{}}'>暂停</button>
|
||||
<button class="page-switch-btn" data-mqtt='{"action":"next","params":{}}'>下一段</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- 语音对话（独立模块） -->
|
||||
<div class="card" data-page="/voice">
|
||||
<div class="card-header">
|
||||
<div class="card-icon"><svg viewBox="0 0 24 24" width="20" height="20" fill="none" stroke="currentColor"
|
||||
stroke-width="1.5">
|
||||
<path d="M9 2a3 3 0 0 1 3 3v5a3 3 0 0 1-6 0V5a3 3 0 0 1 3-3z" />
|
||||
<path d="M5 10a7 7 0 0 0 14 0" />
|
||||
<path d="M12 19v3" />
|
||||
</svg></div>
|
||||
<h3>语音对话</h3>
|
||||
</div>
|
||||
<div class="mqtt-row">
|
||||
<button class="page-switch-btn" data-mqtt='{"action":"voice_start","params":{}}'>启动对话</button>
|
||||
<button class="page-switch-btn" data-mqtt='{"action":"voice_stop","params":{}}'>关闭对话</button>
|
||||
<button class="page-switch-btn" data-mqtt='{"action":"voice_refresh","params":{}}'>刷新页面</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- AI 助手（独立模块） -->
|
||||
<div class="card" data-page="/ai">
|
||||
<div class="card-header">
|
||||
<div class="card-icon"><svg viewBox="0 0 24 24" width="20" height="20" fill="none" stroke="currentColor"
|
||||
stroke-width="1.5">
|
||||
<path d="M12 5V3" />
|
||||
<path d="M18 4a2 2 0 0 1 2 2v11a2 2 0 0 1-2 2H6a2 2 0 0 1-2-2V6a2 2 0 0 1 2-2z" />
|
||||
<path d="M9 7h6" />
|
||||
<path d="M10 11h.01M14 11h.01" />
|
||||
<path d="M12 13v3" />
|
||||
<path d="M7 21h10" />
|
||||
</svg></div>
|
||||
<h3>AI 助手</h3>
|
||||
</div>
|
||||
<div class="mqtt-row">
|
||||
<input class="mqtt-input" id="mqttAiInput" type="text" placeholder="输入问题，回车发送">
|
||||
<button class="page-switch-btn" id="mqttAiSend">发送问题</button>
|
||||
</div>
|
||||
<div class="mqtt-row">
|
||||
<button class="page-switch-btn" data-mqtt='{"action":"ai_preset","params":{"index":0}}'>预设①入驻</button>
|
||||
<button class="page-switch-btn" data-mqtt='{"action":"ai_preset","params":{"index":1}}'>预设②政策</button>
|
||||
<button class="page-switch-btn" data-mqtt='{"action":"ai_preset","params":{"index":2}}'>预设③企业</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- 数字孪生（独立模块） -->
|
||||
<div class="card" data-page="/twin">
|
||||
<div class="card-header">
|
||||
<div class="card-icon"><svg viewBox="0 0 24 24" width="20" height="20" fill="none" stroke="currentColor"
|
||||
stroke-width="1.5">
|
||||
<path d="M4 8l8-5 8 5-8 5z" />
|
||||
<path d="M4 12l8 5 8-5" />
|
||||
<path d="M4 16l8 5 8-5" />
|
||||
</svg></div>
|
||||
<h3>数字孪生</h3>
|
||||
</div>
|
||||
<div class="mqtt-row">
|
||||
<button class="page-switch-btn" data-mqtt='{"action":"ai_company","params":{"action":"next"}}'>下一个企业</button>
|
||||
<button class="page-switch-btn" data-mqtt='{"action":"ai_company","params":{"action":"prev"}}'>上一个企业</button>
|
||||
</div>
|
||||
<div class="mqtt-row">
|
||||
<button class="page-switch-btn" data-mqtt='{"action":"ai_zone","params":{"zone":"孵化加速区"}}'>孵化加速区</button>
|
||||
<button class="page-switch-btn" data-mqtt='{"action":"ai_zone","params":{"zone":"创业苗圃区"}}'>创业苗圃区</button>
|
||||
<button class="page-switch-btn" data-mqtt='{"action":"ai_zone","params":{"zone":"国际创客区"}}'>国际创客区</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- 企业展示墙（独立模块） -->
|
||||
<div class="card" data-page="/wall">
|
||||
<div class="card-header">
|
||||
<div class="card-icon"><svg viewBox="0 0 24 24" width="20" height="20" fill="none" stroke="currentColor"
|
||||
stroke-width="1.5">
|
||||
<rect x="3" y="21" width="18" height="3" />
|
||||
<path d="M5 21V5a2 2 0 0 1 2-2h10a2 2 0 0 1 2 2v16" />
|
||||
<path d="M9 21v-4h6v4" />
|
||||
</svg></div>
|
||||
<h3>企业展示墙</h3>
|
||||
</div>
|
||||
<div class="mqtt-row">
|
||||
<button class="page-switch-btn" data-mqtt='{"action":"wall_pause","params":{}}'>暂停滚动</button>
|
||||
<button class="page-switch-btn" data-mqtt='{"action":"wall_resume","params":{}}'>继续滚动</button>
|
||||
@@ -150,176 +252,191 @@
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- 视觉识别 -->
|
||||
<div class="mqtt-group">
|
||||
<div class="mqtt-group-title">人物识别（全局）</div>
|
||||
<!-- 人物识别（独立模块） -->
|
||||
<!-- <div class="card">
|
||||
<div class="card-header">
|
||||
<div class="card-icon"><svg viewBox="0 0 24 24" width="20" height="20" fill="none" stroke="currentColor"
|
||||
stroke-width="1.5">
|
||||
<circle cx="9" cy="7" r="4" />
|
||||
<path d="M2 21v-2a4 4 0 0 1 4-4h6a4 4 0 0 1 4 4v2" />
|
||||
<path d="M16 3.13a4 4 0 0 1 0 7.75" />
|
||||
</svg></div>
|
||||
<h3>人物识别</h3>
|
||||
</div>
|
||||
<div class="mqtt-row">
|
||||
<button class="page-switch-btn" data-mqtt='{"action":"vision_set","params":{"enabled":true}}'>开启识别</button>
|
||||
<button class="page-switch-btn" data-mqtt='{"action":"vision_set","params":{"enabled":false}}'>关闭识别</button>
|
||||
</div>
|
||||
</div>
|
||||
</div> -->
|
||||
|
||||
<!-- AI 助手页 -->
|
||||
<div class="mqtt-group">
|
||||
<div class="mqtt-group-title">AI 助手页</div>
|
||||
<div class="mqtt-row">
|
||||
<input class="mqtt-input" id="mqttAiInput" type="text" placeholder="输入问题，回车发送到 AI 页">
|
||||
<button class="page-switch-btn" id="mqttAiSend">发送问题</button>
|
||||
<!-- 双屏（独立模块） -->
|
||||
<!-- <div class="card">
|
||||
<div class="card-header">
|
||||
<div class="card-icon"><svg viewBox="0 0 24 24" width="20" height="20" fill="none" stroke="currentColor"
|
||||
stroke-width="1.5">
|
||||
<rect x="2" y="3" width="20" height="14" rx="2" />
|
||||
<path d="M8 21h8M12 17v4" />
|
||||
</svg></div>
|
||||
<h3>双屏控制</h3>
|
||||
</div>
|
||||
<div class="mqtt-row">
|
||||
<button class="page-switch-btn" data-mqtt='{"action":"ai_preset","params":{"index":0}}'>预设①入驻</button>
|
||||
<button class="page-switch-btn" data-mqtt='{"action":"ai_preset","params":{"index":1}}'>预设②政策</button>
|
||||
<button class="page-switch-btn" data-mqtt='{"action":"ai_preset","params":{"index":2}}'>预设③企业</button>
|
||||
<button class="page-switch-btn" data-mqtt='{"action":"ai_company","params":{"action":"next"}}'>下一个企业</button>
|
||||
<button class="page-switch-btn" data-mqtt='{"action":"ai_company","params":{"action":"prev"}}'>上一个企业</button>
|
||||
<button class="page-switch-btn" data-mqtt='{"action":"dual_screen","params":{"on":true}}'>启动双屏</button>
|
||||
<button class="page-switch-btn" data-mqtt='{"action":"dual_screen","params":{"on":false}}'>关闭双屏</button>
|
||||
</div>
|
||||
<div class="mqtt-row">
|
||||
<button class="page-switch-btn" data-mqtt='{"action":"ai_zone","params":{"zone":"加速区"}}'>加速区</button>
|
||||
<button class="page-switch-btn" data-mqtt='{"action":"ai_zone","params":{"zone":"国际区"}}'>国际区</button>
|
||||
<button class="page-switch-btn" data-mqtt='{"action":"ai_zone","params":{"zone":"成长区"}}'>成长区</button>
|
||||
</div>
|
||||
</div>
|
||||
<div class="mqtt-hint">在电脑另一块屏幕全屏展示（需展播机连接显示器）</div>
|
||||
</div> -->
|
||||
|
||||
<!-- 语音对话页 -->
|
||||
<div class="mqtt-group">
|
||||
<div class="mqtt-group-title">语音对话页</div>
|
||||
<div class="mqtt-row">
|
||||
<button class="page-switch-btn" data-mqtt='{"action":"voice_start","params":{}}'>启动对话</button>
|
||||
<button class="page-switch-btn" data-mqtt='{"action":"voice_stop","params":{}}'>关闭对话</button>
|
||||
<button class="page-switch-btn" data-mqtt='{"action":"voice_refresh","params":{}}'>刷新页面</button>
|
||||
<!-- 全局通知（独立模块） -->
|
||||
<!-- <div class="card">
|
||||
<div class="card-header">
|
||||
<div class="card-icon"><svg viewBox="0 0 24 24" width="20" height="20" fill="none" stroke="currentColor"
|
||||
stroke-width="1.5">
|
||||
<path d="M10 5a2 2 0 1 1 4 0a7 7 0 0 1 4 6v3a4 4 0 0 0 2 3H4a4 4 0 0 0 2-3v-3a7 7 0 0 1 4-6z" />
|
||||
<path d="M9 17v1a3 3 0 0 0 6 0v-1" />
|
||||
</svg></div>
|
||||
<h3>全局通知</h3>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- 媒体控制 -->
|
||||
<div class="mqtt-group">
|
||||
<div class="mqtt-group-title">媒体轮播</div>
|
||||
<div class="mqtt-row">
|
||||
<button class="page-switch-btn" data-mqtt='{"action":"prev","params":{}}'>上一段</button>
|
||||
<button class="page-switch-btn" data-mqtt='{"action":"play","params":{}}'>播放</button>
|
||||
<button class="page-switch-btn" data-mqtt='{"action":"pause","params":{}}'>暂停</button>
|
||||
<button class="page-switch-btn" data-mqtt='{"action":"next","params":{}}'>下一段</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- 通知 -->
|
||||
<div class="mqtt-group">
|
||||
<div class="mqtt-group-title">全局通知</div>
|
||||
<div class="mqtt-row">
|
||||
<input class="mqtt-input" id="mqttAlertText" type="text" placeholder="通知内容，回车广播">
|
||||
<button class="page-switch-btn" id="mqttAlertSend">广播通知</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div> -->
|
||||
|
||||
<!-- 播放设置 -->
|
||||
<div class="card">
|
||||
<div class="card-header">
|
||||
<div class="card-icon"><svg viewBox="0 0 24 24" width="20" height="20" fill="none" stroke="#1D5DCE" stroke-width="1.6"><circle cx="12" cy="12" r="9"/><path d="M12 7v5l3 2"/></svg></div>
|
||||
<h3>播放设置</h3>
|
||||
</div>
|
||||
<div class="form-inline">
|
||||
<div class="form-group">
|
||||
<label>播放模式</label>
|
||||
<select id="playMode">
|
||||
<option value="sequential" {% if play_mode == 'sequential' %}selected{% endif %}>顺序播放</option>
|
||||
<option value="random" {% if play_mode == 'random' %}selected{% endif %}>随机播放</option>
|
||||
</select>
|
||||
<!-- 播放设置 -->
|
||||
<div class="card">
|
||||
<div class="card-header">
|
||||
<div class="card-icon"><svg viewBox="0 0 24 24" width="20" height="20" fill="none" stroke="currentColor"
|
||||
stroke-width="1.5">
|
||||
<path d="M10.3 21H5a2 2 0 0 1-2-2V5a2 2 0 0 1 2-2h14a2 2 0 0 1 2 2v10" />
|
||||
<circle cx="18" cy="18" r="3" />
|
||||
<path d="M19 19l2 2" />
|
||||
</svg></div>
|
||||
<h3>播放设置</h3>
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label>图片时长（秒）</label>
|
||||
<input type="number" id="imageDuration" value="{{ image_duration }}" min="1" max="300">
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label>音量（媒体）</label>
|
||||
<input type="range" id="volumeRange" min="0" max="100" value="{{ volume }}">
|
||||
<div class="volume-label" id="volumeLabel">{{ volume }}%</div>
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label>音量（音效）</label>
|
||||
<input type="range" id="sfxVolumeRange" min="0" max="100" value="{{ sfx_volume }}">
|
||||
<div class="volume-label" id="sfxVolumeLabel">{{ sfx_volume }}%</div>
|
||||
<div class="form-inline">
|
||||
<div class="form-group">
|
||||
<label>播放模式</label>
|
||||
<select id="playMode">
|
||||
<option value="sequential" {% if play_mode=='sequential' %}selected{% endif %}>顺序播放</option>
|
||||
<option value="random" {% if play_mode=='random' %}selected{% endif %}>随机播放</option>
|
||||
</select>
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label>图片时长（秒）</label>
|
||||
<input type="number" id="imageDuration" value="{{ image_duration }}" min="1" max="300">
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label>音量（媒体）</label>
|
||||
<input type="range" id="volumeRange" min="0" max="100" value="{{ volume }}">
|
||||
<div class="volume-label" id="volumeLabel">{{ volume }}%</div>
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label>音量（音效）</label>
|
||||
<input type="range" id="sfxVolumeRange" min="0" max="100" value="{{ sfx_volume }}">
|
||||
<div class="volume-label" id="sfxVolumeLabel">{{ sfx_volume }}%</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- 语音 AI 测试（admin → 前端主动问候） -->
|
||||
<div class="card">
|
||||
<div class="card-header">
|
||||
<div class="card-icon"><svg viewBox="0 0 24 24" width="20" height="20" fill="none" stroke="#1D5DCE" stroke-width="1.6"><path d="M12 2a3 3 0 0 0-3 3v7a3 3 0 0 0 6 0V5a3 3 0 0 0-3-3Z"/><path d="M19 10v2a7 7 0 0 1-14 0v-2M12 19v3"/></svg></div>
|
||||
<h3>语音 AI 测试</h3>
|
||||
</div>
|
||||
<div class="form-inline">
|
||||
<div class="form-group" style="flex:2;">
|
||||
<label>问候语（前端将主动播报，测试语音 AI）</label>
|
||||
<input type="text" id="voiceGreetText" value="您好呀，我是园区智能语音助手，有什么可以帮您？" style="width:100%;">
|
||||
<!-- 语音 AI 测试（admin → 前端主动问候） -->
|
||||
<div class="card">
|
||||
<div class="card-header">
|
||||
<div class="card-icon"><svg viewBox="0 0 24 24" width="20" height="20" fill="none" stroke="currentColor"
|
||||
stroke-width="1.5">
|
||||
<path d="M9 2a3 3 0 0 1 3 3v5a3 3 0 0 1-6 0V5a3 3 0 0 1 3-3z" />
|
||||
<path d="M5 10a7 7 0 0 0 14 0" />
|
||||
<path d="M12 19v3" />
|
||||
</svg></div>
|
||||
<h3>语音 AI 测试</h3>
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label> </label>
|
||||
<button class="btn-primary" id="btnVoiceGreet" style="width:100%;">播报问候</button>
|
||||
<div class="form-inline">
|
||||
<div class="form-group" style="flex:2;">
|
||||
<label>问候语（前端将主动播报，测试语音 AI）</label>
|
||||
<input type="text" id="voiceGreetText" value="您好呀，我是园区智能语音助手，有什么可以帮您？" style="width:100%;">
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label> </label>
|
||||
<button class="btn-primary" id="btnVoiceGreet" style="width:100%;">播报问候</button>
|
||||
</div>
|
||||
</div>
|
||||
<div class="hint" id="voiceGreetHint">点击后展播端将自动进入语音对话页并主动播报上述问候（LLM 生成 + TTS），用于验证语音链路。</div>
|
||||
</div>
|
||||
<div class="hint" id="voiceGreetHint">点击后展播端将自动进入语音对话页并主动播报上述问候（LLM 生成 + TTS），用于验证语音链路。</div>
|
||||
</div>
|
||||
|
||||
<!-- 显示控制 -->
|
||||
<div class="card">
|
||||
<div class="card-header">
|
||||
<div class="card-icon"><svg viewBox="0 0 24 24" width="20" height="20" fill="none" stroke="#1D5DCE" stroke-width="1.6"><rect x="3" y="4" width="18" height="13" rx="2"/><path d="M8 21h8M12 17v4"/></svg></div>
|
||||
<h3>显示控制</h3>
|
||||
</div>
|
||||
<div class="form-inline">
|
||||
<div class="form-group">
|
||||
<label>显示模式</label>
|
||||
<button class="btn-outline" id="btnFullscreen" style="width:100%;">切换大屏模式</button>
|
||||
<!-- 显示控制 -->
|
||||
<div class="card">
|
||||
<div class="card-header">
|
||||
<div class="card-icon"><svg viewBox="0 0 24 24" width="20" height="20" fill="none" stroke="currentColor"
|
||||
stroke-width="1.5">
|
||||
<rect x="2" y="3" width="20" height="14" rx="2" />
|
||||
<path d="M8 21h8M12 17v4" />
|
||||
</svg></div>
|
||||
<h3>显示控制</h3>
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label>窗口操作</label>
|
||||
<button class="btn-outline" data-control="minimize" style="width:100%;">最小化窗口</button>
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label>开机自启动</label>
|
||||
<button class="btn-outline" id="btnAutostart" style="width:100%;">已关闭</button>
|
||||
<div class="form-inline">
|
||||
<div class="form-group">
|
||||
<label>显示模式</label>
|
||||
<button class="btn-outline" id="btnFullscreen" style="width:100%;">切换大屏模式</button>
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label>窗口操作</label>
|
||||
<button class="btn-outline" data-control="minimize" style="width:100%;">最小化窗口</button>
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label>开机自启动</label>
|
||||
<button class="btn-outline" id="btnAutostart" style="width:100%;">已关闭</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- 添加媒体 -->
|
||||
<div class="card">
|
||||
<div class="card-header">
|
||||
<div class="card-icon"><svg viewBox="0 0 24 24" width="20" height="20" fill="none" stroke="#1D5DCE" stroke-width="1.6"><path d="M12 16V4m0 0L7 9m5-5l5 5"/><path d="M4 16v3a2 2 0 0 0 2 2h12a2 2 0 0 0 2-2v-3"/></svg></div>
|
||||
<h3>添加媒体</h3>
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label>本地上传</label>
|
||||
<div class="form-row">
|
||||
<input type="file" id="fileInput" accept=".mp4,.mkv,.avi,.jpg,.jpeg,.png" multiple>
|
||||
<button class="btn-accent" id="btnUpload">上传</button>
|
||||
<!-- 添加媒体 -->
|
||||
<div class="card">
|
||||
<div class="card-header">
|
||||
<div class="card-icon"><svg viewBox="0 0 24 24" width="20" height="20" fill="none" stroke="currentColor"
|
||||
stroke-width="1.5">
|
||||
<path d="M12 16V4m0 0L7 9m5-5l5 5" />
|
||||
<path d="M4 17v2a2 2 0 0 0 2 2h12a2 2 0 0 0 2-2v-2" />
|
||||
</svg></div>
|
||||
<h3>添加媒体</h3>
|
||||
</div>
|
||||
<div class="upload-progress" id="uploadProgress" style="display:none;">
|
||||
<div class="progress-bar"><div class="progress-fill" id="progressFill" style="width:0%"></div></div>
|
||||
<span class="progress-text" id="progressText"></span>
|
||||
<div class="form-group">
|
||||
<label>本地上传</label>
|
||||
<div class="form-row">
|
||||
<input type="file" id="fileInput" accept=".mp4,.mkv,.avi,.jpg,.jpeg,.png" multiple>
|
||||
<button class="btn-accent" id="btnUpload">上传</button>
|
||||
</div>
|
||||
<div class="upload-progress" id="uploadProgress" style="display:none;">
|
||||
<div class="progress-bar">
|
||||
<div class="progress-fill" id="progressFill" style="width:0%"></div>
|
||||
</div>
|
||||
<span class="progress-text" id="progressText"></span>
|
||||
</div>
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label>远程 URL / 播放链接</label>
|
||||
<div class="form-row">
|
||||
<input type="text" id="urlInput" placeholder="https://example.com/media.mp4">
|
||||
<button class="btn-accent" id="btnAddUrl">添加到媒体库</button>
|
||||
<button class="btn-accent" id="btnAddUrlPlay">加入播放列表</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="form-group">
|
||||
<label>远程 URL / 播放链接</label>
|
||||
<div class="form-row">
|
||||
<input type="text" id="urlInput" placeholder="https://example.com/media.mp4">
|
||||
<button class="btn-accent" id="btnAddUrl">添加到媒体库</button>
|
||||
<button class="btn-accent" id="btnAddUrlPlay">加入播放列表</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- 媒体库 -->
|
||||
<div class="card full">
|
||||
<div class="card-header">
|
||||
<div class="card-icon"><svg viewBox="0 0 24 24" width="20" height="20" fill="none" stroke="#1D5DCE" stroke-width="1.6"><path d="M4 20V10l6-5 6 5v10"/><path d="M10 20v-5h4v5"/></svg></div>
|
||||
<h3>媒体库（{{ files|length }}）</h3>
|
||||
</div>
|
||||
<div class="media-grid">
|
||||
{% if files %}
|
||||
<!-- 媒体库 -->
|
||||
<div class="card full">
|
||||
<div class="card-header">
|
||||
<div class="card-icon"><svg viewBox="0 0 24 24" width="20" height="20" fill="none" stroke="currentColor"
|
||||
stroke-width="1.5">
|
||||
<rect x="3" y="3" width="18" height="18" rx="2" />
|
||||
<circle cx="8.5" cy="8.5" r="1.5" />
|
||||
<path d="M21 15l-5-5L5 21" />
|
||||
</svg></div>
|
||||
<h3>媒体库（{{ files|length }}）</h3>
|
||||
</div>
|
||||
<div class="media-grid">
|
||||
{% if files %}
|
||||
{% for item in files %}
|
||||
<div class="media-card" data-path="{{ item.relative_path }}" data-type="{{ item.type }}" data-source="{{ item.source }}" data-name="{{ item.name }}">
|
||||
<div class="media-card" data-path="{{ item.relative_path }}" data-type="{{ item.type }}"
|
||||
data-source="{{ item.source }}" data-name="{{ item.name }}">
|
||||
{% if item.type == 'image' %}
|
||||
<img src="{{ item.url }}" loading="lazy" class="thumb" alt="{{ item.name }}" data-preview>
|
||||
{% else %}
|
||||
@@ -331,8 +448,10 @@
|
||||
<div class="body">
|
||||
<span class="name" title="{{ item.name }}">{{ item.name }}</span>
|
||||
<div class="meta-row">
|
||||
<span class="badge {{ 'badge-url' if item.source == 'url' else 'badge-local' }}">{{ '远程' if item.source == 'url' else '本地' }}</span>
|
||||
<span class="badge {{ 'badge-video' if item.type == 'video' else 'badge-image' }}">{{ '视频' if item.type == 'video' else '图片' }}</span>
|
||||
<span class="badge {{ 'badge-url' if item.source == 'url' else 'badge-local' }}">{{ '远程' if
|
||||
item.source == 'url' else '本地' }}</span>
|
||||
<span class="badge {{ 'badge-video' if item.type == 'video' else 'badge-image' }}">{{ '视频' if
|
||||
item.type == 'video' else '图片' }}</span>
|
||||
</div>
|
||||
<div class="media-url" data-url="{{ item.url }}" title="FastAPI 输出的资源 URL">
|
||||
<span>{{ item.url }}</span>
|
||||
@@ -346,25 +465,34 @@
|
||||
</div>
|
||||
</div>
|
||||
{% endfor %}
|
||||
{% else %}
|
||||
{% else %}
|
||||
<div class="empty-state">
|
||||
<div class="empty-icon"><svg viewBox="0 0 24 24" width="28" height="28" fill="none" stroke="#9ca3af" stroke-width="1.4"><rect x="3" y="5" width="18" height="14" rx="2"/><circle cx="9" cy="10" r="2"/><path d="M3 17l5-4 4 3 4-4 5 5"/></svg></div>
|
||||
<div class="empty-icon"><svg viewBox="0 0 24 24" width="28" height="28" fill="none" stroke="#9ca3af"
|
||||
stroke-width="1.4">
|
||||
<rect x="3" y="5" width="18" height="14" rx="2" />
|
||||
<circle cx="9" cy="10" r="2" />
|
||||
<path d="M3 17l5-4 4 3 4-4 5 5" />
|
||||
</svg></div>
|
||||
暂无媒体文件<br>请上传或通过 URL 添加
|
||||
</div>
|
||||
{% endif %}
|
||||
{% endif %}
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- 播放列表 -->
|
||||
<div class="card full">
|
||||
<div class="card-header">
|
||||
<div class="card-icon"><svg viewBox="0 0 24 24" width="20" height="20" fill="none" stroke="#1D5DCE" stroke-width="1.6"><path d="M4 6h16M4 12h16M4 18h10"/></svg></div>
|
||||
<h3>播放列表（{{ playlist|length }}）</h3>
|
||||
</div>
|
||||
<div class="media-grid">
|
||||
{% if playlist %}
|
||||
<!-- 播放列表 -->
|
||||
<div class="card full">
|
||||
<div class="card-header">
|
||||
<div class="card-icon"><svg viewBox="0 0 24 24" width="20" height="20" fill="none" stroke="currentColor"
|
||||
stroke-width="1.5">
|
||||
<path d="M4 6h16M4 12h16M4 18h10" />
|
||||
</svg></div>
|
||||
<h3>播放列表（{{ playlist|length }}）</h3>
|
||||
</div>
|
||||
<div class="media-grid">
|
||||
{% if playlist %}
|
||||
{% for item in playlist %}
|
||||
<div class="media-card" data-path="{{ item.relative_path }}" data-type="{{ item.type }}" data-source="{{ item.source }}" data-name="{{ item.name }}">
|
||||
<div class="media-card" data-path="{{ item.relative_path }}" data-type="{{ item.type }}"
|
||||
data-source="{{ item.source }}" data-name="{{ item.name }}">
|
||||
{% if item.type == 'image' %}
|
||||
<img src="{{ item.url }}" loading="lazy" class="thumb" alt="{{ item.name }}" data-preview>
|
||||
{% else %}
|
||||
@@ -376,8 +504,10 @@
|
||||
<div class="body">
|
||||
<span class="name" title="{{ item.name }}">{{ item.name }}</span>
|
||||
<div class="meta-row">
|
||||
<span class="badge {{ 'badge-url' if item.source == 'url' else 'badge-local' }}">{{ '远程' if item.source == 'url' else '本地' }}</span>
|
||||
<span class="badge {{ 'badge-video' if item.type == 'video' else 'badge-image' }}">{{ '视频' if item.type == 'video' else '图片' }}</span>
|
||||
<span class="badge {{ 'badge-url' if item.source == 'url' else 'badge-local' }}">{{ '远程' if
|
||||
item.source == 'url' else '本地' }}</span>
|
||||
<span class="badge {{ 'badge-video' if item.type == 'video' else 'badge-image' }}">{{ '视频' if
|
||||
item.type == 'video' else '图片' }}</span>
|
||||
</div>
|
||||
<div class="media-url" data-url="{{ item.url }}" title="FastAPI 输出的资源 URL">
|
||||
<span>{{ item.url }}</span>
|
||||
@@ -390,47 +520,48 @@
|
||||
</div>
|
||||
</div>
|
||||
{% endfor %}
|
||||
{% else %}
|
||||
{% else %}
|
||||
<div class="empty-state">
|
||||
<div class="empty-icon">▶</div>
|
||||
播放列表为空<br>从上方媒体库添加内容
|
||||
</div>
|
||||
{% endif %}
|
||||
{% endif %}
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- 预览弹窗 -->
|
||||
<div class="modal-overlay" id="previewModal" style="display:none;">
|
||||
<div class="modal-panel">
|
||||
<div class="modal-header">
|
||||
<span class="modal-title" id="previewTitle"></span>
|
||||
<button class="modal-close" id="previewClose">×</button>
|
||||
</div>
|
||||
<div class="modal-body" id="previewBody"></div>
|
||||
<div class="modal-footer">
|
||||
<span class="modal-info" id="previewMeta"></span>
|
||||
<div class="modal-actions">
|
||||
<button class="btn-accent btn-sm" id="previewPlay">立即播放</button>
|
||||
<button class="btn-accent btn-sm" id="previewAdd">加入播放</button>
|
||||
<button class="btn-danger btn-sm" id="previewDelete">删除</button>
|
||||
<!-- 预览弹窗 -->
|
||||
<div class="modal-overlay" id="previewModal" style="display:none;">
|
||||
<div class="modal-panel">
|
||||
<div class="modal-header">
|
||||
<span class="modal-title" id="previewTitle"></span>
|
||||
<button class="modal-close" id="previewClose">×</button>
|
||||
</div>
|
||||
<div class="modal-body" id="previewBody"></div>
|
||||
<div class="modal-footer">
|
||||
<span class="modal-info" id="previewMeta"></span>
|
||||
<div class="modal-actions">
|
||||
<button class="btn-accent btn-sm" id="previewPlay">立即播放</button>
|
||||
<button class="btn-accent btn-sm" id="previewAdd">加入播放</button>
|
||||
<button class="btn-danger btn-sm" id="previewDelete">删除</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Toast -->
|
||||
<div class="toast-container" id="toastContainer"></div>
|
||||
</div>
|
||||
|
||||
<!-- Toast -->
|
||||
<div class="toast-container" id="toastContainer"></div>
|
||||
</div>
|
||||
|
||||
<script>
|
||||
window.__DPM_INIT = {
|
||||
fullscreen: {{ 'true' if fullscreen else 'false' }},
|
||||
autostart: {{ 'true' if autostart else 'false' }}
|
||||
};
|
||||
</script>
|
||||
<script src="/static/admin.js"></script>
|
||||
{% endif %}
|
||||
<script>
|
||||
window.__DPM_INIT = {
|
||||
fullscreen: {{ 'true' if fullscreen else 'false' }},
|
||||
autostart: {{ 'true' if autostart else 'false' }},
|
||||
};
|
||||
</script>
|
||||
<script src="/static/admin.js"></script>
|
||||
{% endif %}
|
||||
</body>
|
||||
</html>
|
||||
|
||||
</html>
|
||||
@@ -1,234 +0,0 @@
|
||||
# ESP32 红外遥控控制大屏 —— 开发文档
|
||||
|
||||
> 通过红外遥控器（NEC 协议）直接控制园区大屏页面切换与操作。
|
||||
> 链路：红外遥控 → ESP32（红外接收 + 解码）→ MQTT → 前端调度器 → 页面动作
|
||||
|
||||
## 1. 系统架构
|
||||
|
||||
```
|
||||
┌──────────┐ 38kHz 红外 ┌─────────┐ WiFi/MQTT ┌──────────────┐ opc/display/command ┌────────┐
|
||||
│ 红外遥控器 │ ───────────► │ ESP32 │ ───────────► │ EMQX Broker │ ─────────────────────► │ 大屏前端 │
|
||||
│ (NEC协议) │ │ 接收+解码 │ │ 192.168.1.3 │ │ 调度器 │
|
||||
└──────────┘ └─────────┘ └──────────────┘ └────────┘
|
||||
```
|
||||
|
||||
- **红外遥控器**：任意 NEC 协议遥控器（电视/机顶盒/学习型均可），按键映射可自定义
|
||||
- **ESP32**：接收红外信号 → 解码键码 → 映射成指令 → MQTT 发布
|
||||
- **MQTT**：复用现有 `opc/display/command` 频道（见 `docs/mqtt-commands.md`）
|
||||
- **大屏前端**：`useMqttControl` 调度器接收指令执行（切页/媒体/语音/视觉）
|
||||
|
||||
## 2. 硬件清单与接线
|
||||
|
||||
| 硬件 | 型号/规格 | 数量 |
|
||||
|---|---|---|
|
||||
| 开发板 | ESP32（DevKitC / NodeMCU-32S） | 1 |
|
||||
| 红外接收头 | VS1838B / TL1838（38kHz，3 脚） | 1 |
|
||||
| 电阻 | 100Ω（限流，可选） | 1 |
|
||||
| 红外遥控器 | NEC 协议（电视/机顶盒遥控器） | 1 |
|
||||
|
||||
**VS1838B 接线**（面对接收头平面，引脚从左到右）：
|
||||
|
||||
| 引脚 | 接 ESP32 |
|
||||
|---|---|
|
||||
| 1 (OUT/信号) | GPIO 15（可改，见代码 `IR_RX_PIN`） |
|
||||
| 2 (GND) | GND |
|
||||
| 3 (VCC) | 3.3V（或 5V，接收头宽压 2.7~5.5V） |
|
||||
|
||||
## 3. 红外协议（NEC）
|
||||
|
||||
- NEC 协议：38kHz 载波，地址码 + 命令码 + 反码
|
||||
- 本方案**只取命令码（command byte）**即可区分按键，地址码忽略
|
||||
- 任意 NEC 遥控器的按键都有固定键码；不同遥控器键码不同 → **首次需"学习模式"采集键码**（见 §6）
|
||||
|
||||
## 4. Arduino 固件
|
||||
|
||||
依赖库（Arduino IDE 库管理器安装）：
|
||||
- `IRremote`（by Armin Joachimsmeyer，v4.x）
|
||||
- `PubSubClient`（by Nick O'Leary）
|
||||
|
||||
```cpp
|
||||
// esp32_ir_mqtt.ino —— 红外遥控 → MQTT 控制大屏
|
||||
#include <WiFi.h>
|
||||
#include <PubSubClient.h>
|
||||
#include <IRremote.h>
|
||||
|
||||
// ── WiFi / MQTT 配置 ─────────────────────────────────────
|
||||
const char* WIFI_SSID = "你的WiFi";
|
||||
const char* WIFI_PASS = "你的WiFi密码";
|
||||
const char* MQTT_HOST = "192.168.1.3"; // EMQX Broker
|
||||
const int MQTT_PORT = 1883;
|
||||
const char* MQTT_USER = "mqtt用户名"; // backend/.env MQTT_USERNAME
|
||||
const char* MQTT_PASS = "mqtt密码"; // backend/.env MQTT_PASSWORD
|
||||
const char* MQTT_TOPIC = "opc/display/command";
|
||||
|
||||
// ── 红外引脚 ─────────────────────────────────────────────
|
||||
const uint8_t IR_RX_PIN = 15;
|
||||
|
||||
// ── 按键映射表：遥控器键码 → 大屏指令 ─────────────────────
|
||||
// 键码需先用「学习模式」采集（串口打印 decodedIRData.command），再填入
|
||||
struct CmdMap { uint32_t key; const char* payload; };
|
||||
const CmdMap MAP[] = {
|
||||
// 数字键 1~5：直接切页
|
||||
{ 0x16, R"({"action":"navigate","params":{"page":"/"}})" }, // 1 → 数据大屏
|
||||
{ 0x19, R"({"action":"navigate","params":{"page":"/twin"}})" }, // 2 → 数字孪生
|
||||
{ 0x0D, R"({"action":"navigate","params":{"page":"/ai"}})" }, // 3 → AI 助手
|
||||
{ 0x0C, R"({"action":"navigate","params":{"page":"/voice"}})" }, // 4 → 语音对话
|
||||
{ 0x18, R"({"action":"navigate","params":{"page":"/screen"}})" }, // 5 → 媒体轮播
|
||||
// CH+/CH-（或左右方向）：向左/右切页
|
||||
{ 0x00, R"({"action":"navigate_rel","params":{"delta":-1}})" }, // CH- → 左切
|
||||
{ 0x01, R"({"action":"navigate_rel","params":{"delta":1}})" }, // CH+ → 右切
|
||||
// 音量+/−：媒体播放/暂停（或 next/prev）
|
||||
{ 0x02, R"({"action":"play","params":{}})" }, // VOL+ → 播放
|
||||
{ 0x03, R"({"action":"pause","params":{}})" }, // VOL- → 暂停
|
||||
// 电源键：语音对话开关
|
||||
{ 0x0B, R"({"action":"voice_start","params":{}})" }, // 电源 → 语音启动
|
||||
// OK/静音键：视觉识别开关（toggle 示例：直接开/关，可换 toggle 逻辑）
|
||||
{ 0x0A, R"({"action":"vision_set","params":{"enabled":true}})" }, // 静音 → 开识别
|
||||
};
|
||||
const int MAP_LEN = sizeof(MAP) / sizeof(MAP[0]);
|
||||
|
||||
// ── 全局 ─────────────────────────────────────────────────
|
||||
WiFiClient espClient;
|
||||
PubSubClient mqtt(espClient);
|
||||
unsigned long lastReconnect = 0;
|
||||
|
||||
void setup() {
|
||||
Serial.begin(115200);
|
||||
Serial.println("ESP32 IR → MQTT 启动");
|
||||
|
||||
IrReceiver.begin(IR_RX_PIN, ENABLE_LED_FEEDBACK);
|
||||
|
||||
WiFi.mode(WIFI_STA);
|
||||
WiFi.begin(WIFI_SSID, WIFI_PASS);
|
||||
while (WiFi.status() != WL_CONNECTED) {
|
||||
delay(500);
|
||||
Serial.print(".");
|
||||
}
|
||||
Serial.println("\nWiFi 已连接: " + String(WiFi.localIP()));
|
||||
|
||||
mqtt.setServer(MQTT_HOST, MQTT_PORT);
|
||||
mqtt.setKeepAlive(30);
|
||||
}
|
||||
|
||||
void loop() {
|
||||
if (!mqtt.connected()) reconnect();
|
||||
mqtt.loop();
|
||||
|
||||
// ── 红外解码 ──
|
||||
if (IrReceiver.decode()) {
|
||||
uint32_t cmd = IrReceiver.decodedIRData.command;
|
||||
Serial.printf("[IR] 地址=0x%02X 键码=0x%02X\n",
|
||||
IrReceiver.decodedIRData.address, cmd);
|
||||
IrReceiver.resume(); // 立即恢复接收
|
||||
|
||||
sendCommand(cmd);
|
||||
}
|
||||
}
|
||||
|
||||
// ── 键码 → MQTT ──
|
||||
void sendCommand(uint32_t key) {
|
||||
for (int i = 0; i < MAP_LEN; i++) {
|
||||
if (MAP[i].key == key) {
|
||||
Serial.println("[MQTT] 发送: " + String(MAP[i].payload));
|
||||
mqtt.publish(MQTT_TOPIC, MAP[i].payload);
|
||||
return;
|
||||
}
|
||||
}
|
||||
Serial.println("[IR] 未映射键码（可在学习模式后加入映射表）");
|
||||
}
|
||||
|
||||
// ── MQTT 重连 ──
|
||||
void reconnect() {
|
||||
if (millis() - lastReconnect < 3000) return;
|
||||
lastReconnect = millis();
|
||||
String id = "esp32-ir-" + String((uint32_t)ESP.getEfuseMac(), HEX);
|
||||
if (mqtt.connect(id.c_str(), MQTT_USER, MQTT_PASS)) {
|
||||
Serial.println("MQTT 已连接");
|
||||
} else {
|
||||
Serial.printf("MQTT 连接失败 rc=%d\n", mqtt.state());
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## 5. 键位映射建议
|
||||
|
||||
| 遥控器按键 | 键码（示例） | 指令 | 效果 |
|
||||
|---|---|---|---|
|
||||
| 1 ~ 5 | 0x16/0x19/0x0D/0x0C/0x18 | `navigate` | 直达各页面 |
|
||||
| CH− / CH＋ | 0x00 / 0x01 | `navigate_rel` | 向左/右循环切页 |
|
||||
| VOL＋ / VOL− | 0x02 / 0x03 | `play` / `pause` | 媒体播放/暂停 |
|
||||
| 电源 | 0x0B | `voice_start` | 启动语音对话 |
|
||||
| 静音 | 0x0A | `vision_set` | 开/关人物识别 |
|
||||
|
||||
> 键码因遥控器而异——上面的 0x16/0x19… 是常见 NEC 遥控器的值，**首次使用务必先跑学习模式实测**。
|
||||
|
||||
其他可用指令（自定义映射）：`ai_input`（AI 页提问）、`ai_company`（切企业）、`ai_zone`（切分区）、`voice_stop`、`next`/`prev`（媒体）、`alert`（通知）——参数见 `docs/mqtt-commands.md`。
|
||||
|
||||
## 5.1 企业展示墙控制（/wall，39 家入驻企业滚动墙）
|
||||
|
||||
**MQTT 控制消息**（ESP32 解码红外键码后发布到 `opc/display/command`，JSON 载荷）：
|
||||
|
||||
| 遥控器按键 | 建议指令 action | params | 效果 | 示例消息 |
|
||||
|---|---|---|---|---|
|
||||
| 1 / 2 / 3 | `wall_speed` | `speed: "slow" \| "normal" \| "fast"` | 慢速 150s / 标准 80s / 快速 40s 一圈 | `{"action":"wall_speed","params":{"speed":"slow"}}` |
|
||||
| CH− | `wall_pause` | `{}` | 暂停滚动 | `{"action":"wall_pause","params":{}}` |
|
||||
| CH＋ | `wall_resume` | `{}` | 继续滚动 | `{"action":"wall_resume","params":{}}` |
|
||||
| 电源 | `navigate` | `page: "/wall"` | 切换到企业展示墙 | `{"action":"navigate","params":{"page":"/wall"}}` |
|
||||
|
||||
**ESP32 固件映射示例**（在 §4 固件的 `MAP[]` 键码映射表中追加）：
|
||||
|
||||
```cpp
|
||||
// 键码 → MQTT 指令（payload 为 JSON 字符串，经 publishCommand 发布）
|
||||
{ 0x02, R"({"action":"wall_pause","params":{}})" }, // VOL− 暂停墙滚动
|
||||
{ 0x03, R"({"action":"wall_resume","params":{}})" }, // VOL＋ 继续墙滚动
|
||||
{ 0x00, R"({"action":"wall_speed","params":{"speed":"fast"}})" }, // CH− 快速
|
||||
{ 0x01, R"({"action":"wall_speed","params":{"speed":"slow"}})" }, // CH＋ 慢速
|
||||
{ 0x0B, R"({"action":"navigate","params":{"page":"/wall"}})" }, // 电源 切到展示墙
|
||||
```
|
||||
|
||||
> 前端 `useMqttControl` 收到 `wall_*` 指令后派发 `dpm:wall-control` 事件，展示墙页据此暂停/继续/变速；未在展示墙页时 `wall_*` 指令不产生副作用（安全幂等）。
|
||||
|
||||
## 6. 学习模式（首次必须）
|
||||
|
||||
键码采集固件（只需串口打印，不用连 MQTT）：
|
||||
|
||||
```cpp
|
||||
#include <IRremote.h>
|
||||
const uint8_t IR_RX_PIN = 15;
|
||||
void setup() {
|
||||
Serial.begin(115200);
|
||||
IrReceiver.begin(IR_RX_PIN, ENABLE_LED_FEEDBACK);
|
||||
Serial.println("按遥控器按键，串口会打印键码（学习模式）");
|
||||
}
|
||||
void loop() {
|
||||
if (IrReceiver.decode()) {
|
||||
Serial.printf("address=0x%02X command=0x%02X\n",
|
||||
IrReceiver.decodedIRData.address,
|
||||
IrReceiver.decodedIRData.command);
|
||||
IrReceiver.resume();
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
步骤：
|
||||
1. 烧录学习固件，打开串口监视器（115200）
|
||||
2. 逐个按遥控器按键，记录每个键的 `command` 十六进制值
|
||||
3. 把键码填入正式固件的 `MAP[]` 映射表
|
||||
4. 烧录正式固件
|
||||
|
||||
## 7. 调试与故障排查
|
||||
|
||||
| 现象 | 排查 |
|
||||
|---|---|
|
||||
| 串口无任何 IR 输出 | 接线检查（OUT→GPIO15、GND、VCC）；接收头正面对遥控器；换遥控器电池 |
|
||||
| 有 IR 输出但 MQTT 不发送 | 打印显示"未映射键码"→ 键码表没配对该遥控器；WiFi/MQTT 未连接 |
|
||||
| MQTT 连接失败 rc=5 | 用户名/密码错误（对照 backend/.env） |
|
||||
| 前端无反应但 MQTT 收到 | `mosquitto_sub -t opc/display/command` 验证消息到达 Broker；前端需在运行且 MQTT 已连（页面右上角在线状态） |
|
||||
| 连按无效 | NEC 重复码（0xFFFFFFFF）被忽略属正常；长按只发一次指令 |
|
||||
|
||||
## 8. 进阶扩展（可选）
|
||||
|
||||
- **学习型配置**：ESP32 加按钮 + OLED，现场学习键码并存入 NVS，免改代码
|
||||
- **多键组合**：短按/长按区分（记录按下时长）→ 同一键多指令
|
||||
- **OTA 升级**：ArduinoOTA 远程更新固件
|
||||
- **多屏控制**：指令包带 `client_id` 过滤，指定某台大屏执行
|
||||
@@ -1,175 +0,0 @@
|
||||
# MQTT 标准操作手册（OPC 智能园区大屏系统）
|
||||
|
||||
> 通过 MQTT 对前端大屏进行集中控制：切页、页面操作、媒体控制、视觉开关、状态上报。
|
||||
|
||||
## 1. Broker 连接
|
||||
|
||||
| 项 | 值 | 说明 |
|
||||
|---|---|---|
|
||||
| 地址 | `192.168.1.3` | 局域网 EMQX Broker |
|
||||
| TCP 端口 | `1883` | MQTT 标准端口（后端/命令行用） |
|
||||
| WebSocket 端口 | `8083` | 前端浏览器用（`ws://192.168.1.3:8083/mqtt`） |
|
||||
| 用户名/密码 | `backend/.env` 的 `MQTT_USERNAME` / `MQTT_PASSWORD` | 连接鉴权 |
|
||||
|
||||
## 2. 频道（Topic）
|
||||
|
||||
| Topic | 方向 | 载荷 | 用途 |
|
||||
|---|---|---|---|
|
||||
| `opc/display/command` | 服务端 → 前端 | 指令信封 | **控制指令**（切页/页面操作/媒体/视觉） |
|
||||
| `opc/dashboard/tick` | 服务端 → 前端 | 园区数据快照 | 数据大屏实时刷新（2.2s） |
|
||||
| `opc/display/heartbeat` | 前端 → 服务端 | `{client_id, page, ts}` | 大屏在线心跳（10s） |
|
||||
| `opc/frontend/state` | 前端 → 服务端 | `{client_id, page, ts}` | **页面状态即时上报**（路由变化即发） |
|
||||
| `opc/display/ack` | 前端 → 服务端 | 指令回执 | 指令确认（可选） |
|
||||
|
||||
## 3. 统一指令信封
|
||||
|
||||
所有控制指令发到 `opc/display/command`，JSON 格式：
|
||||
|
||||
```json
|
||||
{
|
||||
"cmd_id": "uuid", // 指令 ID（可选）
|
||||
"ts": 1755490000000, // 时间戳 ms（可选）
|
||||
"action": "navigate", // 命令名（必填）
|
||||
"params": { "page": "/ai" } // 参数（按命令）
|
||||
}
|
||||
```
|
||||
|
||||
后端 `hub.publish_command(action, params)` 自动生成信封；命令行手动发送时只需 `action` + `params`。
|
||||
|
||||
## 4. 命令全集
|
||||
|
||||
### 4.1 页面切换
|
||||
|
||||
| action | params | 效果 |
|
||||
|---|---|---|
|
||||
| `navigate` | `page: "/" \| "/twin" \| "/ai" \| "/voice" \| "/wall" \| "/screen"`（或别名 home/twin/ai/voice/wall/screen/数据大屏/企业展示墙…） | 切换到指定页面 |
|
||||
| `navigate_rel` | `delta: 1 \| -1` | 向右 / 向左循环切换页面 |
|
||||
|
||||
### 4.2 全局视觉识别
|
||||
|
||||
| action | params | 效果 |
|
||||
|---|---|---|
|
||||
| `vision_set` | `enabled: true \| false` | 开启 / 关闭全局人物识别（YOLO 人脸/姿态/手势） |
|
||||
|
||||
### 4.3 AI 助手页（/ai）
|
||||
|
||||
| action | params | 效果 |
|
||||
|---|---|---|
|
||||
| `ai_input` | `text: "介绍一下入驻政策"` | 输入问题并自动发送 |
|
||||
| `ai_preset` | `index: 0 \| 1 \| 2` | 选择预设问题并发送 |
|
||||
| `ai_company` | `action: "next" \| "prev"` | 切换展示企业 |
|
||||
| `ai_zone` | `zone: "加速区" \| "国际区" \| "成长区"` | 切换企业分区 |
|
||||
|
||||
### 4.4 语音对话页（/voice）
|
||||
|
||||
| action | params | 效果 |
|
||||
|---|---|---|
|
||||
| `voice_start` | — | 启动语音对话 |
|
||||
| `voice_stop` | — | 关闭语音对话 |
|
||||
| `voice_refresh` | — | 刷新页面 |
|
||||
|
||||
### 4.5 媒体轮播页（/screen）
|
||||
|
||||
| action | params | 效果 |
|
||||
|---|---|---|
|
||||
| `play` / `pause` | — | 播放 / 暂停 |
|
||||
| `next` / `prev` | — | 下一项 / 上一项 |
|
||||
| `set_mode` | `mode: sequential \| shuffle \| loop` | 播放模式 |
|
||||
| `play_target` | `path: "媒体路径"` | 播放指定媒体 |
|
||||
| `settings_changed` | 设置对象 | 应用设置变更 |
|
||||
| `playlist_changed` | — | 重新加载播放列表 |
|
||||
|
||||
### 4.6 企业展示墙页（/wall）
|
||||
|
||||
> 39 家入驻企业信息缓慢滚动展示墙（数据来源：《入驻企业信息表》）
|
||||
|
||||
| action | params | 效果 |
|
||||
|---|---|---|
|
||||
| `wall_pause` | — | 暂停滚动 |
|
||||
| `wall_resume` | — | 继续滚动 |
|
||||
| `wall_speed` | `speed: slow \| normal \| fast` | 滚动速度：慢速 150s / 标准 80s / 快速 40s 一圈 |
|
||||
|
||||
### 4.7 全局
|
||||
|
||||
| action | params | 效果 |
|
||||
|---|---|---|
|
||||
| `alert` | `text: "通知内容"` | 全局通知弹窗 |
|
||||
| `show_card` | 卡片数据 | 展示信息卡片 |
|
||||
| `minimize` | — | 最小化窗口（Tauri） |
|
||||
|
||||
## 5. 命令行发送
|
||||
|
||||
### mosquitto_pub（推荐，需安装 `mosquitto-clients`）
|
||||
|
||||
```bash
|
||||
# 切页到 AI 助手
|
||||
mosquitto_pub -h 192.168.1.3 -p 1883 -u <用户名> -P <密码> \
|
||||
-t opc/display/command \
|
||||
-m '{"action":"navigate","params":{"page":"/ai"}}'
|
||||
|
||||
# 向左切页
|
||||
mosquitto_pub -h 192.168.1.3 -p 1883 -u <用户名> -P <密码> \
|
||||
-t opc/display/command -m '{"action":"navigate_rel","params":{"delta":-1}}'
|
||||
|
||||
# 关闭人物识别
|
||||
mosquitto_pub -h 192.168.1.3 -p 1883 -u <用户名> -P <密码> \
|
||||
-t opc/display/command -m '{"action":"vision_set","params":{"enabled":false}}'
|
||||
|
||||
# AI 页提问
|
||||
mosquitto_pub -h 192.168.1.3 -p 1883 -u <用户名> -P <密码> \
|
||||
-t opc/display/command -m '{"action":"ai_input","params":{"text":"介绍一下入驻政策"}}'
|
||||
|
||||
# 切换企业分区
|
||||
mosquitto_pub -h 192.168.1.3 -p 1883 -u <用户名> -P <密码> \
|
||||
-t opc/display/command -m '{"action":"ai_zone","params":{"zone":"国际区"}}'
|
||||
|
||||
# 语音启动 / 媒体下一项
|
||||
mosquitto_pub -h 192.168.1.3 -p 1883 -u <用户名> -P <密码> \
|
||||
-t opc/display/command -m '{"action":"voice_start","params":{}}'
|
||||
mosquitto_pub -h 192.168.1.3 -p 1883 -u <用户名> -P <密码> \
|
||||
-t opc/display/command -m '{"action":"next","params":{}}'
|
||||
|
||||
# 切到企业展示墙 / 暂停滚动
|
||||
mosquitto_pub -h 192.168.1.3 -p 1883 -u <用户名> -P <密码> \
|
||||
-t opc/display/command -m '{"action":"navigate","params":{"page":"/wall"}}'
|
||||
mosquitto_pub -h 192.168.1.3 -p 1883 -u <用户名> -P <密码> \
|
||||
-t opc/display/command -m '{"action":"wall_pause","params":{}}'
|
||||
```
|
||||
|
||||
### Python（paho-mqtt，后端 venv 自带）
|
||||
|
||||
```bash
|
||||
cd /Volumes/Pine/mycode/DPM/dpm/backend
|
||||
.venv/bin/python - <<'EOF'
|
||||
import paho.mqtt.client as mqtt
|
||||
|
||||
def send(action, params=None):
|
||||
c = mqtt.Client()
|
||||
c.username_pw_set("用户名", "密码")
|
||||
c.connect("192.168.1.3", 1883, 30)
|
||||
c.publish("opc/display/command", str({"action": action, "params": params or {}}))
|
||||
c.disconnect()
|
||||
|
||||
send("navigate", {"page": "/voice"})
|
||||
send("ai_company", {"action": "next"})
|
||||
send("voice_stop")
|
||||
EOF
|
||||
```
|
||||
|
||||
### MQTTX（图形界面）
|
||||
|
||||
1. 新建连接：`mqtt://192.168.1.3:1883`，填用户名/密码
|
||||
2. 发布到 `opc/display/command`，消息体如上 JSON
|
||||
|
||||
## 6. 订阅调试
|
||||
|
||||
```bash
|
||||
# 监听前端页面上报（验证"当前页面"）
|
||||
mosquitto_sub -h 192.168.1.3 -p 1883 -u <用户名> -P <密码> -t 'opc/frontend/state'
|
||||
|
||||
# 监听心跳
|
||||
mosquitto_sub -h 192.168.1.3 -p 1883 -u <用户名> -P <密码> -t 'opc/display/heartbeat'
|
||||
|
||||
# 监听全部指令
|
||||
mosquitto_sub -h 192.168.1.3 -p 1883 -u <用户名> -P <密码> -t 'opc/display/command' -v
|
||||
```
|
||||
@@ -1,168 +0,0 @@
|
||||
# DPM 实时语音对话整合方案（复用 live-avatar 云化栈）
|
||||
|
||||
> 目标：砍掉数字人/口型/wav2lip 等全部视觉部分，只保留「语音识别 + 对话 + 实时交互」链路；
|
||||
> 前端在 DPM React 应用中新增对话页（复用 live-avatar demo 前端的样式与逻辑）；
|
||||
> 后端并入 DPM `backend/pyproject.toml`。
|
||||
> 基线：live-avatar 云化栈已在 `/Volumes/Pine/mycode/opc/live` 跑通（s2s v0.2.11 + cloud-migration 补丁 + demo 前端）。
|
||||
|
||||
---
|
||||
|
||||
## 0. 现状对比
|
||||
|
||||
| | live-avatar（已跑通） | DPM（现状） |
|
||||
|---|---|---|
|
||||
| 前端 | 原生 JS demo（index.html / main.js / s2s-ws-client.js / 两个 worklet） | React 19 + Vite + Tailwind；`/ai` 页已有 `AiChatPanel`（文字 + **录音上传式**语音） |
|
||||
| 后端 | s2s 独立进程 :8765（VAD 本地 + 云 ASR/LLM/TTS） | FastAPI :8000；已有 `app/asr.py`（dashscope paraformer）、`app/llm.py`（qwen + 工具）、`SSL_CERT_FILE=certifi` 同款处理 |
|
||||
| 语音体验 | **实时全双工**：本地 VAD → 流式云 ASR → 流式 LLM → 流式 TTS → 音频回放 | 录音 → 上传 → 转写 → 文字回复（无流式、无 TTS 回音） |
|
||||
|
||||
**DPM 缺的正是 live-avatar 栈的实时交互能力**；两边的云凭据/CA 处理已一致（dashscope + certifi），合并成本低。
|
||||
|
||||
---
|
||||
|
||||
## 1. 两个关键决策点
|
||||
|
||||
### 1.1 后端：内嵌进 DPM FastAPI（推荐） vs 独立 s2s 进程
|
||||
|
||||
- **推荐：内嵌**。s2s v0.2.11 的 `api/openai_realtime/websocket_router.create_app(pool, stop_event)` 本身就返回一个 FastAPI 应用，
|
||||
可在 DPM `app.main` 里构建管线池后 `app.mount("/v1", s2s_app)`，与现有 `/api/*` 共存、单进程单端口。
|
||||
Tauri 桌面打包只需一个后端进程，无子进程管理。
|
||||
- 备选：s2s 独立跑 :8765，DPM 负责拉起/守护。改动最小，但两进程两端口，打包与部署更重。
|
||||
|
||||
### 1.2 前端：iframe 内嵌（推荐 v1） vs React 组件移植（v2）
|
||||
|
||||
- **推荐 v1：iframe 内嵌**。把 live-avatar `demo/` 目录拷入 DPM `public/voice-demo/`（去掉数字人 iframe/视觉元素），
|
||||
React 新增 `VoiceAssistant.jsx`（路由 `/voice`）里 `<iframe src="/voice-demo/">` 全屏。
|
||||
**100% 复用现有样式与逻辑、零 React 改写、风险最低**；WS 地址由 `voice-demo/config.js` 按 `VITE_API_BASE` 派生。
|
||||
- v2：React 组件移植。把 `s2s-ws-client.js` + 两个 worklet 拷入 `src/voice/`，用 hooks 驱动、JSX 重绘圆球/气泡，
|
||||
与 DPM 布局/Tailwind 完全融合；工作量较大，建议先 v1 跑通再决定。
|
||||
|
||||
---
|
||||
|
||||
## 2. 后端合并清单
|
||||
|
||||
### 2.1 `backend/pyproject.toml`
|
||||
|
||||
```toml
|
||||
dependencies = [
|
||||
# ...现有依赖（fastapi/uvicorn/dashscope/certifi 已覆盖）...
|
||||
# s2s 云化栈（path 依赖，指向补丁后的 s2s-cloud；或 vendored 到 backend/vendor/）
|
||||
"speech-to-speech @ file:///Volumes/Pine/mycode/opc/live/s2s-cloud",
|
||||
# s2s 运行所需（base 依赖里 Darwin 专用的 mlx/transformers 等不要全装）
|
||||
"torch>=2.4", # VAD(silero) 本地必需；CPU 亦可
|
||||
"numpy>=1.26",
|
||||
"scipy>=1.10",
|
||||
"httpx>=0.28",
|
||||
"websockets>=12",
|
||||
"nltk==3.9.4",
|
||||
"openai==2.28.0",
|
||||
"soundfile>=0.13",
|
||||
]
|
||||
```
|
||||
|
||||
> 安装用 `pip install -e "backend[voice]" --no-deps` + 手动补依赖，避免 s2s base 依赖把 mlx/transformers 等大件拖进来。
|
||||
> `torch` 是唯一的大依赖（VAD 本地必需）；若部署机无 GPU，CPU torch 跑 silero 足够。
|
||||
|
||||
### 2.2 配置（`app/config.py` + `backend/.env`）
|
||||
|
||||
```env
|
||||
DASHSCOPE_API_KEY=sk-... # 已有（与 app/asr.py、app/llm.py 共用）
|
||||
S2S_ENABLED=1
|
||||
S2S_NUM_PIPELINES=4 # 并发会话数
|
||||
S2S_WS_PATH=/v1/realtime
|
||||
S2S_STT_MODEL=qwen3-asr-flash-realtime
|
||||
S2S_LLM_MODEL=qwen-plus
|
||||
S2S_TTS_MODEL=qwen3-tts-flash-realtime
|
||||
S2S_TTS_VOICE=Cherry
|
||||
```
|
||||
|
||||
### 2.3 代码（新 `app/s2s_bridge.py` + 挂载）
|
||||
|
||||
1. `s2s_bridge.py`：读取 Settings → 用与 `start_backend.sh` 等价的参数构建管线池
|
||||
（`--stt dashscope-asr --llm_backend chat-completions --tts qwen3-cloud --enable_live_transcription false`），
|
||||
返回 `websocket_router.create_app(pool, stop_event)`。
|
||||
2. `app/main.py` lifespan：`ENABLED` 时启动池 → `app.mount(settings.S2S_WS_PATH, s2s_app)`；关闭时 stop。
|
||||
3. 现有 `/api/ai/chat`、`/api/ai/asr` 保留（旧 AiChatPanel 流程不破坏）。
|
||||
|
||||
---
|
||||
|
||||
## 3. 前端合并清单
|
||||
|
||||
1. `src/pages/VoiceAssistant.jsx`（新）：
|
||||
- 全屏 `<iframe src="/voice-demo/" />`（v1），或未来 React 移植版。
|
||||
- 加入 `App.jsx` 路由：`<Route path="/voice" element={<VoiceAssistant />} />`（建议放在 ScreenLayout 内或独立全屏）。
|
||||
2. `public/voice-demo/`：拷贝 live-avatar `demo/`（index.html / main.js / style.css / ws/ / worklets/ / textchat.html），
|
||||
并删除数字人 iframe（`index.html` 里的 `#avatar-frame` → 背景改纯色/园区图）。
|
||||
3. `public/voice-demo/config.js`：暴露 `window.VOICE_WS_URL`（从 `VITE_API_BASE` 派生 `ws://<host>:<port>/v1/realtime`）。
|
||||
4. 麦克风权限：
|
||||
- 浏览器开发模式：`localhost` 直接可用。
|
||||
- **Tauri v2 打包**：`src-tauri/tauri.conf.json` 需 `app.macOSPrivateApi: true`（macOS 麦克风）+ 系统隐私授权说明；
|
||||
Windows 需在 manifest/权限描述中声明 microphone。
|
||||
|
||||
---
|
||||
|
||||
## 4. 实施步骤
|
||||
|
||||
| 步骤 | 内容 | 工作量 |
|
||||
|---|---|---|
|
||||
| P1 | 后端内嵌：pyproject 加依赖 → s2s_bridge.py → mount → 冒烟（WS 连得上、说话→回复→出声） | 0.5–1 天 |
|
||||
| P2 | 前端：public/voice-demo 拷贝去视觉化 → VoiceAssistant.jsx + 路由 → WS 指向 DPM 后端 | 0.5 天 |
|
||||
| P3 | 联调：DPM 页面语音对话全链路 + 并发验证 | 0.5 天 |
|
||||
| P4（可选） | React 原生移植（hooks 驱动 client + worklets，Tailwind 重绘 UI） | 1–2 天 |
|
||||
|
||||
---
|
||||
|
||||
## 5. 风险与注意
|
||||
|
||||
- **torch 体积**：VAD 本地必需（s2s 管线结构固定 VAD 本地），CPU 版可接受；若部署机极简可评估 VAD 上云（不推荐，延迟差）。
|
||||
- **单管线槽**：`S2S_NUM_PIPELINES` 默认 1 → 并发对话会拒连；按需调到 4。
|
||||
- **WS 鉴权**：内网/桌面场景可无鉴权；如需，在 session.update 里带 token 校验。
|
||||
- **iframe 限制**：v1 iframe 内样式隔离、无法用 DPM 主题；v2 移植可解决。
|
||||
- **Tauri 麦克风**：打包前务必配置 macOSPrivateApi + 权限描述，否则桌面版无声/无权限。
|
||||
|
||||
---
|
||||
|
||||
## 附录 A · 视觉触发子系统（USB 摄像头 + 桌面端检测）
|
||||
|
||||
> 目标：大屏前 USB 摄像头拍观众；检测有人**正对且停留 ≥10 秒** → 主动问候 + 全屏联动；问候后**静默 60 秒**。
|
||||
|
||||
### A.1 架构与数据通路
|
||||
|
||||
```
|
||||
大屏机（Tauri 应用，fullscreen）
|
||||
└─ USB 摄像头（大屏前，拍观众）
|
||||
└─ webview getUserMedia（一路视频流）
|
||||
├─ 持续检测：MediaPipe FaceMesh 1~2fps（本机、不出设备）
|
||||
│ ├─ 头姿 yaw/pitch 各 ±15° = 正对
|
||||
│ └─ 连续 10s 正对 → 触发（窗口内 ≥80% 帧正对，防抖）
|
||||
└─ 按需抓帧：LLM 决定时 canvas 取当前帧 → input_image → 云 qwen3-vl（人数/人物）
|
||||
触发动作：
|
||||
① 主动问候：client.sendUserText("检测到有人正对摄像头…请主动问候") + requestResponse()
|
||||
→ s2s 走 LLM 生成问候语 → TTS 播报（零后端改动，复用现有 WS 协议）
|
||||
② 全屏联动：调后端 → event_bus / mqtt.publish → 各屏响应
|
||||
静默窗口：问候触发后 60s 冷却，期内检测照跑但不触发
|
||||
```
|
||||
|
||||
### A.2 Tauri 摄像头权限（必须配置）
|
||||
|
||||
- **macOS**：`tauri.conf.json` → `app.macOSPrivateApi: true`；Info.plist 加
|
||||
`NSCameraUsageDescription` / `NSMicrophoneUsageDescription`（麦克风语音对话也需要）。
|
||||
- **Windows（NSIS 打包目标）**：WebView2 的 getUserMedia 跟随系统隐私设置
|
||||
（设置 → 隐私和安全性 → 相机/麦克风 → 允许该应用）；安装包建议在应用清单声明设备能力。
|
||||
- 大屏机摄像头常开：确认系统隐私里该应用允许相机，否则检测与抓帧同时失效。
|
||||
|
||||
### A.3 隐私与合规
|
||||
|
||||
- 持续检测帧**仅在本机内存处理、即用即弃**，不上云；只有 LLM 决定抓的那一帧发往百炼 VLM。
|
||||
- 大屏前采集观众人脸属个人信息处理 → 大屏旁需公示"影像识别用于互动服务"。
|
||||
|
||||
### A.4 落地清单（并入 P2/P3）
|
||||
|
||||
| 项 | 说明 |
|
||||
|---|---|
|
||||
| 前端新增 `useCameraDetect` hook | getUserMedia + MediaPipe FaceMesh + 状态机（IDLE→FACING→TRIGGER→SILENT） |
|
||||
| 依赖 | `@mediapipe/tasks-vision`（face_detector + face_landmarker） |
|
||||
| 参数化 | 停留 10s / 静默 60s / 角度 ±15° / 频率 1~2fps → 配置文件或设置页 |
|
||||
| 触发动作 | 主动问候（sendUserText+create）+ MQTT 联动（后端 API） |
|
||||
| 首对话抓帧 | 与检测共用同一路视频流，canvas 取帧 → input_image → qwen3-vl |
|
||||
|
||||
> 注：视觉子系统是**可选增强**，不影响主语音对话方案；摄像头/检测不可用时语音链路照常工作。
|
||||
@@ -1,78 +0,0 @@
|
||||
# Windows 打包部署指南（免重打包改地址）
|
||||
|
||||
## 一、打包后在 exe 同目录放置 config.json（推荐）
|
||||
|
||||
打包安装后，在 exe 所在目录（如 `C:\Program Files\云超服昆创园OPC运营中心\`）新建 **`config.json`**，
|
||||
改地址**不需要重新打包**，重启应用即生效：
|
||||
|
||||
```json
|
||||
{
|
||||
"api_base": "http://192.168.1.9:10085",
|
||||
"mqtt_url": "ws://192.168.1.3:8083/mqtt",
|
||||
"mqtt_username": "dpm",
|
||||
"mqtt_password": "123456",
|
||||
"voice_url": "ws://192.168.1.3:8765/v1/realtime"
|
||||
}
|
||||
```
|
||||
|
||||
- `api_base`：FastAPI 后端地址（API / 管理后台 / 媒体）
|
||||
- `mqtt_url`：EMQX Broker 的 **WebSocket** 地址（端口 8083）
|
||||
- `mqtt_username/password`：Broker 鉴权账号（默认 dpm / 123456，与后端 .env 一致）
|
||||
|
||||
地址解析优先级：**config.json > 后端 /api/config > 构建默认（192.168.1.9）**。
|
||||
|
||||
## 二、后端 /api/config 方式（同机部署自动生效）
|
||||
|
||||
后端 `.env`（backend/.env）配置 Broker 后，展播端启动时自动从
|
||||
`GET /api/config` 获取 MQTT 地址，**无需 config.json**：
|
||||
|
||||
```
|
||||
DPM_MQTT_HOST=192.168.1.3
|
||||
DPM_MQTT_PORT=1883
|
||||
DPM_MQTT_WS=ws://192.168.1.3:8083/mqtt # 展播端可访问的 WebSocket 地址
|
||||
DPM_VOICE_WS=ws://192.168.1.3:8765/v1/realtime # s2s 语音 WS（可选，默认按主机推导）
|
||||
MQTT_USERNAME=dpmserver
|
||||
MQTT_PASSWORD=你的密码
|
||||
```
|
||||
|
||||
> 若后端与 Broker 同机：后端默认 `MQTT_WS_URL=ws://localhost:8083/mqtt` 即可，
|
||||
> 展播端连 `192.168.1.9` 也通；跨机必须显式配置上面的 IP。
|
||||
|
||||
## 三、构建期固定地址（不推荐，需重新打包）
|
||||
|
||||
在项目根目录（**构建机器上**）配置 `.env.local`（已被 gitignore）：
|
||||
|
||||
```
|
||||
VITE_API_BASE=http://192.168.1.9:10085
|
||||
VITE_MQTT_URL=ws://192.168.1.3:8083/mqtt
|
||||
VITE_MQTT_USERNAME=dpm
|
||||
VITE_MQTT_PASSWORD=123456
|
||||
```
|
||||
|
||||
然后 `yarn build:win`（脚本自动清理旧 exe 进程再打包）。
|
||||
|
||||
## 四、打包前必须确认
|
||||
|
||||
| 检查项 | 说明 |
|
||||
|---|---|
|
||||
| exe 未被运行 | `taskkill /f /im "昆明大学生创业园展播系统.exe"`，或直接用 `yarn build:win` |
|
||||
| 后端已启动 | 浏览器打开 `http://<后端IP>:10085/api/health` 应返回 `{"ok":true,...}` |
|
||||
| Broker WebSocket 已开 | EMQX 需开启 8083 端口 WS 监听，且账号密码与 config.json 一致 |
|
||||
| Windows 防火墙 | 首次运行允许，或放行出站 10085/8083（管理端上传用 10085） |
|
||||
|
||||
## 五、常见排查
|
||||
|
||||
```
|
||||
# 测试后端连通（展播机 PowerShell）
|
||||
Invoke-WebRequest http://192.168.1.9:10085/api/health
|
||||
|
||||
# 测试 Broker WS 端口
|
||||
Test-NetConnection 192.168.1.3 -Port 8083
|
||||
|
||||
# 看前端启动日志（bootstrap 会打印 MQTT 地址来源）
|
||||
打开应用后按 F12（WebView2 开发者工具）→ Console
|
||||
```
|
||||
|
||||
- 应用日志打印 `[bootstrap] config.json MQTT -> ...` = config.json 生效
|
||||
- 打印 `[bootstrap] /api/config MQTT -> ...` = 后端下发
|
||||
- 两者都没有 = 两个来源都不可达，请检查 IP/防火墙
|
||||
@@ -12,6 +12,11 @@
|
||||
|
||||
<body>
|
||||
<div id="root"></div>
|
||||
<div id="fatal-error" style="display:none;position:fixed;inset:0;z-index:99999;background:#f7fafd;color:#1d2c44;padding:40px;font-family:system-ui,sans-serif;overflow:auto;">
|
||||
<h3 style="margin:0 0 12px;">页面加载出错</h3>
|
||||
<pre id="fatal-error-text" style="white-space:pre-wrap;font-size:12px;background:#fff;border:1px solid #e3ecf7;padding:14px;border-radius:8px;"></pre>
|
||||
<button onclick="location.reload()" style="margin-top:12px;padding:8px 18px;cursor:pointer;">重载</button>
|
||||
</div>
|
||||
<script type="module" src="/src/main.jsx"></script>
|
||||
</body>
|
||||
</html>
|
||||
|
||||
+1
-1
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"name": "dpm",
|
||||
"private": true,
|
||||
"version": "0.1.0",
|
||||
"version": "0.1.2",
|
||||
"type": "module",
|
||||
"scripts": {
|
||||
"dev": "vite",
|
||||
|
||||
Generated
+1
-1
@@ -5135,7 +5135,7 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "昆明大学生创业园展播系统"
|
||||
version = "0.1.0"
|
||||
version = "0.1.2"
|
||||
dependencies = [
|
||||
"serde",
|
||||
"tauri",
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "昆明大学生创业园展播系统"
|
||||
version = "0.1.0"
|
||||
version = "0.1.2"
|
||||
description = "大屏媒体轮播系统 - Tauri App"
|
||||
authors = ["Pine"]
|
||||
edition = "2021"
|
||||
|
||||
@@ -2,9 +2,10 @@
|
||||
"$schema": "../gen/schemas/desktop-schema.json",
|
||||
"identifier": "default",
|
||||
"description": "Capability for the main window",
|
||||
"windows": ["main"],
|
||||
"windows": ["main", "secondary"],
|
||||
"permissions": [
|
||||
"core:default",
|
||||
"core:app:default",
|
||||
"core:window:default",
|
||||
"core:window:allow-minimize",
|
||||
"core:window:allow-set-fullscreen",
|
||||
|
||||
+96
-4
@@ -4,7 +4,92 @@
|
||||
|
||||
use std::fs;
|
||||
use std::path::PathBuf;
|
||||
use tauri::{AppHandle, Manager};
|
||||
use tauri::{AppHandle, Manager, WebviewUrl, WebviewWindowBuilder};
|
||||
|
||||
/// 双屏控制：on=true 在第二块屏幕上新建一个全屏窗口（加载前端首页），on=false 关闭它。
|
||||
/// 用于 admin 端「启动双屏」，让展播内容同时投到电脑的另一块显示器。
|
||||
#[tauri::command]
|
||||
fn set_dual_screen(app: AppHandle, on: bool, parent_id: Option<String>) -> Result<String, String> {
|
||||
if !on {
|
||||
if let Some(w) = app.get_webview_window("secondary") {
|
||||
let _ = w.close();
|
||||
}
|
||||
return Ok("off".into());
|
||||
}
|
||||
// 已存在则直接显示/全屏/聚焦
|
||||
if let Some(w) = app.get_webview_window("secondary") {
|
||||
let _ = w.show();
|
||||
let _ = w.unminimize();
|
||||
let _ = w.set_fullscreen(true);
|
||||
let _ = w.set_focus();
|
||||
return Ok("on".into());
|
||||
}
|
||||
// 主屏位置（用于排除主屏；必须把副窗放到主屏之外）
|
||||
let primary_pos = app
|
||||
.primary_monitor()
|
||||
.ok()
|
||||
.flatten()
|
||||
.map(|pm| { let p = pm.position(); (p.x, p.y) });
|
||||
// 获取全部显示器，寻找一块与主屏位置不同的副屏
|
||||
let monitors = app.available_monitors().map_err(|e| format!("读取显示器失败: {e}"))?;
|
||||
eprintln!("[dual] monitors={} primary_pos={:?}", monitors.len(), primary_pos);
|
||||
let mut target: Option<(i32, i32, u32, u32)> = None; // (x, y, w, h)
|
||||
for m in &monitors {
|
||||
let p = m.position();
|
||||
if Some((p.x, p.y)) != primary_pos {
|
||||
let s = m.size();
|
||||
target = Some((p.x, p.y, s.width, s.height));
|
||||
break;
|
||||
}
|
||||
}
|
||||
// 绝对不允许在主屏桌面上开启；没有副屏则拒绝
|
||||
let (x, y, w, h) = match target {
|
||||
Some(t) => t,
|
||||
None => return Err("未检测到副屏（需连接第二块显示器），无法开启双屏".into()),
|
||||
};
|
||||
eprintln!("[dual] secondary {}x{} @ {},{}", w, h, x, y);
|
||||
let win = WebviewWindowBuilder::new(&app, "secondary", WebviewUrl::default())
|
||||
.title("OPC 运营中心 · 双屏")
|
||||
.position(x as f64, y as f64)
|
||||
.inner_size(w as f64, h as f64)
|
||||
// 由 Rust 明确告知副窗：它的身份是 secondary，且从属于主屏 device_id（parent_id）
|
||||
.initialization_script(format!(
|
||||
"window.__DPM_WINDOW_LABEL__='secondary';window.__DPM_PARENT_ID__='{}';",
|
||||
parent_id.unwrap_or_default().replace('\'', "")
|
||||
))
|
||||
.build()
|
||||
.map_err(|e| format!("创建副窗失败: {e}"))?;
|
||||
// 等窗口内容就绪后再全屏（避免过早全屏导致白屏/布局异常）
|
||||
let win2 = win.clone();
|
||||
std::thread::spawn(move || {
|
||||
std::thread::sleep(std::time::Duration::from_millis(400));
|
||||
let _ = win2.set_fullscreen(true);
|
||||
let _ = win2.set_focus();
|
||||
});
|
||||
Ok("on".into())
|
||||
}
|
||||
|
||||
/// 安装更新：把前端已下载的安装包字节写入磁盘 → 启动安装程序 → 退出应用。
|
||||
/// bytes 由前端 fetch /download 得到；启动安装程序后进程独立，应用退出不影响它。
|
||||
#[tauri::command]
|
||||
fn install_update(app: AppHandle, bytes: Vec<u8>, filename: String) -> Result<String, String> {
|
||||
if bytes.len() < 1000 {
|
||||
return Err("下载内容无效或为空".into());
|
||||
}
|
||||
let dest = std::env::temp_dir().join(filename);
|
||||
std::fs::write(&dest, &bytes).map_err(|e| format!("写入安装包失败: {e}"))?;
|
||||
// 启动安装程序（Windows 下 Command::spawn 为独立进程，应用退出后仍在运行）
|
||||
std::process::Command::new(&dest)
|
||||
.spawn()
|
||||
.map_err(|e| format!("启动安装程序失败: {e}"))?;
|
||||
// 稍后退出应用，让安装程序接管
|
||||
let app2 = app.clone();
|
||||
std::thread::spawn(move || {
|
||||
std::thread::sleep(std::time::Duration::from_millis(800));
|
||||
app2.exit(0);
|
||||
});
|
||||
Ok(dest.to_string_lossy().into_owned())
|
||||
}
|
||||
|
||||
/// 读取运行时配置（部署用，免重新打包）：
|
||||
/// 1) exe 同目录 config.json / dpm.config.json（最高优先）
|
||||
@@ -47,11 +132,18 @@ pub fn run() {
|
||||
tauri_plugin_autostart::MacosLauncher::LaunchAgent,
|
||||
Some(vec![]),
|
||||
))
|
||||
.invoke_handler(tauri::generate_handler![read_runtime_config])
|
||||
.invoke_handler(tauri::generate_handler![
|
||||
read_runtime_config,
|
||||
set_dual_screen,
|
||||
install_update
|
||||
])
|
||||
.on_window_event(|window, event| {
|
||||
if let tauri::WindowEvent::CloseRequested { api, .. } = event {
|
||||
api.prevent_close();
|
||||
let _ = window.minimize();
|
||||
// 仅主窗口关闭=最小化（进程常驻）；双屏副窗口可正常关闭
|
||||
if window.label() == "main" {
|
||||
api.prevent_close();
|
||||
let _ = window.minimize();
|
||||
}
|
||||
}
|
||||
})
|
||||
.run(tauri::generate_context!())
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"$schema": "https://schema.tauri.app/config/2",
|
||||
"productName": "云超服昆创园OPC运营中心",
|
||||
"version": "0.1.0",
|
||||
"version": "0.1.2",
|
||||
"identifier": "com.pine.dpm",
|
||||
"build": {
|
||||
"beforeDevCommand": "yarn build && yarn dev",
|
||||
|
||||
@@ -3,7 +3,7 @@ import Icon from './Icons';
|
||||
import ShowCard from './ShowCard';
|
||||
import { getApiBase as API_BASE } from '../config';
|
||||
import { Sfx } from '../utils/sounds';
|
||||
import { getMqttStatus } from '../utils/mqtt';
|
||||
import { getMqttStatus, getClientId } from '../utils/mqtt';
|
||||
import { aiStatus } from '../utils/statusBus';
|
||||
import { renderMarkdown } from '../voice/markdown.jsx';
|
||||
|
||||
@@ -161,7 +161,7 @@ export default function AiChatPanel() {
|
||||
const res = await fetch(`${API_BASE()}/api/ai/chat`, {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ messages: history }),
|
||||
body: JSON.stringify({ messages: history, screen_id: getClientId() }),
|
||||
});
|
||||
if (res.ok) {
|
||||
data = await res.json();
|
||||
|
||||
@@ -0,0 +1,35 @@
|
||||
import { Component } from 'react';
|
||||
|
||||
/* 全局错误边界：渲染崩溃时不白屏，显示具体错误 + 重载按钮，便于双屏/异常定位 */
|
||||
export default class ErrorBoundary extends Component {
|
||||
constructor(props) {
|
||||
super(props);
|
||||
this.state = { err: null };
|
||||
}
|
||||
static getDerivedStateFromError(err) {
|
||||
return { err };
|
||||
}
|
||||
componentDidCatch(err, info) {
|
||||
// eslint-disable-next-line no-console
|
||||
console.error('[AppError]', err, info);
|
||||
}
|
||||
render() {
|
||||
if (this.state.err) {
|
||||
return (
|
||||
<div style={{ padding: 40, fontFamily: 'sans-serif', color: '#333' }}>
|
||||
<h3>页面加载出错</h3>
|
||||
<pre style={{ whiteSpace: 'pre-wrap', fontSize: 12, background: '#f5f5f5', padding: 12, borderRadius: 8 }}>
|
||||
{String(this.state.err && (this.state.err.message || this.state.err))}
|
||||
</pre>
|
||||
<button
|
||||
style={{ marginTop: 12, padding: '8px 16px', cursor: 'pointer' }}
|
||||
onClick={() => { this.setState({ err: null }); window.location.reload(); }}
|
||||
>
|
||||
重载
|
||||
</button>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
return this.props.children;
|
||||
}
|
||||
}
|
||||
@@ -141,9 +141,19 @@ export default function PageHeader() {
|
||||
{/* <span className="bd-logo">
|
||||
<Image src="/logo.png" alt="Logo" />
|
||||
</span> */}
|
||||
<Avatar size="lg">
|
||||
<AvatarImage src="/logo.png" alt="" />
|
||||
</Avatar>
|
||||
<span
|
||||
className="bd-logo-update"
|
||||
role="button"
|
||||
tabIndex={0}
|
||||
title="点击检测更新"
|
||||
aria-label="点击检测更新"
|
||||
onClick={() => window.dispatchEvent(new CustomEvent('dpm:check-update'))}
|
||||
onKeyDown={(e) => { if (e.key === 'Enter' || e.key === ' ') window.dispatchEvent(new CustomEvent('dpm:check-update')); }}
|
||||
>
|
||||
<Avatar size="lg">
|
||||
<AvatarImage src="/logo.png" alt="" />
|
||||
</Avatar>
|
||||
</span>
|
||||
<div className="bd-header-titles">
|
||||
<h1 className="bd-header-title">昆明市大学生创业园 <span className="bd-header-title-accent">· OPC 智能园区数字运营中心</span></h1>
|
||||
<div className="bd-header-sub">云南省首家政府主办大学生创业孵化园区 · <span className="bd-header-content—highlights">空间+孵化+融资+政策+资源+AI赋能+综合服务</span></div>
|
||||
|
||||
@@ -2,6 +2,7 @@ import { Outlet } from 'react-router-dom';
|
||||
import PageHeader from './PageHeader';
|
||||
import DpmOverlays from './DpmOverlays';
|
||||
import ToolStatusToast from './ToolStatusToast';
|
||||
import UpdateNotify from './UpdateNotify';
|
||||
import GlobalVision from './GlobalVision';
|
||||
import { usePageNav } from '../utils/pageNav';
|
||||
import { useMqttControl } from '../utils/useMqttControl';
|
||||
@@ -29,6 +30,7 @@ export default function ScreenLayout() {
|
||||
<GlobalVision />
|
||||
<DpmOverlays />
|
||||
<ToolStatusToast />
|
||||
<UpdateNotify />
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
@@ -0,0 +1,91 @@
|
||||
import { useState, useEffect, useRef, useCallback } from 'react';
|
||||
import Icon from './Icons';
|
||||
import { checkForUpdate, downloadAndInstall } from '../utils/updateCheck';
|
||||
|
||||
/* =========================================================
|
||||
UpdateNotify —— 检测新版本（Windows）
|
||||
· 启动静默检查一次 + 每 30 分钟一次（仅在有更新时提示）
|
||||
· 页眉 logo 点击 → dpm:check-update → 主动检测
|
||||
· 「下载更新」→ 下载安装包 → 启动安装程序 → 应用退出
|
||||
========================================================= */
|
||||
export default function UpdateNotify() {
|
||||
const [upd, setUpd] = useState(null); // 更新提示
|
||||
const [msg, setMsg] = useState(null); // 临时提示（已是最新 / 失败）
|
||||
const [dismissed, setDismissed] = useState(false);
|
||||
const [busy, setBusy] = useState(null); // 'downloading' | 'installing'
|
||||
const [errText, setErrText] = useState('');
|
||||
const timerRef = useRef(null);
|
||||
|
||||
const run = useCallback(async (manual) => {
|
||||
const u = await checkForUpdate();
|
||||
if (u) {
|
||||
if (u.available) { setUpd(u); setDismissed(false); }
|
||||
else if (manual) { setMsg(`已是最新版本 v${u.latest}`); flash(); }
|
||||
} else if (manual) { setMsg('更新检测失败'); flash(); }
|
||||
}, []);
|
||||
|
||||
const flash = () => {
|
||||
if (timerRef.current) clearTimeout(timerRef.current);
|
||||
timerRef.current = setTimeout(() => setMsg(null), 3000);
|
||||
};
|
||||
|
||||
const handleDownload = useCallback(async () => {
|
||||
if (!upd || busy) return;
|
||||
setBusy('downloading');
|
||||
setErrText('');
|
||||
try {
|
||||
await downloadAndInstall(upd.url, upd.name);
|
||||
setBusy('installing'); // 应用即将退出
|
||||
} catch (e) {
|
||||
setErrText(String((e && e.message) || e));
|
||||
setBusy(null);
|
||||
}
|
||||
}, [upd, busy]);
|
||||
|
||||
useEffect(() => {
|
||||
const onCheck = () => run(true);
|
||||
window.addEventListener('dpm:check-update', onCheck);
|
||||
run(false);
|
||||
const iv = setInterval(() => run(false), 30 * 60 * 1000);
|
||||
return () => {
|
||||
window.removeEventListener('dpm:check-update', onCheck);
|
||||
clearInterval(iv);
|
||||
if (timerRef.current) clearTimeout(timerRef.current);
|
||||
};
|
||||
}, [run]);
|
||||
|
||||
if (!upd || dismissed) {
|
||||
if (msg) {
|
||||
return (
|
||||
<div className="upd-toast upd-msg" role="status">
|
||||
<div className="upd-ico"><Icon name="check" size={15} /></div>
|
||||
<div className="upd-body"><b>{msg}</b></div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
return (
|
||||
<div className="upd-toast" role="status">
|
||||
<div className="upd-ico"><Icon name="rocket" size={15} /></div>
|
||||
<div className="upd-body">
|
||||
{busy === 'downloading' && <b>正在下载更新 v{upd.latest}…</b>}
|
||||
{busy === 'installing' && <b>正在启动安装程序，应用即将退出…</b>}
|
||||
{!busy && <b>发现新版本 v{upd.latest}</b>}
|
||||
{!busy && <span>当前 v{upd.current} · 点击下载更新并安装（Windows）</span>}
|
||||
{errText && <span style={{ color: '#f04438' }}>{errText}</span>}
|
||||
</div>
|
||||
<button
|
||||
className="upd-btn"
|
||||
disabled={!!busy}
|
||||
onClick={handleDownload}
|
||||
>
|
||||
{busy ? '处理中…' : '下载更新'}
|
||||
</button>
|
||||
{!busy && (
|
||||
<button className="upd-close" onClick={() => setDismissed(true)} aria-label="关闭">×</button>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
+23
-3
@@ -1,11 +1,29 @@
|
||||
import React from "react";
|
||||
import ReactDOM from "react-dom/client";
|
||||
import App from "./App";
|
||||
import ErrorBoundary from "./components/ErrorBoundary";
|
||||
import { ThemeProvider } from "@appica/ui-react/providers/theme-provider";
|
||||
import { getApiBase } from "./config";
|
||||
import { invoke } from "@tauri-apps/api/core";
|
||||
import "./styles/global.css";
|
||||
|
||||
/* 全局致命错误捕获：任何模块加载/运行/异步错误 → 显示到页面（替代白屏），便于副屏等定位 */
|
||||
function installFatalErrorDiagnostics() {
|
||||
const show = (label, err) => {
|
||||
const el = document.getElementById("fatal-error");
|
||||
if (!el || el.style.display === "block") return;
|
||||
const text = document.getElementById("fatal-error-text");
|
||||
if (text) {
|
||||
const msg = (err && (err.stack || err.message)) || String(err);
|
||||
text.textContent = `${label}\n${msg}`;
|
||||
}
|
||||
el.style.display = "block";
|
||||
};
|
||||
window.addEventListener("error", (e) => show("加载错误:", e.error || e.message));
|
||||
window.addEventListener("unhandledrejection", (e) => show("未处理异常:", e.reason));
|
||||
}
|
||||
installFatalErrorDiagnostics();
|
||||
|
||||
/* =========================================================
|
||||
启动引导（打包部署关键）—— 地址解析优先级：
|
||||
1. exe 旁 config.json（Tauri 命令读取，部署免重打包改 IP）★
|
||||
@@ -60,9 +78,11 @@ async function bootstrapRuntimeConfig() {
|
||||
bootstrapRuntimeConfig().then(() => {
|
||||
ReactDOM.createRoot(document.getElementById("root")).render(
|
||||
<React.StrictMode>
|
||||
<ThemeProvider>
|
||||
<App />
|
||||
</ThemeProvider>
|
||||
<ErrorBoundary>
|
||||
<ThemeProvider>
|
||||
<App />
|
||||
</ThemeProvider>
|
||||
</ErrorBoundary>
|
||||
</React.StrictMode>,
|
||||
);
|
||||
});
|
||||
|
||||
@@ -9,6 +9,7 @@ import { useCallback, useEffect, useRef, useState } from 'react';
|
||||
import { S2sWsRealtimeClient } from '../voice/s2s-ws-client.js';
|
||||
import useVisionDetection, { VISION_DWELL_MS } from '../voice/useVisionDetection.js';
|
||||
import { getApiBase as API_BASE, getVoiceWsUrl } from '../config';
|
||||
import { getClientId } from '../utils/mqtt';
|
||||
import { aiStatus } from '../utils/statusBus';
|
||||
import '../styles/datascreen.css';
|
||||
import './voice-original.css';
|
||||
@@ -242,7 +243,7 @@ export default function VoiceAssistant() {
|
||||
const res = await fetch(`${API_BASE()}/api/tools/exec`, {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ name, args }),
|
||||
body: JSON.stringify({ name, args, screen_id: getClientId() }),
|
||||
});
|
||||
const data = await res.json();
|
||||
output = data?.result ?? JSON.stringify(data);
|
||||
@@ -480,10 +481,14 @@ export default function VoiceAssistant() {
|
||||
});
|
||||
|
||||
// 手势 → 对话控制：举手 toggle —— 无对话则开始，正在对话则结束
|
||||
// 【已按需求关闭】视觉手势识别误判会触发右上角弹窗/误开对话，故这里不再响应手势。
|
||||
const onGesture = useCallback(() => {
|
||||
// 视觉手势已关闭：识别到人脸/手势不再自动开始/结束对话
|
||||
}, []);
|
||||
const onGesture = useCallback((g) => {
|
||||
if (g !== 'raise') return;
|
||||
if (clientRef.current) {
|
||||
void stop();
|
||||
} else {
|
||||
void start();
|
||||
}
|
||||
}, [start, stop]);
|
||||
|
||||
useEffect(() => {
|
||||
gestureRef.current = onGesture;
|
||||
@@ -517,7 +522,7 @@ export default function VoiceAssistant() {
|
||||
loading: '摄像头启动中…',
|
||||
detecting: `识别中 · ${vision.faces} 人${vision.faces > 0 ? `（后端 ${vision.latency ?? '-'}ms）` : ' · 未检到人脸'} · 链路${vision.modelPhase === 'ok' ? 'OK' : vision.modelPhase === 'failed' ? `失败:${(vision.modelErr || '').slice(0, 40)}` : '连接中'} · 亮度${vision.frameLum ?? '-'}${vision.detectErr ? ` · 异常:${String(vision.detectErr).slice(0, 50)}` : ''}`,
|
||||
facing: `面向大屏 ${(vision.dwellMs / 1000).toFixed(1)}s / ${VISION_DWELL_MS / 1000}s`,
|
||||
triggered: '已触发 · 主动问候',
|
||||
triggered: '已识别到访客（主动问候已关闭）',
|
||||
silent: '静默中',
|
||||
error: '摄像头不可用',
|
||||
}[vision.status] ?? '';
|
||||
@@ -629,7 +634,7 @@ export default function VoiceAssistant() {
|
||||
</span>
|
||||
<span>
|
||||
{vision.gesture === 'raise' && (
|
||||
<b style={{ color: '#2f6bff' }}>您好呀 · {live ? '开始对话' : '结束对话'}</b>
|
||||
<b style={{ color: '#2f6bff' }}>{live ? '开始对话' : '结束对话'}</b>
|
||||
)}
|
||||
{visionStatusText}
|
||||
</span>
|
||||
|
||||
@@ -2201,6 +2201,86 @@
|
||||
color: #fff;
|
||||
background: linear-gradient(140deg, var(--bd-blue), var(--bd-cyan));
|
||||
}
|
||||
/* ---- 页眉 logo：点击检测更新 ---- */
|
||||
.bd-logo-update {
|
||||
display: inline-flex;
|
||||
cursor: pointer;
|
||||
border-radius: 12px;
|
||||
transition: transform 0.15s ease, box-shadow 0.15s ease;
|
||||
-webkit-tap-highlight-color: transparent;
|
||||
}
|
||||
.bd-logo-update:hover {
|
||||
transform: translateY(-1px);
|
||||
box-shadow: 0 4px 12px rgba(29, 93, 206, 0.25);
|
||||
}
|
||||
.bd-logo-update:active { transform: translateY(0); }
|
||||
.bd-logo-update:focus-visible { outline: 2px solid var(--bd-blue); outline-offset: 2px; }
|
||||
|
||||
/* ---- 底部：更新提示（Windows 检测到新版本） ---- */
|
||||
.upd-toast {
|
||||
position: fixed;
|
||||
bottom: 24px;
|
||||
left: 50%;
|
||||
transform: translateX(-50%);
|
||||
z-index: 3200;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 12px;
|
||||
max-width: 540px;
|
||||
padding: 12px 16px;
|
||||
background: #fff;
|
||||
border: 1px solid var(--bd-line);
|
||||
border-radius: 12px;
|
||||
box-shadow: 0 12px 30px rgba(29, 44, 68, 0.16);
|
||||
animation: bdOverlayIn 0.3s ease;
|
||||
}
|
||||
.upd-ico {
|
||||
width: 30px;
|
||||
height: 30px;
|
||||
flex-shrink: 0;
|
||||
border-radius: 8px;
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
color: #fff;
|
||||
background: linear-gradient(140deg, var(--bd-blue), var(--bd-cyan));
|
||||
}
|
||||
.upd-body {
|
||||
flex: 1;
|
||||
min-width: 0;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 2px;
|
||||
}
|
||||
.upd-body b { font-size: 13px; color: var(--bd-ink); }
|
||||
.upd-body span { font-size: 11.5px; color: var(--bd-sub); }
|
||||
.upd-btn {
|
||||
flex-shrink: 0;
|
||||
padding: 7px 15px;
|
||||
border: none;
|
||||
border-radius: 8px;
|
||||
background: linear-gradient(140deg, var(--bd-blue), var(--bd-cyan));
|
||||
color: #fff;
|
||||
font-size: 12px;
|
||||
font-weight: 600;
|
||||
cursor: pointer;
|
||||
}
|
||||
.upd-btn:hover { opacity: 0.9; }
|
||||
.upd-msg .upd-ico { background: linear-gradient(140deg, var(--bd-green), #4ade80); }
|
||||
.upd-close {
|
||||
width: 24px;
|
||||
height: 24px;
|
||||
flex-shrink: 0;
|
||||
border: none;
|
||||
background: transparent;
|
||||
color: var(--bd-sub);
|
||||
font-size: 16px;
|
||||
line-height: 1;
|
||||
cursor: pointer;
|
||||
border-radius: 6px;
|
||||
}
|
||||
.upd-close:hover { color: var(--bd-red); background: rgba(240, 68, 56, 0.1); }
|
||||
|
||||
/* ---- 右上角：工具 / 操作进行中状态提示 ---- */
|
||||
.ts-toast {
|
||||
position: fixed;
|
||||
|
||||
+3
-1
@@ -46,7 +46,9 @@ export function connectMqtt() {
|
||||
client.on('connect', () => {
|
||||
status = 'connected';
|
||||
notifyStatus();
|
||||
client.subscribe([MQTT_TOPIC_COMMAND, MQTT_TOPIC_TICK], { qos: 1 });
|
||||
// 客户端只监听自己唯一的频道 opc/display/command/<clientId>（+ 数据 tick）。
|
||||
// 不再订阅全局命令频道 → 后端按客户端路由时，他屏的指令绝收不到。
|
||||
client.subscribe([MQTT_TOPIC_TICK, `${MQTT_TOPIC_COMMAND}/${clientId}`], { qos: 1 });
|
||||
});
|
||||
client.on('reconnect', () => {
|
||||
status = 'connecting';
|
||||
|
||||
@@ -0,0 +1,52 @@
|
||||
/* =========================================================
|
||||
客户端更新检测与安装（仅 Windows）
|
||||
后端 POST /api/update/check 返回最新版本 + 安装包下载地址；
|
||||
前端 fetch 下载安装包字节 → invoke install_update（写盘 → 启动安装程序 → 退出应用）。
|
||||
========================================================= */
|
||||
import { getApiBase } from '../config';
|
||||
import { invoke } from '@tauri-apps/api/core';
|
||||
|
||||
/** 读取真实安装的应用版本（Tauri getVersion = package.json/tauri.conf.json version）；失败时兜底。 */
|
||||
async function getCurrentVersion() {
|
||||
try {
|
||||
const { getVersion } = await import('@tauri-apps/api/app');
|
||||
const v = await getVersion();
|
||||
return v || '0.1.0';
|
||||
} catch {
|
||||
return '0.1.0';
|
||||
}
|
||||
}
|
||||
|
||||
export async function checkForUpdate() {
|
||||
try {
|
||||
const current = await getCurrentVersion();
|
||||
const res = await fetch(`${getApiBase()}/api/update/check`, {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ current }),
|
||||
});
|
||||
const data = await res.json();
|
||||
if (data && data.ok) {
|
||||
return {
|
||||
current,
|
||||
latest: data.latest_version,
|
||||
available: !!data.update_available,
|
||||
url: `${getApiBase()}${data.download_url || '/download'}`,
|
||||
name: data.installer_name || 'dpm_update_setup.exe',
|
||||
};
|
||||
}
|
||||
} catch { /* 后端不可达时忽略 */ }
|
||||
return null;
|
||||
}
|
||||
|
||||
/** 下载安装包并交给后端安装（写盘 → 启动安装程序 → 退出应用）。 */
|
||||
export async function downloadAndInstall(url, filename) {
|
||||
const res = await fetch(url);
|
||||
if (!res.ok) throw new Error(`下载失败（HTTP ${res.status}）`);
|
||||
const buf = new Uint8Array(await res.arrayBuffer());
|
||||
if (buf.length < 1000) throw new Error('安装包内容无效');
|
||||
return invoke('install_update', {
|
||||
bytes: Array.from(buf),
|
||||
filename: filename || 'dpm_update_setup.exe',
|
||||
});
|
||||
}
|
||||
@@ -1,10 +1,21 @@
|
||||
import { useEffect } from 'react';
|
||||
import { useNavigate, useLocation } from 'react-router-dom';
|
||||
import { onMqttMessage, connectMqtt, publishMqtt, getClientId, useMqttStatus } from './mqtt';
|
||||
import { MQTT_TOPIC_HEARTBEAT } from '../config';
|
||||
import { MQTT_TOPIC_HEARTBEAT, MQTT_TOPIC_COMMAND, getApiBase as API_BASE } from '../config';
|
||||
import { Sfx } from './sounds';
|
||||
import { PAGE_ORDER } from './pageNav';
|
||||
import { aiStatus } from './statusBus';
|
||||
import { invoke } from '@tauri-apps/api/core';
|
||||
import { getCurrentWindow } from '@tauri-apps/api/window';
|
||||
|
||||
// 当前窗口标识：主屏 'main' / 双屏副窗 'secondary'
|
||||
// 优先用 Rust 注入的 __DPM_WINDOW_LABEL__（副窗初始化脚本），再回退 getCurrentWindow().label
|
||||
const WINDOW_LABEL = (() => {
|
||||
try {
|
||||
if (window.__DPM_WINDOW_LABEL__) return window.__DPM_WINDOW_LABEL__;
|
||||
return getCurrentWindow().label || 'main';
|
||||
} catch { return 'main'; }
|
||||
})();
|
||||
|
||||
/* =========================================================
|
||||
MQTT 控制 —— 后端通过 opc/display/command 控制大屏
|
||||
@@ -35,6 +46,20 @@ export function useMqttControl() {
|
||||
const location = useLocation();
|
||||
const mqttStatus = useMqttStatus();
|
||||
|
||||
// 设备注册：MQTT 连接后向后端上报「唯一 device_id + 角色 + 从属主屏」，供后端精准控制
|
||||
useEffect(() => {
|
||||
if (mqttStatus !== 'connected' || !getClientId()) return;
|
||||
fetch(`${API_BASE()}/api/display/register`, {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({
|
||||
device_id: getClientId(),
|
||||
role: WINDOW_LABEL,
|
||||
parent_id: window.__DPM_PARENT_ID__ || '', // 副屏标识从属的主屏 device_id
|
||||
}),
|
||||
}).catch(() => {});
|
||||
}, [mqttStatus]);
|
||||
|
||||
// 大屏在线心跳（后端据此统计在线大屏数）
|
||||
useEffect(() => {
|
||||
connectMqtt();
|
||||
@@ -68,7 +93,15 @@ export function useMqttControl() {
|
||||
useEffect(() => {
|
||||
connectMqtt();
|
||||
const off = onMqttMessage((topic, cmd) => {
|
||||
if (topic !== 'opc/display/command' || !cmd || !cmd.action) return;
|
||||
// 接收全局控制 topic 或本客户端独有 topic（后端按屏幕终端分别下发）
|
||||
const isCmd = topic === MQTT_TOPIC_COMMAND || topic.startsWith(`${MQTT_TOPIC_COMMAND}/`);
|
||||
if (!isCmd || !cmd || !cmd.action) return;
|
||||
// 屏幕终端精准控制：后端按 client_id 路由到独有 topic。
|
||||
// 兜底：命令若带 screen_id，只有本客户端 client_id 匹配才处理（防误收全局）。
|
||||
const sc = cmd.screen;
|
||||
console.warn(`[dual] 命令 action=${cmd.action} screen_id=${cmd.screen_id || '(无)'} screen_role=${cmd.screen_role || '(无)'} 本窗口=${WINDOW_LABEL} 本client=${getClientId()}`);
|
||||
if (cmd.screen_id && cmd.screen_id !== getClientId()) return;
|
||||
if (sc && sc !== 'both' && sc !== WINDOW_LABEL) return;
|
||||
|
||||
switch (cmd.action) {
|
||||
case 'navigate': {
|
||||
@@ -86,6 +119,17 @@ export function useMqttControl() {
|
||||
}
|
||||
break;
|
||||
}
|
||||
case 'dual_screen':
|
||||
// 双屏：仅主窗口执行（只有主屏能开启/关闭副屏），把主屏 device_id 传给副窗作 parent_id
|
||||
if (WINDOW_LABEL !== 'main') break;
|
||||
aiStatus(cmd.params?.on === false ? '正在关闭双屏' : '正在启动双屏', { icon: 'page' });
|
||||
invoke('set_dual_screen', { on: cmd.params?.on !== false, parentId: getClientId() })
|
||||
.catch((err) => {
|
||||
const msg = String((err && err.message) || err || '操作失败');
|
||||
aiStatus(`双屏失败：${msg}`, { state: 'error', icon: 'error' });
|
||||
console.error('[dual]', err);
|
||||
});
|
||||
break;
|
||||
case 'alert':
|
||||
// 通知本身即展示，无需再弹「正在发送通知」状态提示
|
||||
window.dispatchEvent(new CustomEvent('dpm:alert', { detail: cmd.params || {} }));
|
||||
|
||||
Reference in New Issue
Block a user