Add TTS markdown sanitization and expand deployment docs.
Strip Markdown and stage directions before ChatTTS synthesis with chunked long scripts; document model pre-download, server-update, and microphone HTTPS notes. Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
@@ -9,7 +9,7 @@
|
||||
|
||||
## 目录
|
||||
|
||||
0. [**一键部署(推荐)**](#0-一键部署推荐)
|
||||
0. [**一键部署(推荐)**](#0-一键部署推荐) — 含 [模型预下载](#08-ai-模型预下载内网服务器必做)、[服务器更新](#042-代码推送后的服务器更新推荐)、[手机麦克风](#09-手机找不到麦克风)
|
||||
1. [硬件与系统前提](#1-硬件与系统前提)
|
||||
2. [3060 Ti 120W 功耗墙配置](#2-3060-ti-120w-功耗墙配置)
|
||||
3. [NVIDIA 驱动与 CUDA](#3-nvidia-驱动与-cuda)
|
||||
@@ -96,6 +96,8 @@ bash deploy.sh
|
||||
http://<服务器局域网IP>:5683
|
||||
```
|
||||
|
||||
> **重要:首次部署后必须预下载 AI 模型**(Whisper + ChatTTS)。内网服务器无法访问 HuggingFace / GitHub 时,不执行此步会在 Web UI 报 `Network is unreachable` 或 `Read timed out`。详见 [0.8 AI 模型预下载](#08-ai-模型预下载内网服务器必做)。
|
||||
|
||||
### 0.3 脚本命令速查
|
||||
|
||||
```bash
|
||||
@@ -123,6 +125,45 @@ bash deploy.sh update
|
||||
> **git pull 报本地修改冲突?** 新版 `deploy.sh` 会自动 `stash` 后同步;若仍失败可手动:
|
||||
> `git fetch origin && git reset --hard origin/main`
|
||||
|
||||
### 0.4.2 代码推送后的服务器更新(推荐)
|
||||
|
||||
本地开发机 `git push` 到远端后,在 **Ubuntu 服务器**上同步并重启:
|
||||
|
||||
```bash
|
||||
cd /opt/Trading_Studio
|
||||
bash server-update.sh
|
||||
```
|
||||
|
||||
`server-update.sh` 会执行:
|
||||
|
||||
1. `git fetch origin main`
|
||||
2. `git reset --hard origin/main`(覆盖 CRLF 等幽灵改动;**Ollama 地址请写在 `.env`,勿改 `config.py`**)
|
||||
3. `pm2 restart trading_studio`
|
||||
|
||||
若本次更新涉及 **Whisper / ChatTTS 离线加载**(首次部署或新增模型脚本),还需预下载模型:
|
||||
|
||||
```bash
|
||||
cd /opt/Trading_Studio
|
||||
bash server-update.sh
|
||||
bash scripts/download_all_models.sh
|
||||
pm2 restart trading_studio
|
||||
```
|
||||
|
||||
| 脚本 | 作用 |
|
||||
|------|------|
|
||||
| `bash server-update.sh` | 强制与远端 `main` 同步 + PM2 重启 |
|
||||
| `bash scripts/download_all_models.sh` | 一次性下载 Whisper (small) + ChatTTS |
|
||||
| `bash scripts/download_whisper_models.sh small` | 仅下载 Whisper |
|
||||
| `bash scripts/download_chattts_models.sh` | 仅下载 ChatTTS |
|
||||
|
||||
验证 Whisper 是否就绪:
|
||||
|
||||
```bash
|
||||
ls -lh /opt/Trading_Studio/models/whisper/small/model.bin
|
||||
```
|
||||
|
||||
应看到约 **500MB** 的 `model.bin` 文件。
|
||||
|
||||
### 0.4.1 pip / PyTorch 下载超时
|
||||
|
||||
PyTorch + triton 约 2-3GB,国内网络默认启用清华镜像,并延长超时到 600 秒:
|
||||
@@ -161,6 +202,72 @@ SKIP_PYTORCH=1 bash deploy.sh deps
|
||||
4. 反代须透传 **WebSocket**(Gradio 必需)
|
||||
5. 用户通过 `https://你的域名` 访问后再安装 App
|
||||
|
||||
### 0.8 AI 模型预下载(内网服务器必做)
|
||||
|
||||
Trading Studio 的 **Whisper 语音识别** 与 **ChatTTS 音色合成** 均需在服务器本地存放模型文件。
|
||||
内网物理机通常无法访问 `huggingface.co` / `github.com`,若未预下载,Web UI 会出现:
|
||||
|
||||
| 模块 | 典型报错 |
|
||||
|------|----------|
|
||||
| Whisper | `Network is unreachable` / `ConnectError` |
|
||||
| ChatTTS | `Read timed out` / `github.com` 连接失败 |
|
||||
|
||||
**推荐:一键下载全部模型**
|
||||
|
||||
```bash
|
||||
cd /opt/Trading_Studio
|
||||
bash scripts/download_all_models.sh
|
||||
pm2 restart trading_studio
|
||||
```
|
||||
|
||||
**分步下载(可选)**
|
||||
|
||||
```bash
|
||||
# Whisper small(约 500MB,识别默认模型)
|
||||
bash scripts/download_whisper_models.sh small
|
||||
|
||||
# ChatTTS(约 1–2GB,音色锁定与合成必需)
|
||||
bash scripts/download_chattts_models.sh
|
||||
|
||||
pm2 restart trading_studio
|
||||
```
|
||||
|
||||
**模型落盘路径**
|
||||
|
||||
| 模型 | 目录 | 关键文件 |
|
||||
|------|------|----------|
|
||||
| Whisper `small` | `/opt/Trading_Studio/models/whisper/small/` | `model.bin` |
|
||||
| ChatTTS | `/opt/Trading_Studio/models/ChatTTS/` | `asset/` 等 |
|
||||
| HF 缓存 | `/opt/Trading_Studio/models/hf_cache/` | 下载中间缓存 |
|
||||
|
||||
**`.env` 可选配置**(复制 `.env.example` → `.env`):
|
||||
|
||||
```ini
|
||||
HF_ENDPOINT=https://hf-mirror.com
|
||||
WHISPER_MODEL_DIR=/opt/Trading_Studio/models/whisper
|
||||
WHISPER_MODEL_SIZE=small
|
||||
CHATTTS_MODEL_DIR=/opt/Trading_Studio/models/ChatTTS
|
||||
```
|
||||
|
||||
国内服务器建议保留 `HF_ENDPOINT=https://hf-mirror.com`(脚本与 `whisper_service.py` 均会读取)。
|
||||
|
||||
### 0.9 手机「找不到麦克风」
|
||||
|
||||
通过 `http://192.168.x.x:5683` 内网 HTTP 访问时,手机浏览器会显示 **「找不到麦克风」** 或 **「检测不到麦克风」**。
|
||||
这是浏览器安全策略:`getUserMedia`(麦克风)**仅在 HTTPS 或 localhost 下可用**,不是程序 bug。
|
||||
|
||||
| 访问方式 | 电脑录音 | 手机录音 |
|
||||
|----------|----------|----------|
|
||||
| `http://内网IP:5683` | 可能可用 | ❌ 不可用 |
|
||||
| `https://域名`(NPS + 云反代) | ✅ | ✅ |
|
||||
|
||||
**解决办法(任选其一):**
|
||||
|
||||
1. 按 [PWA_NPS.md](./PWA_NPS.md) 配置 **NPS 穿透 + 云服务器 HTTPS 域名**,用手机访问 `https://你的域名`
|
||||
2. 在 HTTP 内网环境下,使用音频区域的 **「上传」** 标签,上传手机「语音备忘录」导出的 `.m4a` / `.wav`(与现场录音效果相同)
|
||||
|
||||
Whisper 离线模型就绪后,**上传音频文件** 可正常识别;麦克风实时录音需 HTTPS。
|
||||
|
||||
---
|
||||
|
||||
### 0.5 PM2 运维(root 环境)
|
||||
@@ -181,11 +288,21 @@ tail -f /opt/Trading_Studio/logs/pm2-out.log
|
||||
```
|
||||
/opt/Trading_Studio/
|
||||
├── deploy.sh # 一键部署脚本
|
||||
├── server-update.sh # 强制同步远端 + PM2 重启
|
||||
├── app.py # Gradio 主入口
|
||||
├── venv/ # Python 虚拟环境
|
||||
├── scripts/
|
||||
│ ├── download_all_models.sh # Whisper + ChatTTS 一键下载
|
||||
│ ├── download_whisper_models.sh # Whisper 预下载(HF 镜像)
|
||||
│ └── download_chattts_models.sh # ChatTTS 预下载(HF 镜像)
|
||||
├── models/ # AI 模型(预下载脚本写入,不入 Git)
|
||||
│ ├── whisper/small/ # Faster-Whisper(含 model.bin)
|
||||
│ ├── ChatTTS/ # ChatTTS 权重
|
||||
│ └── hf_cache/ # HuggingFace 缓存
|
||||
├── logs/ # PM2 日志
|
||||
├── uploads/ # 上传临时文件
|
||||
├── outputs/ # 合成 wav 输出
|
||||
├── .env # 服务器本地配置(Ollama IP 等,不入 Git)
|
||||
├── speaker_emb.pt # 音色文件(Web UI 生成,需手动备份)
|
||||
└── trading_studio.log # 应用日志
|
||||
```
|
||||
@@ -363,18 +480,24 @@ pip install -r requirements.txt
|
||||
|
||||
### 6.1 Faster-Whisper(必须预下载)
|
||||
|
||||
随 `requirements.txt` 安装。**内网服务器无法访问 HuggingFace 时会报 `Network is unreachable`。**
|
||||
随 `requirements.txt` 安装。`whisper_service.py` **优先从本地目录加载**,未预下载时会尝试在线拉取 HuggingFace 模型。
|
||||
|
||||
部署后执行(推荐与 ChatTTS 一起):
|
||||
内网服务器无法访问外网时会报:
|
||||
|
||||
```
|
||||
Whisper 模型加载失败: Network is unreachable
|
||||
```
|
||||
|
||||
**处理:** 见 [0.8 AI 模型预下载](#08-ai-模型预下载内网服务器必做),或执行:
|
||||
|
||||
```bash
|
||||
cd /opt/Trading_Studio
|
||||
bash scripts/download_all_models.sh
|
||||
# 或仅 Whisper: bash scripts/download_whisper_models.sh small
|
||||
bash scripts/download_whisper_models.sh small
|
||||
pm2 restart trading_studio
|
||||
```
|
||||
|
||||
模型目录:`/opt/Trading_Studio/models/whisper/small/`(含 `model.bin`)。
|
||||
本地路径:`/opt/Trading_Studio/models/whisper/small/model.bin`(约 500MB)。
|
||||
|
||||
可选环境变量(`.env`):`WHISPER_MODEL_DIR`、`WHISPER_MODEL_SIZE`、`HF_ENDPOINT`。
|
||||
|
||||
### 6.2 ChatTTS(必须预下载,勿依赖 GitHub)
|
||||
|
||||
@@ -384,27 +507,19 @@ pm2 restart trading_studio
|
||||
pip install ChatTTS @ git+https://github.com/2noise/ChatTTS.git
|
||||
```
|
||||
|
||||
**重要:** 默认 `chat.load()` 会访问 **github.com** 下载 asset,国内服务器常报 `Read timed out (3)`。
|
||||
部署后**必须**执行预下载脚本(走 HuggingFace 镜像):
|
||||
**重要:** 默认 `chat.load()` 会访问 **github.com** 下载 asset,国内/内网服务器常报 `Read timed out`。
|
||||
`tts_service.py` 已支持从 `models/ChatTTS` 离线加载,部署后**必须**预下载:
|
||||
|
||||
```bash
|
||||
cd /opt/Trading_Studio
|
||||
source venv/bin/activate
|
||||
bash scripts/download_chattts_models.sh
|
||||
pm2 restart trading_studio
|
||||
```
|
||||
|
||||
模型保存至 `/opt/Trading_Studio/models/ChatTTS`(约 1–2GB,不入 Git)。
|
||||
|
||||
`.env` 可自定义:
|
||||
|
||||
```ini
|
||||
HF_ENDPOINT=https://hf-mirror.com
|
||||
CHATTTS_MODEL_DIR=/opt/Trading_Studio/models/ChatTTS
|
||||
```
|
||||
|
||||
下载完成后再在 Web UI 点击「锁定音色」。
|
||||
|
||||
一键下载 Whisper + ChatTTS:`bash scripts/download_all_models.sh`
|
||||
|
||||
### 6.3 Gradio
|
||||
|
||||
```bash
|
||||
@@ -481,9 +596,11 @@ http://<本机局域网IP>:5683
|
||||
|
||||
### 8.1 验证清单
|
||||
|
||||
- [ ] `models/whisper/small/model.bin` 存在(`bash scripts/download_whisper_models.sh small`)
|
||||
- [ ] `models/ChatTTS/` 已预下载(`bash scripts/download_chattts_models.sh`)
|
||||
- [ ] 页面加载,Ollama 状态显示在线
|
||||
- [ ] 上传 10-30s 参考人声 → 音色锁定成功,生成 `speaker_emb.pt`
|
||||
- [ ] 上传复盘录音 → Whisper 识别出中文文本
|
||||
- [ ] 上传复盘录音 → Whisper 识别出中文文本(无需外网)
|
||||
- [ ] 点击润色 → 返回 Gemma4 处理后的文稿
|
||||
- [ ] 点击合成 → `outputs/` 下生成 24kHz wav
|
||||
|
||||
@@ -621,7 +738,24 @@ Whisper 与 ChatTTS 不会同时常驻最大显存,但首次加载模型时峰
|
||||
- 锁定 120W 功耗墙
|
||||
- `max_memory_restart: "6G"` 已在 PM2 配置中设置
|
||||
|
||||
### 10.3 Whisper CUDA 报错
|
||||
### 10.3 Whisper 模型加载失败
|
||||
|
||||
#### A. `Network is unreachable` / `ConnectError`(内网无外网)
|
||||
|
||||
**原因:** 未预下载 Whisper 模型,程序尝试访问 HuggingFace Hub 失败。
|
||||
|
||||
**处理:**
|
||||
|
||||
```bash
|
||||
cd /opt/Trading_Studio
|
||||
bash scripts/download_whisper_models.sh small
|
||||
ls -lh models/whisper/small/model.bin # 确认约 500MB
|
||||
pm2 restart trading_studio
|
||||
```
|
||||
|
||||
若服务器可访问外网但 HuggingFace 慢,在 `.env` 中设置 `HF_ENDPOINT=https://hf-mirror.com` 后重试下载。
|
||||
|
||||
#### B. CUDA / 显存报错
|
||||
|
||||
```
|
||||
错误: CUDA initialization failed / out of memory
|
||||
@@ -631,7 +765,7 @@ Whisper 与 ChatTTS 不会同时常驻最大显存,但首次加载模型时峰
|
||||
|
||||
1. 重启 PM2 进程释放显存
|
||||
2. 确认 `compute_type="float16"`(已在 config.py 配置)
|
||||
3. 降级模型为 `base`(修改 `config.py` 中 `WHISPER_MODEL_SIZE`)
|
||||
3. 在 `.env` 中降级模型:`WHISPER_MODEL_SIZE=base`,并执行 `bash scripts/download_whisper_models.sh base`
|
||||
|
||||
### 10.4 Ollama 超时
|
||||
|
||||
@@ -660,6 +794,13 @@ sudo lsof -i :5683
|
||||
ss -tlnp | grep 5683
|
||||
```
|
||||
|
||||
### 10.7 手机「找不到麦克风」
|
||||
|
||||
内网 `http://192.168.x.x:5683` 下手机无法使用实时录音,属浏览器 HTTPS 安全限制。
|
||||
完整说明与 NPS 穿透方案见 [0.9 手机「找不到麦克风」](#09-手机找不到麦克风) 与 [PWA_NPS.md](./PWA_NPS.md) 第九节。
|
||||
|
||||
**临时方案:** Web UI 音频区域使用 **「上传」** 导入录音文件,Whisper 识别流程相同。
|
||||
|
||||
---
|
||||
|
||||
## 附录:防火墙(本机 Gradio)
|
||||
@@ -675,16 +816,30 @@ sudo ufw reload
|
||||
|
||||
---
|
||||
|
||||
## 附录:config.py 关键常量速查
|
||||
## 附录:config.py / .env 关键配置速查
|
||||
|
||||
**服务器本地覆盖请用 `.env`**(`cp .env.example .env`),避免 `git pull` 冲突:
|
||||
|
||||
```ini
|
||||
OLLAMA_HOST=192.168.8.64
|
||||
OLLAMA_PORT=11434
|
||||
HF_ENDPOINT=https://hf-mirror.com
|
||||
WHISPER_MODEL_DIR=/opt/Trading_Studio/models/whisper
|
||||
WHISPER_MODEL_SIZE=small
|
||||
CHATTTS_MODEL_DIR=/opt/Trading_Studio/models/ChatTTS
|
||||
```
|
||||
|
||||
`config.py` 默认值(可被 `.env` 覆盖):
|
||||
|
||||
```python
|
||||
HOST = "0.0.0.0"
|
||||
PORT = 5683
|
||||
OLLAMA_URL = "http://192.168.8.64:11434/api/chat"
|
||||
MODEL_NAME = "huihui_ai/gemma-4-abliterated:e4b"
|
||||
WHISPER_MODEL_SIZE = "small"
|
||||
WHISPER_MODEL_SIZE = "small" # .env: WHISPER_MODEL_SIZE
|
||||
WHISPER_MODEL_DIR = "models/whisper" # .env: WHISPER_MODEL_DIR
|
||||
WHISPER_DEVICE = "cuda"
|
||||
WHISPER_COMPUTE_TYPE = "float16"
|
||||
CHATTTS_MODEL_DIR = "models/ChatTTS"
|
||||
HF_ENDPOINT = "https://hf-mirror.com"
|
||||
SPEAKER_EMB_PATH = "speaker_emb.pt"
|
||||
TTS_SAMPLE_RATE = 24000
|
||||
```
|
||||
|
||||
+3
-2
@@ -220,6 +220,7 @@ Trading Studio 应用层也会发送该头;若反代覆盖了响应头,需
|
||||
|
||||
## 相关文档
|
||||
|
||||
- 内网部署:`DEPLOY.md`
|
||||
- 服务器更新:`bash server-update.sh`
|
||||
- 内网部署与模型预下载:[DEPLOY.md](./DEPLOY.md)(§0.4.2 服务器更新、§0.8 模型预下载、§0.9 麦克风)
|
||||
- 服务器快速更新:`bash server-update.sh`(同步远端 `main` + PM2 重启)
|
||||
- 首次部署后下载模型:`bash scripts/download_all_models.sh`
|
||||
- 麦克风问题:见上文 **第九节**
|
||||
|
||||
@@ -66,12 +66,30 @@ bash deploy.sh
|
||||
|
||||
浏览器访问:`http://<服务器IP>:5683`
|
||||
|
||||
日常更新:
|
||||
**首次部署后必做 — 预下载 AI 模型**(内网服务器无外网时必需,否则 Whisper 报 `Network is unreachable`):
|
||||
|
||||
```bash
|
||||
cd /opt/Trading_Studio && bash deploy.sh update
|
||||
cd /opt/Trading_Studio
|
||||
bash scripts/download_all_models.sh
|
||||
pm2 restart trading_studio
|
||||
```
|
||||
|
||||
日常更新(代码已 `git push` 到远端后,在服务器执行):
|
||||
|
||||
```bash
|
||||
cd /opt/Trading_Studio
|
||||
bash server-update.sh
|
||||
```
|
||||
|
||||
若更新涉及模型脚本或首次部署,追加:
|
||||
|
||||
```bash
|
||||
bash scripts/download_all_models.sh
|
||||
pm2 restart trading_studio
|
||||
```
|
||||
|
||||
完整说明见 [DEPLOY.md §0.4.2 / §0.8](./DEPLOY.md)。
|
||||
|
||||
### 手动部署(开发调试)
|
||||
|
||||
```bash
|
||||
@@ -116,7 +134,9 @@ python app.py
|
||||
| 中控端口 | `5683`(`0.0.0.0` 局域网可访问) |
|
||||
| Ollama 地址 | `http://192.168.8.64:11434` |
|
||||
| 模型名称 | `huihui_ai/gemma-4-abliterated:e4b` |
|
||||
| Whisper 模型 | `small` / CUDA / float16 |
|
||||
| Whisper 模型 | `small` / CUDA / float16,本地路径 `models/whisper/small/` |
|
||||
| ChatTTS 模型 | `models/ChatTTS/`(须预下载脚本) |
|
||||
| HF 镜像 | `HF_ENDPOINT=https://hf-mirror.com`(`.env` 可改) |
|
||||
| 音色文件 | `speaker_emb.pt` |
|
||||
| 音频输出 | `outputs/` 目录 |
|
||||
|
||||
@@ -170,16 +190,25 @@ outputs/
|
||||
```
|
||||
Trading_Studio/
|
||||
├── deploy.sh # 一键部署脚本(/opt + PM2)
|
||||
├── server-update.sh # 强制同步远端 main + PM2 重启
|
||||
├── app.py # Gradio 主入口
|
||||
├── config.py # 全局配置
|
||||
├── whisper_service.py # Whisper CUDA 识别
|
||||
├── config.py # 全局配置(Ollama 等请用 .env 覆盖)
|
||||
├── whisper_service.py # Whisper CUDA 识别(优先本地模型)
|
||||
├── llm_service.py # Ollama 远程润色
|
||||
├── tts_service.py # ChatTTS 音色与合成
|
||||
├── tts_service.py # ChatTTS 音色与合成(优先本地模型)
|
||||
├── scripts/
|
||||
│ ├── download_all_models.sh # Whisper + ChatTTS 一键下载
|
||||
│ ├── download_whisper_models.sh
|
||||
│ └── download_chattts_models.sh
|
||||
├── models/ # AI 模型(预下载,不入 Git)
|
||||
│ ├── whisper/small/
|
||||
│ └── ChatTTS/
|
||||
├── ecosystem.config.js # PM2 守护配置
|
||||
├── requirements.txt # Python 依赖
|
||||
├── .env.example # 服务器本地配置模板 → 复制为 .env
|
||||
├── README.md # 本文件
|
||||
├── DEPLOY.md # 部署指南(含一键部署教程)
|
||||
├── PWA_NPS.md # 云服务器反代 + NPS 穿透 + PWA 安装教程
|
||||
├── DEPLOY.md # 部署指南(含模型预下载、故障排查)
|
||||
├── PWA_NPS.md # HTTPS / NPS 穿透 / 手机麦克风教程
|
||||
├── .gitignore
|
||||
├── speaker_emb.pt # 音色文件(运行时生成,不入库)
|
||||
├── uploads/ # 上传临时目录
|
||||
@@ -201,11 +230,20 @@ Trading_Studio/
|
||||
|
||||
## 常见问题
|
||||
|
||||
**Q: Whisper 报 `Network is unreachable`?**
|
||||
A: 内网服务器无法访问 HuggingFace。执行 `bash scripts/download_whisper_models.sh small`,确认 `models/whisper/small/model.bin` 存在后 `pm2 restart trading_studio`。详见 [DEPLOY.md §0.8](./DEPLOY.md)。
|
||||
|
||||
**Q: Whisper 报 CUDA 错误?**
|
||||
A: 确认 `nvidia-smi` 正常,且未同时运行其他占显存任务。Whisper 使用 `float16` 已针对 8GB 优化。
|
||||
A: 确认 `nvidia-smi` 正常,且未同时运行其他占显存任务。Whisper 使用 `float16` 已针对 8GB 优化。可在 `.env` 设置 `WHISPER_MODEL_SIZE=base` 并重新下载。
|
||||
|
||||
**Q: ChatTTS 报 GitHub / 下载超时?**
|
||||
A: 执行 `bash scripts/download_chattts_models.sh`,或一键 `bash scripts/download_all_models.sh`。
|
||||
|
||||
**Q: Ollama 连接失败?**
|
||||
A: 在服务器上执行 `curl http://192.168.8.64:11434/api/tags` 验证连通性,确认模型已 `ollama pull`。
|
||||
A: 在服务器上执行 `curl http://192.168.8.64:11434/api/tags` 验证连通性,确认模型已 `ollama pull`。Ollama IP 写在 `.env` 的 `OLLAMA_HOST`。
|
||||
|
||||
**Q: 手机显示「找不到麦克风」?**
|
||||
A: `http://内网IP:5683` 非 HTTPS,浏览器禁用麦克风。请按 [PWA_NPS.md](./PWA_NPS.md) 配置 HTTPS 域名,或改用 Web UI **「上传」** 录音文件。
|
||||
|
||||
**Q: TTS 音色不稳定?**
|
||||
A: 重新锁定音色,填写参考音频精确转写,并保持 `temperature=0.3` 低随机性。
|
||||
|
||||
@@ -940,8 +940,13 @@ def build_app() -> gr.Blocks:
|
||||
|
||||
with gr.Column(scale=1):
|
||||
gr.Markdown("### Step 3 · ChatTTS 配音合成")
|
||||
gr.Markdown(
|
||||
"> 合成前会自动去掉 **Markdown**(`#`、`**`)、emoji、"
|
||||
"舞台提示(如前奏/转场)和文末「修改笔记」。"
|
||||
"也可手动删成纯口语文本再点合成。"
|
||||
)
|
||||
polished_text = gr.Textbox(
|
||||
label="润色配音稿(可编辑)",
|
||||
label="润色配音稿(可编辑,支持含 Markdown,合成时自动清洗)",
|
||||
lines=10,
|
||||
placeholder="润色结果将显示在此...",
|
||||
)
|
||||
|
||||
@@ -74,6 +74,11 @@ SYSTEM_PROMPT = (
|
||||
"如果原视频中有由于心态不好、违背交易纪律(如手贱乱开仓、提前平仓)"
|
||||
"导致少赚或亏损的部分,请用冷酷、严厉的语气狠狠地自我吐槽、反思该点。"
|
||||
"去掉所有无意义的口头禅,字数不做删减。"
|
||||
"【输出格式硬性要求】"
|
||||
"只输出可直接朗读的纯文本正文,不要 Markdown(禁止 #、**、---、列表符号、emoji)。"
|
||||
"不要写舞台提示(如前奏、转场、BGM、语气说明等括号备注)。"
|
||||
"不要写「以下是润色后的文案」等前言,也不要写修改笔记或点评。"
|
||||
"可用《》作为标题,正文按自然段换行即可。"
|
||||
)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -131,6 +136,9 @@ TTS_TOP_P = 0.7
|
||||
TTS_TOP_K = 20
|
||||
TTS_SPEED_PROMPT = "[speed_5]"
|
||||
|
||||
# 单段 TTS 最大字数(超长稿按句切分后逐段合成再拼接)
|
||||
TTS_MAX_CHARS_PER_CHUNK = _env_int("TTS_MAX_CHARS_PER_CHUNK", 280)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 上传临时文件目录
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
+158
-6
@@ -8,12 +8,13 @@ from __future__ import annotations
|
||||
import inspect
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
import traceback
|
||||
import uuid
|
||||
import warnings
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import Any, Dict, Optional, Tuple
|
||||
from typing import Any, Dict, List, Optional, Tuple
|
||||
|
||||
import numpy as np
|
||||
import torch
|
||||
@@ -29,6 +30,7 @@ from config import (
|
||||
SPEAKER_EMB_PATH,
|
||||
SPEAKER_SAMPLE_MAX_SEC,
|
||||
SPEAKER_SAMPLE_MIN_SEC,
|
||||
TTS_MAX_CHARS_PER_CHUNK,
|
||||
TTS_SAMPLE_RATE,
|
||||
TTS_SPEED_PROMPT,
|
||||
TTS_TEMPERATURE,
|
||||
@@ -377,6 +379,121 @@ def speaker_is_ready() -> Tuple[bool, str]:
|
||||
return True, f"已加载固定音色: {SPEAKER_EMB_PATH}"
|
||||
|
||||
|
||||
_EMOJI_RE = re.compile(
|
||||
"["
|
||||
"\U0001F300-\U0001FAFF"
|
||||
"\U00002700-\U000027BF"
|
||||
"\U00002600-\U000026FF"
|
||||
"]+",
|
||||
flags=re.UNICODE,
|
||||
)
|
||||
|
||||
_TTS_NOTE_MARKERS = (
|
||||
"💡",
|
||||
"量化交易员的修改笔记",
|
||||
"修改笔记(供你参考)",
|
||||
"修改笔记",
|
||||
"供你参考",
|
||||
)
|
||||
|
||||
_STAGE_DIRECTION_RE = re.compile(
|
||||
r"[((][^))]{0,80}(?:前奏|转场|语气|背景|BGM|配乐|节奏|环节)[^))]{0,80}[))]"
|
||||
)
|
||||
|
||||
|
||||
def prepare_text_for_tts(text: str) -> str:
|
||||
"""
|
||||
将 LLM 润色稿转为 ChatTTS 可朗读的纯文本。
|
||||
去除 Markdown、emoji、舞台提示、修改笔记等非朗读内容。
|
||||
"""
|
||||
if not text:
|
||||
return ""
|
||||
|
||||
cleaned = text.replace("\r\n", "\n").strip()
|
||||
|
||||
for marker in _TTS_NOTE_MARKERS:
|
||||
idx = cleaned.find(marker)
|
||||
if idx >= 0:
|
||||
cleaned = cleaned[:idx]
|
||||
|
||||
# 去掉模型常见前言,从标题或正文起点开始
|
||||
for pattern in (
|
||||
r"^作为一名极其严谨的量化交易员.*?配音稿。\s*",
|
||||
r"^以下是为你润色后的文案[::]*\s*",
|
||||
r"^以下(?:是|为).*?润色.*?文案[::]*\s*",
|
||||
):
|
||||
cleaned = re.sub(pattern, "", cleaned, count=1, flags=re.DOTALL)
|
||||
|
||||
cleaned = re.sub(r"^\*{3,}\s*$", "", cleaned, flags=re.MULTILINE)
|
||||
cleaned = re.sub(r"^-{3,}\s*$", "", cleaned, flags=re.MULTILINE)
|
||||
cleaned = re.sub(r"^#{1,6}\s*", "", cleaned, flags=re.MULTILINE)
|
||||
cleaned = re.sub(r"\*\*([^*\n]+)\*\*", r"\1", cleaned)
|
||||
cleaned = re.sub(r"\*([^*\n]+)\*", r"\1", cleaned)
|
||||
cleaned = re.sub(r"__([^_\n]+)__", r"\1", cleaned)
|
||||
cleaned = _STAGE_DIRECTION_RE.sub("", cleaned)
|
||||
cleaned = _EMOJI_RE.sub("", cleaned)
|
||||
cleaned = re.sub(r"^\d+\.\s*", "", cleaned, flags=re.MULTILINE)
|
||||
cleaned = re.sub(r"^[-*]\s+", "", cleaned, flags=re.MULTILINE)
|
||||
cleaned = re.sub(r"[ \t]+\n", "\n", cleaned)
|
||||
cleaned = re.sub(r"\n{3,}", "\n\n", cleaned)
|
||||
|
||||
lines = [ln.strip() for ln in cleaned.split("\n")]
|
||||
lines = [ln for ln in lines if ln and not re.fullmatch(r"[*\-#]+", ln)]
|
||||
return "\n".join(lines).strip()
|
||||
|
||||
|
||||
def split_text_for_tts(text: str, max_chars: int = TTS_MAX_CHARS_PER_CHUNK) -> List[str]:
|
||||
"""按句号/换行切分长稿,避免 ChatTTS 单段过长失败。"""
|
||||
text = text.strip()
|
||||
if not text:
|
||||
return []
|
||||
if len(text) <= max_chars:
|
||||
return [text]
|
||||
|
||||
parts = re.split(r"(?<=[。!?!?;;])\s*|\n+", text)
|
||||
chunks: List[str] = []
|
||||
buf = ""
|
||||
|
||||
for part in parts:
|
||||
part = part.strip()
|
||||
if not part:
|
||||
continue
|
||||
candidate = f"{buf}{part}" if buf else part
|
||||
if len(candidate) <= max_chars:
|
||||
buf = candidate
|
||||
continue
|
||||
if buf:
|
||||
chunks.append(buf)
|
||||
buf = ""
|
||||
if len(part) <= max_chars:
|
||||
buf = part
|
||||
continue
|
||||
for i in range(0, len(part), max_chars):
|
||||
chunks.append(part[i : i + max_chars])
|
||||
|
||||
if buf:
|
||||
chunks.append(buf)
|
||||
|
||||
return [c.strip() for c in chunks if c.strip()]
|
||||
|
||||
|
||||
def _concat_wavs(
|
||||
wavs: List[np.ndarray],
|
||||
sample_rate: int,
|
||||
pause_sec: float = 0.35,
|
||||
) -> np.ndarray:
|
||||
if not wavs:
|
||||
return np.array([], dtype=np.float32)
|
||||
|
||||
pause = np.zeros(int(sample_rate * pause_sec), dtype=np.float32)
|
||||
segments: List[np.ndarray] = []
|
||||
for i, wav in enumerate(wavs):
|
||||
segments.append(np.asarray(wav, dtype=np.float32).flatten())
|
||||
if i < len(wavs) - 1:
|
||||
segments.append(pause)
|
||||
return np.concatenate(segments)
|
||||
|
||||
|
||||
def generate_voice(refined_text: str) -> Tuple[bool, str, Optional[str]]:
|
||||
"""
|
||||
使用 ChatTTS 将润色后的文稿合成为 wav 配音。
|
||||
@@ -401,6 +518,19 @@ def generate_voice(refined_text: str) -> Tuple[bool, str, Optional[str]]:
|
||||
try:
|
||||
import ChatTTS
|
||||
|
||||
speak_text = prepare_text_for_tts(refined_text)
|
||||
if not speak_text:
|
||||
return (
|
||||
False,
|
||||
"清洗后无有效朗读文本。请删除 Markdown(#、**)、emoji、舞台提示和「修改笔记」,"
|
||||
"只保留可念出的正文后再合成。",
|
||||
None,
|
||||
)
|
||||
|
||||
chunks = split_text_for_tts(speak_text)
|
||||
if not chunks:
|
||||
return False, "无法切分朗读文本,请检查润色稿内容。", None
|
||||
|
||||
spk_emb = payload.get("spk_emb")
|
||||
spk_smp = payload.get("spk_smp")
|
||||
txt_smp = payload.get("txt_smp", "")
|
||||
@@ -419,17 +549,35 @@ def generate_voice(refined_text: str) -> Tuple[bool, str, Optional[str]]:
|
||||
prompt="[oral_2][laugh_0][break_4]",
|
||||
)
|
||||
|
||||
logger.info(
|
||||
"TTS 合成: 原文 %d 字 → 清洗后 %d 字,分 %d 段",
|
||||
len(refined_text),
|
||||
len(speak_text),
|
||||
len(chunks),
|
||||
)
|
||||
|
||||
segment_wavs: List[np.ndarray] = []
|
||||
for idx, chunk in enumerate(chunks, start=1):
|
||||
wavs = chat.infer(
|
||||
refined_text.strip(),
|
||||
chunk,
|
||||
skip_refine_text=False,
|
||||
params_refine_text=params_refine_text,
|
||||
params_infer_code=params_infer_code,
|
||||
)
|
||||
|
||||
if not wavs or len(wavs) == 0:
|
||||
return False, "ChatTTS 未生成有效音频。", None
|
||||
return (
|
||||
False,
|
||||
f"ChatTTS 第 {idx}/{len(chunks)} 段未生成音频。"
|
||||
f"(段内容前 40 字: {chunk[:40]}…)",
|
||||
None,
|
||||
)
|
||||
segment_wavs.append(np.asarray(wavs[0], dtype=np.float32))
|
||||
|
||||
wav_array = np.asarray(wavs[0], dtype=np.float32)
|
||||
wav_array = (
|
||||
segment_wavs[0]
|
||||
if len(segment_wavs) == 1
|
||||
else _concat_wavs(segment_wavs, TTS_SAMPLE_RATE)
|
||||
)
|
||||
|
||||
peak = np.max(np.abs(wav_array)) or 1.0
|
||||
wav_int16 = (wav_array / peak * 32767).astype(np.int16)
|
||||
@@ -440,7 +588,11 @@ def generate_voice(refined_text: str) -> Tuple[bool, str, Optional[str]]:
|
||||
|
||||
wavfile.write(str(output_path), TTS_SAMPLE_RATE, wav_int16)
|
||||
|
||||
msg = f"配音合成成功: {output_path}"
|
||||
chunk_note = f",共 {len(chunks)} 段拼接" if len(chunks) > 1 else ""
|
||||
msg = (
|
||||
f"配音合成成功: {output_path}"
|
||||
f"(朗读 {len(speak_text)} 字{chunk_note})"
|
||||
)
|
||||
logger.info(msg)
|
||||
return True, msg, str(output_path)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user