2 Commits
8 changed files with 420 additions and 37 deletions
+268
View File
@@ -0,0 +1,268 @@
# LLM 速度测试台 · API 文档
> 系统所有页面功能均通过 REST API 提供,前端(网页)只是这些 API 的一个可视化客户端。
> 任何页面可见/可操作的数据都可以通过下面的接口调用或访问。
- **服务地址:** `http://<IP>:16097`
- **数据格式:** 请求/响应均为 `application/json`(导出类接口除外)
- **鉴权:** 内部工具,当前无鉴权;如需对外暴露,建议在网关/Nginx 层加访问控制
- **测试启动为异步**`POST /api/tests` 返回后,任务在后台线程执行,用 `GET /api/tests/<id>` 或日志轮询接口跟踪进度
---
## 目录
1. [健康检查](#1-健康检查)
2. [提供商配置](#2-提供商配置-configs)
3. [连接测试](#3-连接测试)
4. [速度测试](#4-速度测试-tests)
5. [测试详情与日志](#5-测试详情与日志)
6. [导出(Excel / JSON](#6-导出excel--json)
7. [数据模型](#7-数据模型)
8. [curl 使用示例](#8-curl-使用示例)
---
## 1. 健康检查
### `GET /api/health`
返回服务状态与正在运行的测试。
**响应:**
```json
{ "ok": true, "port": 16097, "running_tests": [] }
```
---
## 2. 提供商配置(Configs
### `GET /api/configs`
列出所有已保存配置(不返回 API Key 明文,仅 `has_key` 标记)。
**响应:**
```json
[
{ "id": 2, "name": "epyc-test", "provider": "openai",
"base_url": "http://121.40.164.32:18003/v1",
"model": "unsloth/Qwen3.8-27B-Q4_K_M", "temperature": 0.7, "has_key": 1 }
]
```
### `POST /api/configs`
新增配置。请求体字段:`name`(必填), `provider`, `base_url`, `api_key`, `model`, `temperature`
**请求:**
```json
{ "name": "DeepSeek", "provider": "openai",
"base_url": "https://api.deepseek.com/v1",
"api_key": "sk-xxx", "model": "deepseek-chat", "temperature": 0.7 }
```
**响应:** `{ "ok": true, "id": 3 }`
### `GET /api/configs/<id>`
获取单个配置(**含 API Key 明文**,用于前端回填;注意保管)。
### `PUT /api/configs/<id>`
更新配置,**局部更新**(只改请求里出现的字段)。
**请求:** `{ "model": "deepseek-v4-flash" }` → 响应 `{ "ok": true, "id": 3 }`
### `DELETE /api/configs/<id>`
删除配置。响应 `{ "ok": true }`
---
## 3. 连接测试
### `POST /api/configs/test`
验证 API Key / Base URL / 模型名连通性。只要流式请求成功返回(HTTP 200 + 收到响应流)即视为连通;支持推理型模型(Qwen3/DeepSeek 思维链)。
**请求:** 同配置对象(`provider`, `base_url`, `api_key`, `model`, `temperature`
**成功响应:**
```json
{
"ok": true, "total_ms": 1308.2,
"note": "",
"metrics": { "prompt_tokens": 63, "output_tokens": 32, "output_chars": 110,
"ttft_ms": 715.9, "prefill_speed": 86.7, "decode_speed": 52.7, "total_ms": 1308.2 }
}
```
> `note` 非空表示连接正常但未返回正文(可能为只输出思维链的模型)。
**失败响应:** `{ "ok": false, "error": "HTTP 401: ..." }`
---
## 4. 速度测试(Tests
### `POST /api/tests`
启动一次速度测试(异步,立即返回测试 id)。
**请求体:**
```json
{
"config": {
"provider": "openai",
"name": "Qwen3 对比",
"base_url": "http://121.40.164.32:18003/v1",
"api_key": "sk-xxx",
"model": "unsloth/Qwen3.8-27B-Q4_K_M",
"temperature": 0.7
},
"gen": {
"name": "Qwen3 不同上下文长度速度对比",
"context_lengths": [512, 2048, 8192, 32768, 131072],
"max_tokens": 128,
"samples": 2,
"warmup": true,
"avoid_cache": true
}
}
```
**gen 字段说明:**
| 字段 | 类型 | 默认 | 说明 |
|------|------|------|------|
| `name` | string | `""` | 测试名称/主题(会存入测试记录并展示在历史与详情) |
| `context_lengths` | number[] | `[512,2048,8192,32768,131072]` | 要测试的上下文长度列表,每个长度独立校准+预热+采样 |
| `max_tokens` | number | `128` | 解码输出 token 长度 |
| `samples` | number | `2` | 每个上下文长度的采样次数 |
| `warmup` | bool | `true` | 测试前空转预热(不计速度) |
| `avoid_cache` | bool | `true` | 随机前缀避免缓存命中 |
**响应:** `{ "ok": true, "id": 9 }`
### `GET /api/tests?limit=<n>`
测试历史列表(按 id 倒序)。`limit` 默认 100,最大 1000。
**响应:**
```json
[
{ "id": 9, "created_at": "2026-08-23 18:52:00", "status": "done",
"provider": "openai", "model": "unsloth/Qwen3.8-27B-Q4_K_M",
"name": "Qwen3 不同上下文长度速度对比",
"error": "",
"summary": { "samples_ok": 2, "samples_total": 2, "avg_ttft_ms": 1808.7, ... } }
]
```
### `POST /api/tests/<id>/cancel`
停止正在运行的测试。响应 `{ "ok": true, "msg": "正在停止..." }`
### `DELETE /api/tests/<id>`
删除测试及其全部采样与日志。响应 `{ "ok": true }`
---
## 5. 测试详情与日志
### `GET /api/tests/<id>`
完整测试详情:基本信息 + 配置(API Key 已打码)+ 生成参数 + 汇总 + 每次采样 + 完整日志。
**响应结构:**
```json
{
"id": 9, "created_at": "...", "status": "done",
"provider": "openai", "model": "...", "name": "...", "error": "",
"config": { "base_url": "...", "api_key": "sk-x****", ... },
"gen": { "name": "...", "context_lengths": [512, 2048], "max_tokens": 128, "samples": 1, "warmup": true, "avoid_cache": true },
"summary": {
"samples_total": 2, "samples_ok": 2,
"calibration_chars_per_token": 1.82,
"avg_ttft_ms": 1808.7, "min_ttft_ms": 1122.8, "max_ttft_ms": 2494.6,
"avg_prefill_speed": 694.2, "min_prefill_speed": 515.7, "max_prefill_speed": 872.7,
"avg_decode_speed": 54.7, "min_decode_speed": 54.4, "max_decode_speed": 55.0,
"avg_prompt_tokens": 1378.0, "avg_output_tokens": 128.0,
"avg_total_ms": 4148.4, "min_total_ms": 3449.9, "max_total_ms": 4846.9,
"by_length": {
"512": { "samples_total": 1, "samples_ok": 1, "avg_ttft_ms": 1122.8, "avg_prefill_speed": 515.7, "avg_decode_speed": 55.0, "avg_prompt_tokens": 579, "avg_output_tokens": 128, "avg_total_ms": 3449.9 },
"2048": { "samples_total": 1, "samples_ok": 1, "avg_ttft_ms": 2494.6, "avg_prefill_speed": 872.7, "avg_decode_speed": 54.4, "avg_prompt_tokens": 2177, "avg_output_tokens": 128, "avg_total_ms": 4846.9 }
}
},
"runs": [
{ "run_index": 1, "context_length": 512,
"metrics": { "prompt_tokens": 579, "output_tokens": 128, "cached_tokens": 0,
"ttft_ms": 1122.8, "prefill_speed": 515.7, "decode_speed": 55.0, "total_ms": 3449.9 },
"error": "" }
],
"logs": [ { "id": 1, "level": "INFO", "msg": "═══ 开始速度测试 ═══", "rel": 0.0, "ts": "..." } ]
}
```
### `GET /api/tests/<id>/logs?after=<id>`
增量日志(前端轮询用)。`after` 为上次取到的最大日志 id,返回其后新增日志 + 最新状态/汇总/最后采样。
**响应:**
```json
{
"status": "running", "error": "",
"summary": {},
"last_run": { "...": "..." },
"logs": [ { "id": 68, "level": "METRIC", "msg": "...", "rel": 3.21 } ],
"after": 73
}
```
---
## 6. 导出(Excel / JSON
### `GET /api/tests/<id>/export.xlsx`
导出 Excel 报告(**3 个 Sheet**:汇总 / 采样明细 / 日志),`Content-Type: application/vnd.openxmlformats-officedocument.spreadsheetml.sheet`
- **汇总**:测试信息 + 整体统计指标(平均/最大/最小)+ 按上下文长度分组
- **采样明细**:每次采样的上下文长度与全部指标
- **日志**:完整测试日志
### `GET /api/tests/<id>/export.json`
导出完整测试数据为 JSON(与 `GET /api/tests/<id>` 一致,API Key 打码),`Content-Type: application/json`
---
## 7. 数据模型
| 表 | 说明 | 关键字段 |
|----|------|----------|
| `configs` | 保存的接口配置 | id, name, provider, base_url, api_key, model, temperature |
| `tests` | 测试记录 | id, status(running/done/error/canceled), provider, model, **name**, config_json, gen_cfg_json, summary_json, error |
| `test_runs` | 每次采样 | id, test_id, run_index, **context_length**, metrics_json, error |
| `logs` | 测试日志 | id, test_id, level, msg, rel |
**summary 整体指标字段:**
`avg_/min_/max_` 前缀 × `ttft_ms` / `prefill_speed` / `decode_speed` / `total_ms`,以及 `avg_prompt_tokens` / `avg_output_tokens` / `avg_cached_tokens` / `best_ttft_ms`= min_ttft_ms)。
---
## 8. curl 使用示例
```bash
BASE=http://<IP>:16097
# 健康检查
curl $BASE/api/health
# 新增配置
curl -X POST $BASE/api/configs -H 'Content-Type: application/json' \
-d '{"name":"Qwen3","provider":"openai","base_url":"http://121.40.164.32:18003/v1","api_key":"sk-xxx","model":"unsloth/Qwen3.8-27B-Q4_K_M","temperature":0.7}'
# 测试连接
curl -X POST $BASE/api/configs/test -H 'Content-Type: application/json' \
-d '{"provider":"openai","base_url":"http://121.40.164.32:18003/v1","api_key":"sk-xxx","model":"unsloth/Qwen3.8-27B-Q4_K_M"}'
# 启动速度测试(异步)
curl -X POST $BASE/api/tests -H 'Content-Type: application/json' -d '{
"config": {"provider":"openai","base_url":"http://121.40.164.32:18003/v1","api_key":"sk-xxx","model":"unsloth/Qwen3.8-27B-Q4_K_M"},
"gen": {"name":"各长度对比","context_lengths":[512,2048,8192],"max_tokens":128,"samples":2,"warmup":true,"avoid_cache":true}
}'
# 查询测试列表 / 详情
curl "$BASE/api/tests?limit=10"
curl $BASE/api/tests/9
# 导出
curl -OJ $BASE/api/tests/9/export.xlsx
curl $BASE/api/tests/9/export.json
```
+17 -7
View File
@@ -16,6 +16,7 @@
- 配置可**保存/加载/删除**,方便多模型对比 - 配置可**保存/加载/删除**,方便多模型对比
### 🚀 速度测试配置 ### 🚀 速度测试配置
- **测试名称(主题)**:可为每次测试命名,用于标注测试内容/主题,展示在历史列表、详情弹窗与导出报表中
- **多上下文长度测试**:默认为 `512 / 2048 / 8192 / 32768 / 131072` tokens,可通过标签点击启用/禁用,并支持手动添加任意自定义长度(≥16) - **多上下文长度测试**:默认为 `512 / 2048 / 8192 / 32768 / 131072` tokens,可通过标签点击启用/禁用,并支持手动添加任意自定义长度(≥16)
- **解码输出长度(max tokens**:默认为 `128`,可手动自定义 - **解码输出长度(max tokens**:默认为 `128`,可手动自定义
- **每个长度采样次数**:默认为 `2`,可手动自定义 - **每个长度采样次数**:默认为 `2`,可手动自定义
@@ -24,13 +25,19 @@
### 📊 指标与结果 ### 📊 指标与结果
- 实时指标卡:首字延迟、预填充速度、解码速度、上文/输出 tokens、总耗时 - 实时指标卡:首字延迟、预填充速度、解码速度、上文/输出 tokens、总耗时
- **整体统计(平均/最小/最大)**:详情弹窗与 Excel 汇总展示首字延迟、预填充速度、解码速度、总耗时的平均/最小/最大
- **推理型模型兼容**:支持 Qwen3 / DeepSeek 等思维链模型(`reasoning_content` / `thinking` / `thought`),思维过程计入输出,不会误报“未收到输出” - **推理型模型兼容**:支持 Qwen3 / DeepSeek 等思维链模型(`reasoning_content` / `thinking` / `thought`),思维过程计入输出,不会误报“未收到输出”
- **采样失败不中断**:单次采样失败会记录并继续,不会让整个测试半途终止;全部失败才标记 error - **采样失败不中断**:单次采样失败会记录并继续,不会让整个测试半途终止;全部失败才标记 error
- 实时控制台日志:校准、预热、每次采样明细全程可追溯 - 实时控制台日志:校准、预热、每次采样明细全程可追溯
- **每次完整测试**支持: - **每次完整测试**支持:
- **网页点击查看**:历史记录「查看」按钮弹出详情(整体平均 + 按上下文长度分组 + 每次采样明细 + 完整日志) - **网页点击查看**:历史记录「查看」按钮弹出详情(整体统计 + 按上下文长度分组 + 每次采样明细 + 完整日志)
- **文件下载 Excelxlsx)**:历史记录「Excel」按钮或详情弹窗「导出 Excel」,包含 汇总 / 采样明细 / 日志 三个 Sheet - **文件下载 Excelxlsx)**:历史记录「Excel」按钮或详情弹窗「导出 Excel」,包含 汇总 / 采样明细 / 日志 三个 Sheet
- 测试历史留存,可随时刷新、查看、导出、删除 - **文件下载 JSON**:详情弹窗「导出 JSON」
- 测试历史留存(含测试名称),可随时刷新、查看、导出、删除
### 🔌 开放 API
- 页面所有功能均通过 REST API 提供,前端只是可视化客户端
- **API 文档见 [API.md](API.md)**
--- ---
@@ -81,14 +88,17 @@ pip install -r requirements.txt
|------|------|------| |------|------|------|
| GET | `/api/health` | 健康检查 | | GET | `/api/health` | 健康检查 |
| GET/POST | `/api/configs` | 配置列表 / 新增配置 | | GET/POST | `/api/configs` | 配置列表 / 新增配置 |
| GET/DELETE | `/api/configs/<id>` | 单个配置 / 删除 | | GET/PUT/DELETE | `/api/configs/<id>` | 单个配置 / 更新(局部)/ 删除 |
| POST | `/api/configs/test` | 测试连接 | | POST | `/api/configs/test` | 测试连接 |
| GET/POST | `/api/tests` | 测试历史 / 启动测试 | | GET/POST | `/api/tests?limit=n` | 测试历史 / 启动测试(异步) |
| GET | `/api/tests/<id>` | 测试详情(含 runs / logs / summary | | GET | `/api/tests/<id>` | 测试详情(含 runs / logs / summary / 按长度分组 |
| GET | `/api/tests/<id>/logs?after=<id>` | 增量日志(前端轮询用) | | GET | `/api/tests/<id>/logs?after=<id>` | 增量日志(前端轮询用) |
| POST | `/api/tests/<id>/cancel` | 停止测试 | | POST | `/api/tests/<id>/cancel` | 停止测试 |
| DELETE | `/api/tests/<id>` | 删除测试 | | DELETE | `/api/tests/<id>` | 删除测试 |
| GET | `/api/tests/<id>/export.xlsx` | 导出 Excel 报告 | | GET | `/api/tests/<id>/export.xlsx` | 导出 Excel 报告(汇总/采样明细/日志三 Sheet |
| GET | `/api/tests/<id>/export.json` | 导出完整测试 JSON |
> 完整字段说明、响应示例与 curl 示例见 **API.md**。
### 启动测试请求示例 ### 启动测试请求示例
```json ```json
@@ -151,4 +161,4 @@ llm-speed-tester/
## Git ## Git
- **仓库:** `hz4th_coder/llm-speed-tester` - **仓库:** `hz4th_coder/llm-speed-tester`
- **版本:** v2.0.1(新增多上下文长度测试 + 预热 + Excel 导出 + 界面优化 + 推理型模型兼容 + 采样失败不中断 - **版本:** v2.1.0多上下文长度测试 + 预热 + Excel/JSON 导出 + 测试名称 + 整体统计平均/最小/最大 + 开放 API + 推理型模型兼容
+49 -13
View File
@@ -62,6 +62,16 @@ def get_one_config(cid):
return jsonify(c) return jsonify(c)
@app.route("/api/configs/<int:cid>", methods=["PUT"])
def update_config(cid):
cfg = request.get_json(force=True) or {}
old = db.get_config(cid)
if not old:
return jsonify({"ok": False, "error": "配置不存在"}), 404
db.update_config(cid, cfg)
return jsonify({"ok": True, "id": cid})
@app.route("/api/configs/<int:cid>", methods=["DELETE"]) @app.route("/api/configs/<int:cid>", methods=["DELETE"])
def del_config(cid): def del_config(cid):
db.delete_config(cid) db.delete_config(cid)
@@ -107,7 +117,11 @@ def start_test():
@app.route("/api/tests", methods=["GET"]) @app.route("/api/tests", methods=["GET"])
def list_tests(): def list_tests():
return jsonify(db.list_tests()) try:
limit = int(request.args.get("limit", 100))
except ValueError:
limit = 100
return jsonify(db.list_tests(max(1, min(limit, 1000))))
@app.route("/api/tests/<int:tid>", methods=["GET"]) @app.route("/api/tests/<int:tid>", methods=["GET"])
@@ -117,6 +131,7 @@ def get_test(tid):
return jsonify({"ok": False, "error": "测试不存在"}), 404 return jsonify({"ok": False, "error": "测试不存在"}), 404
t["runs"] = db.get_runs(tid) t["runs"] = db.get_runs(tid)
t["logs"] = db.get_logs(tid) t["logs"] = db.get_logs(tid)
_mask_cfg(t.get("config"))
return jsonify(t) return jsonify(t)
@@ -145,6 +160,25 @@ def del_test(tid):
return jsonify({"ok": True}) return jsonify({"ok": True})
@app.route("/api/tests/<int:tid>/export.json")
def export_json(tid):
t = db.get_test(tid)
if not t:
return jsonify({"ok": False, "error": "测试不存在"}), 404
t["runs"] = db.get_runs(tid)
t["logs"] = db.get_logs(tid)
_mask_cfg(t.get("config"))
return jsonify(t)
def _mask_cfg(cfg):
"""对外隐藏 API Key,仅保留前几位便于识别"""
if cfg and cfg.get("api_key"):
k = cfg["api_key"]
cfg["api_key"] = k[:4] + "****" if len(k) > 6 else "****"
return cfg
# ───────────────────────── Excel 导出 ───────────────────────── # ───────────────────────── Excel 导出 ─────────────────────────
@app.route("/api/tests/<int:tid>/export.xlsx") @app.route("/api/tests/<int:tid>/export.xlsx")
@@ -193,6 +227,7 @@ def _build_xlsx(t):
ws.cell(1, 1).font = Font(bold=True, size=14) ws.cell(1, 1).font = Font(bold=True, size=14)
info = [ info = [
["测试编号", "#%d" % t["id"]], ["测试编号", "#%d" % t["id"]],
["测试名称", t.get("name") or "(未命名)"],
["创建时间", t.get("created_at", "")], ["创建时间", t.get("created_at", "")],
["状态", t.get("status", "")], ["状态", t.get("status", "")],
["提供商", t.get("provider", "")], ["提供商", t.get("provider", "")],
@@ -209,25 +244,26 @@ def _build_xlsx(t):
] ]
for row in info: for row in info:
ws.append(row) ws.append(row)
ws.cell(14, 1).font = title_font ws.cell(15, 1).font = title_font
r0 = len(info) + 2 r0 = len(info) + 2
overall = [ overall = [
["平均首字延迟(ms)", s.get("avg_ttft_ms")], ["首字延迟(ms)", s.get("avg_ttft_ms"), s.get("max_ttft_ms"), s.get("min_ttft_ms")],
["最佳首字延迟(ms)", s.get("best_ttft_ms")], ["预填充速度(tok/s)", s.get("avg_prefill_speed"), s.get("max_prefill_speed"), s.get("min_prefill_speed")],
["平均预填充速度(tok/s)", s.get("avg_prefill_speed")], ["解码速度(tok/s)", s.get("avg_decode_speed"), s.get("max_decode_speed"), s.get("min_decode_speed")],
["平均解码速度(tok/s)", s.get("avg_decode_speed")], ["提示词(tok)", s.get("avg_prompt_tokens"), None, None],
["平均提示词(tok)", s.get("avg_prompt_tokens")], ["输出(tok)", s.get("avg_output_tokens"), None, None],
["平均输出(tok)", s.get("avg_output_tokens")], ["总耗时(ms)", s.get("avg_total_ms"), s.get("max_total_ms"), s.get("min_total_ms")],
["平均总耗时(ms)", s.get("avg_total_ms")],
] ]
ws.cell(r0, 1, "整体平均指标").font = title_font ws.cell(r0, 1, "整体统计指标(平均 / 最大 / 最小)").font = title_font
for i, row in enumerate(overall, start=r0 + 1): for j, c in enumerate(["指标", "平均", "最大", "最小"], start=1):
ws.append([]) ws.cell(row=r0 + 1, column=j, value=c)
style_header(ws, r0 + 1, 4)
for i, row in enumerate(overall, start=r0 + 2):
for j, v in enumerate(row, start=1): for j, v in enumerate(row, start=1):
ws.cell(row=i, column=j, value=v) ws.cell(row=i, column=j, value=v)
# 按上下文长度分组 # 按上下文长度分组
r1 = r0 + len(overall) + 2 r1 = r0 + len(overall) + 3
ws.cell(r1, 1, "按上下文长度分组").font = title_font ws.cell(r1, 1, "按上下文长度分组").font = title_font
cols = ["上下文长度(tok)", "采样(成功/总数)", "首字ms", "预填充tok/s", "解码tok/s", "提示词tok", "输出tok", "总耗时ms"] cols = ["上下文长度(tok)", "采样(成功/总数)", "首字ms", "预填充tok/s", "解码tok/s", "提示词tok", "输出tok", "总耗时ms"]
ws.append([]) ws.append([])
+27 -4
View File
@@ -28,6 +28,7 @@ CREATE TABLE IF NOT EXISTS tests(
status TEXT DEFAULT 'running', status TEXT DEFAULT 'running',
provider TEXT DEFAULT '', provider TEXT DEFAULT '',
model TEXT DEFAULT '', model TEXT DEFAULT '',
name TEXT DEFAULT '',
config_json TEXT DEFAULT '{}', config_json TEXT DEFAULT '{}',
gen_cfg_json TEXT DEFAULT '{}', gen_cfg_json TEXT DEFAULT '{}',
summary_json TEXT DEFAULT '{}', summary_json TEXT DEFAULT '{}',
@@ -67,11 +68,15 @@ def _connect():
def _migrate(conn): def _migrate(conn):
"""老库升级:test_runs 补 context_length""" """老库升级:test_runs 补 context_length、tests 补 name"""
cur = conn.execute("PRAGMA table_info(test_runs)") cur = conn.execute("PRAGMA table_info(test_runs)")
cols = [r[1] for r in cur.fetchall()] cols = [r[1] for r in cur.fetchall()]
if "context_length" not in cols: if "context_length" not in cols:
conn.execute("ALTER TABLE test_runs ADD COLUMN context_length INTEGER DEFAULT 0") conn.execute("ALTER TABLE test_runs ADD COLUMN context_length INTEGER DEFAULT 0")
cur = conn.execute("PRAGMA table_info(tests)")
tcols = [r[1] for r in cur.fetchall()]
if "name" not in tcols:
conn.execute("ALTER TABLE tests ADD COLUMN name TEXT DEFAULT ''")
def init_db(): def init_db():
@@ -134,6 +139,23 @@ def delete_config(cid: int):
conn.close() conn.close()
def update_config(cid: int, cfg: dict):
"""局部更新:只更新请求里出现的字段"""
allow = {"name", "provider", "base_url", "api_key", "model", "temperature"}
fields = {k: v for k, v in cfg.items() if k in allow and v is not None}
if not fields:
return
sets = ",".join("%s=?" % k for k in fields)
vals = list(fields.values()) + [cid]
with _lock:
conn = _connect()
try:
conn.execute("UPDATE configs SET %s WHERE id=?" % sets, vals)
conn.commit()
finally:
conn.close()
# ───────────────────────── 测试记录 ───────────────────────── # ───────────────────────── 测试记录 ─────────────────────────
def create_test(cfg: dict, gen: dict) -> int: def create_test(cfg: dict, gen: dict) -> int:
@@ -141,9 +163,10 @@ def create_test(cfg: dict, gen: dict) -> int:
conn = _connect() conn = _connect()
try: try:
cur = conn.execute( cur = conn.execute(
"INSERT INTO tests(status,provider,model,config_json,gen_cfg_json,started_at) " "INSERT INTO tests(status,provider,model,name,config_json,gen_cfg_json,started_at) "
"VALUES('running',?,?,?,?,?)", "VALUES('running',?,?,?,?,?,?)",
(cfg.get("provider", "openai"), cfg.get("model", ""), (cfg.get("provider", "openai"), cfg.get("model", ""),
gen.get("name") or cfg.get("name") or "",
json.dumps(cfg, ensure_ascii=False), json.dumps(gen, ensure_ascii=False), json.dumps(cfg, ensure_ascii=False), json.dumps(gen, ensure_ascii=False),
time.time())) time.time()))
conn.commit() conn.commit()
@@ -186,7 +209,7 @@ def list_tests(limit=100):
conn = _connect() conn = _connect()
try: try:
rows = conn.execute( rows = conn.execute(
"SELECT id,created_at,status,provider,model,summary_json,error " "SELECT id,created_at,status,provider,model,name,summary_json,error "
"FROM tests ORDER BY id DESC LIMIT ?", (limit,)).fetchall() "FROM tests ORDER BY id DESC LIMIT ?", (limit,)).fetchall()
out = [] out = []
for r in rows: for r in rows:
+4
View File
@@ -184,6 +184,10 @@ table.history { width: 100%; border-collapse: collapse; font-size: 13px; }
.modal-body { padding: 16px 18px; overflow-y: auto; } .modal-body { padding: 16px 18px; overflow-y: auto; }
.modal-body h3 { font-size: 14px; margin: 16px 0 8px; color: var(--muted); } .modal-body h3 { font-size: 14px; margin: 16px 0 8px; color: var(--muted); }
.modal-body h3:first-child { margin-top: 0; } .modal-body h3:first-child { margin-top: 0; }
.detail-name {
background: rgba(79,140,255,.12); border: 1px solid rgba(79,140,255,.35); color: var(--accent);
border-radius: 10px; padding: 10px 14px; font-size: 14px; font-weight: 600; margin-bottom: 6px;
}
.kv { display: grid; grid-template-columns: repeat(auto-fill, minmax(180px, 1fr)); gap: 8px; } .kv { display: grid; grid-template-columns: repeat(auto-fill, minmax(180px, 1fr)); gap: 8px; }
.kv .kv-item { background: var(--panel2); border: 1px solid var(--border); border-radius: 8px; padding: 8px 10px; } .kv .kv-item { background: var(--panel2); border: 1px solid var(--border); border-radius: 8px; padding: 8px 10px; }
+5 -1
View File
@@ -68,6 +68,10 @@
<section class="card"> <section class="card">
<h2>🚀 速度测试配置</h2> <h2>🚀 速度测试配置</h2>
<div class="field">
<label>测试名称(主题)</label>
<input id="gen-name" type="text" placeholder="如:DeepSeek-V4 不同上下文长度速度对比">
</div>
<div class="field"> <div class="field">
<label>上下文长度(tokens,点击切换启停)</label> <label>上下文长度(tokens,点击切换启停)</label>
<div class="chips" id="gen-contexts"></div> <div class="chips" id="gen-contexts"></div>
@@ -136,7 +140,7 @@
<div class="table-wrap"> <div class="table-wrap">
<table class="history" id="history"> <table class="history" id="history">
<thead><tr> <thead><tr>
<th>#</th><th>时间</th><th>提供商</th><th>模型</th><th>采样</th> <th>#</th><th>时间</th><th>名称</th><th>提供商</th><th>模型</th><th>采样</th>
<th>首字 ms</th><th>预填充 tok/s</th><th>解码 tok/s</th><th>状态</th><th>操作</th> <th>首字 ms</th><th>预填充 tok/s</th><th>解码 tok/s</th><th>状态</th><th>操作</th>
</tr></thead> </tr></thead>
<tbody></tbody> <tbody></tbody>
+29 -11
View File
@@ -57,6 +57,7 @@ function currentConfig() {
function currentGen() { function currentGen() {
const lens = contextLengths.filter((l) => contextLengthsActive.has(l)); const lens = contextLengths.filter((l) => contextLengthsActive.has(l));
return { return {
name: $("#gen-name").value.trim(),
context_lengths: lens.length ? lens : [2048], context_lengths: lens.length ? lens : [2048],
max_tokens: parseInt($("#gen-max-tokens").value) || 128, max_tokens: parseInt($("#gen-max-tokens").value) || 128,
samples: parseInt($("#gen-samples").value) || 2, samples: parseInt($("#gen-samples").value) || 2,
@@ -300,7 +301,7 @@ async function loadHistory() {
const tb = $("#history tbody"); const tb = $("#history tbody");
tb.innerHTML = ""; tb.innerHTML = "";
if (!list.length) { if (!list.length) {
tb.innerHTML = '<tr><td colspan="10" style="color:var(--muted);text-align:center">暂无测试记录</td></tr>'; tb.innerHTML = '<tr><td colspan="11" style="color:var(--muted);text-align:center">暂无测试记录</td></tr>';
return; return;
} }
for (const t of list) { for (const t of list) {
@@ -309,6 +310,7 @@ async function loadHistory() {
tr.innerHTML = ` tr.innerHTML = `
<td>#${t.id}</td> <td>#${t.id}</td>
<td>${esc(t.created_at)}</td> <td>${esc(t.created_at)}</td>
<td title="${esc(t.name || "")}">${esc(t.name || "—")}</td>
<td>${esc(PROVIDER_LABEL[t.provider] || t.provider)}</td> <td>${esc(PROVIDER_LABEL[t.provider] || t.provider)}</td>
<td>${esc(t.model)}</td> <td>${esc(t.model)}</td>
<td class="num">${fmt(s.samples_ok)}/${fmt(s.samples_total)}</td> <td class="num">${fmt(s.samples_ok)}/${fmt(s.samples_total)}</td>
@@ -390,28 +392,38 @@ async function viewDetail(id) {
).join("") + `</div>`; ).join("") + `</div>`;
$("#dt-id").textContent = id; $("#dt-id").textContent = id;
const overallRow = (label, avgV, minV, maxV) =>
`<tr><td>${label}</td><td class="num">${fmt(avgV)}</td><td class="num">${fmt(minV)}</td><td class="num">${fmt(maxV)}</td></tr>`;
$("#dt-body").innerHTML = ` $("#dt-body").innerHTML = `
${t.name ? `<div class="detail-name">🏷️ ${esc(t.name)}</div>` : ""}
<h3>📌 汇总指标</h3> <h3>📌 汇总指标</h3>
<div class="kv"> <div class="kv">
<div class="kv-item"><div class="kv-k">状态</div><div class="kv-v">${STATUS_LABEL[t.status] || t.status}</div></div> <div class="kv-item"><div class="kv-k">状态</div><div class="kv-v">${STATUS_LABEL[t.status] || t.status}</div></div>
<div class="kv-item"><div class="kv-k">创建时间</div><div class="kv-v">${esc(t.created_at)}</div></div> <div class="kv-item"><div class="kv-k">创建时间</div><div class="kv-v">${esc(t.created_at)}</div></div>
<div class="kv-item"><div class="kv-k">提供商 / 模型</div><div class="kv-v">${esc(PROVIDER_LABEL[t.provider] || t.provider)} / ${esc(t.model)}</div></div> <div class="kv-item"><div class="kv-k">提供商 / 模型</div><div class="kv-v">${esc(PROVIDER_LABEL[t.provider] || t.provider)} / ${esc(t.model)}</div></div>
<div class="kv-item"><div class="kv-k">采样(成功/总数)</div><div class="kv-v">${fmt(s.samples_ok)} / ${fmt(s.samples_total)}</div></div> <div class="kv-item"><div class="kv-k">采样(成功/总数)</div><div class="kv-v">${fmt(s.samples_ok)} / ${fmt(s.samples_total)}</div></div>
<div class="kv-item"><div class="kv-k">平均首字延迟</div><div class="kv-v">${fmt(s.avg_ttft_ms)} ms</div></div>
<div class="kv-item"><div class="kv-k">最佳首字延迟</div><div class="kv-v">${fmt(s.best_ttft_ms)} ms</div></div>
<div class="kv-item"><div class="kv-k">平均预填充速度</div><div class="kv-v">${fmt(s.avg_prefill_speed)} tok/s</div></div>
<div class="kv-item"><div class="kv-k">平均解码速度</div><div class="kv-v">${fmt(s.avg_decode_speed)} tok/s</div></div>
<div class="kv-item"><div class="kv-k">平均提示词</div><div class="kv-v">${fmt(s.avg_prompt_tokens)} tok</div></div>
<div class="kv-item"><div class="kv-k">平均输出</div><div class="kv-v">${fmt(s.avg_output_tokens)} tok</div></div>
<div class="kv-item"><div class="kv-k">平均缓存命中</div><div class="kv-v">${fmt(s.avg_cached_tokens, 0)} tok</div></div> <div class="kv-item"><div class="kv-k">平均缓存命中</div><div class="kv-v">${fmt(s.avg_cached_tokens, 0)} tok</div></div>
<div class="kv-item"><div class="kv-k">平均总耗时</div><div class="kv-v">${fmt(s.avg_total_ms)} ms</div></div> <div class="kv-item"><div class="kv-k">校准字符/token</div><div class="kv-v">${fmt(s.calibration_chars_per_token)}</div></div>
</div> </div>
<h3>📊 整体统计(平均 / 最小 / 最大)</h3>
<table class="mini"><thead><tr>
<th>指标</th><th>平均</th><th>最小</th><th>最大</th>
</tr></thead><tbody>
${overallRow("首字延迟(ms)", s.avg_ttft_ms, s.min_ttft_ms, s.max_ttft_ms)}
${overallRow("预填充速度(tok/s)", s.avg_prefill_speed, s.min_prefill_speed, s.max_prefill_speed)}
${overallRow("解码速度(tok/s)", s.avg_decode_speed, s.min_decode_speed, s.max_decode_speed)}
${overallRow("提示词(tok)", s.avg_prompt_tokens, null, null)}
${overallRow("输出(tok)", s.avg_output_tokens, null, null)}
${overallRow("总耗时(ms)", s.avg_total_ms, s.min_total_ms, s.max_total_ms)}
</tbody></table>
<h3>📏 按上下文长度汇总</h3> <h3>📏 按上下文长度汇总</h3>
${byLengthHtml} ${byLengthHtml}
<h3>⚙️ 测试参数</h3> <h3>⚙️ 测试参数</h3>
<div class="kv"> <div class="kv">
<div class="kv-item"><div class="kv-k">测试名称</div><div class="kv-v">${esc(t.name || "—")}</div></div>
<div class="kv-item"><div class="kv-k">上下文长度</div><div class="kv-v">${(g.context_lengths || []).join(" / ") || "—"} tok</div></div> <div class="kv-item"><div class="kv-k">上下文长度</div><div class="kv-v">${(g.context_lengths || []).join(" / ") || "—"} tok</div></div>
<div class="kv-item"><div class="kv-k">解码输出长度</div><div class="kv-v">${g.max_tokens ?? "—"} tok</div></div> <div class="kv-item"><div class="kv-k">解码输出长度</div><div class="kv-v">${g.max_tokens ?? "—"} tok</div></div>
<div class="kv-item"><div class="kv-k">每个长度采样</div><div class="kv-v">${g.samples ?? "—"}</div></div> <div class="kv-item"><div class="kv-k">每个长度采样</div><div class="kv-v">${g.samples ?? "—"}</div></div>
@@ -419,7 +431,6 @@ async function viewDetail(id) {
<div class="kv-item"><div class="kv-k">避免缓存</div><div class="kv-v">${g.avoid_cache ? "开" : "关"}</div></div> <div class="kv-item"><div class="kv-k">避免缓存</div><div class="kv-v">${g.avoid_cache ? "开" : "关"}</div></div>
<div class="kv-item"><div class="kv-k">温度</div><div class="kv-v">${fmt(cfg.temperature)}</div></div> <div class="kv-item"><div class="kv-k">温度</div><div class="kv-v">${fmt(cfg.temperature)}</div></div>
<div class="kv-item"><div class="kv-k">Base URL</div><div class="kv-v">${esc(cfg.base_url || "(默认)")}</div></div> <div class="kv-item"><div class="kv-k">Base URL</div><div class="kv-v">${esc(cfg.base_url || "(默认)")}</div></div>
<div class="kv-item"><div class="kv-k">校准字符/token</div><div class="kv-v">${fmt(s.calibration_chars_per_token)}</div></div>
</div> </div>
<h3>📊 每次采样明细</h3> <h3>📊 每次采样明细</h3>
@@ -437,9 +448,16 @@ async function viewDetail(id) {
function closeDetail() { $("#detail-mask").hidden = true; } function closeDetail() { $("#detail-mask").hidden = true; }
function exportDetail() { async function exportDetail() {
if (!window.__detail) return; if (!window.__detail) return;
download(JSON.stringify(window.__detail, null, 2), `test_${window.__detail.id}.json`, "application/json"); try {
const resp = await fetch(`/api/tests/${window.__detail.id}/export.json`);
if (!resp.ok) throw new Error("服务端导出失败");
const j = await resp.json();
download(JSON.stringify(j, null, 2), `test_${j.id}.json`, "application/json");
} catch (e) {
toast("导出失败:" + e.message);
}
} }
async function exportXlsx(id) { async function exportXlsx(id) {
+21 -1
View File
@@ -63,6 +63,9 @@ class TestRunner(threading.Thread):
warmup = bool(gen.get("warmup", True)) # 测试前空转预热 warmup = bool(gen.get("warmup", True)) # 测试前空转预热
self.log("INFO", "═══ 开始速度测试 ═══") self.log("INFO", "═══ 开始速度测试 ═══")
name = gen.get("name") or self.cfg.get("name") or ""
if name:
self.log("INFO", "测试名称(主题): %s" % name)
self.log("INFO", "提供商: %s | 模型: %s" % (lp.PROVIDER_LABELS.get(provider, provider), model)) self.log("INFO", "提供商: %s | 模型: %s" % (lp.PROVIDER_LABELS.get(provider, provider), model))
self.log("INFO", "上下文长度: %s tokens | 生成长度: %d tokens | 每个长度采样: %d 次 | 预热: %s | 避免缓存: %s" self.log("INFO", "上下文长度: %s tokens | 生成长度: %d tokens | 每个长度采样: %d 次 | 预热: %s | 避免缓存: %s"
% (" / ".join(str(x) for x in lengths), max_tokens, n, % (" / ".join(str(x) for x in lengths), max_tokens, n,
@@ -212,16 +215,33 @@ class TestRunner(threading.Thread):
} }
okm = [s["metrics"] for s in ok] okm = [s["metrics"] for s in ok]
def mn(k):
vals = [m[k] for m in okm if m.get(k) is not None]
return round(min(vals), 1) if vals else None
def mx(k):
vals = [m[k] for m in okm if m.get(k) is not None]
return round(max(vals), 1) if vals else None
summary = dict(base) summary = dict(base)
summary.update({ summary.update({
"by_length": by_length, "by_length": by_length,
"avg_ttft_ms": avg(okm, "ttft_ms"), "avg_ttft_ms": avg(okm, "ttft_ms"),
"min_ttft_ms": mn("ttft_ms"),
"max_ttft_ms": mx("ttft_ms"),
"avg_prefill_speed": avg(okm, "prefill_speed"), "avg_prefill_speed": avg(okm, "prefill_speed"),
"min_prefill_speed": mn("prefill_speed"),
"max_prefill_speed": mx("prefill_speed"),
"avg_decode_speed": avg(okm, "decode_speed"), "avg_decode_speed": avg(okm, "decode_speed"),
"min_decode_speed": mn("decode_speed"),
"max_decode_speed": mx("decode_speed"),
"avg_prompt_tokens": avg(okm, "prompt_tokens"), "avg_prompt_tokens": avg(okm, "prompt_tokens"),
"avg_output_tokens": avg(okm, "output_tokens"), "avg_output_tokens": avg(okm, "output_tokens"),
"avg_cached_tokens": avg(okm, "cached_tokens"), "avg_cached_tokens": avg(okm, "cached_tokens"),
"avg_total_ms": avg(okm, "total_ms"), "avg_total_ms": avg(okm, "total_ms"),
"best_ttft_ms": min([m["ttft_ms"] for m in okm if m.get("ttft_ms") is not None], default=None), "min_total_ms": mn("total_ms"),
"max_total_ms": mx("total_ms"),
"best_ttft_ms": mn("ttft_ms"),
}) })
return summary return summary