Compare commits

6 Commits
Author SHA1 Message Date
hz4th_coder ccfd7eb689 v2.8.0 清空历史真正物理删除 + 请求超时可配修复慢接口被掐断
1. 清空历史=真正清空(不再只是删行留痕):
   - 清空前先停止所有运行中测试线程,避免其继续写入孤儿数据
   - DELETE 后重置自增序列(sqlite_sequence),新测试 ID 从 1 开始
   - WAL checkpoint(TRUNCATE) + VACUUM 物理回收文件空间,数据不可恢复
   - 单条删除删光时同样重置自增序列

2. 慢接口被掐断修复(预处理/首字耗时过长 Read timed out):
   - 读取超时默认 300s→1800s(30分钟),连接 60s
   - 「速度测试配置」新增 请求超时(秒) 输入框(默认1800),测试连接+速度采样共用,按次可调(如3600)
   - 超时错误改为友好提示:等待响应超时(超过N秒未收到数据),引导调大请求超时,不再裸抛 Read timed out
   - 测试启动日志打印当前超时配置
   - 修复案例: 18008 Qwen3.8-FP8 131072 长上下文预填充被 300s 掐断
2026-09-13 12:38:40 +08:00
hz4th_coder 31e5bbb873 v2.7.2 查看模型接口不通时不显示原始错误,优雅提示接口不通 2026-09-09 11:06:39 +08:00
hz4th_coder 8c413cc997 v2.7.1 左侧配置面板独立滚动:鼠标在左侧滚动只翻左侧几个块,右侧滚右侧 2026-09-06 10:56:50 +08:00
hz4th_coder f499144465 v2.7.0 一键导出全部历史(GET /api/tests/export JSON, Key打码);左侧三块配置卡片可折叠展开(状态localStorage持久化);置顶/置底按钮高亮蓝渐变高对比配色 2026-09-06 02:54:16 +08:00
hz4th_coder edbaedce15 v2.6.0 历史分页+筛选+搜索+表头升降排序;一键清空历史(DELETE /api/tests);右下角置顶/置底悬浮按钮 2026-09-06 02:32:35 +08:00
hz4th_coder 901232c75f v2.5.1 修复解码速度荒谬值:兼容 vLLM reasoning 字段思维链 + 幻影token防护(usage有token但流无正文时解码速度置空) 2026-09-06 02:17:40 +08:00
10 changed files with 582 additions and 71 deletions
+27 -7
View File
@@ -137,20 +137,35 @@
**响应:** `{ "ok": true, "id": 9 }`
### `GET /api/tests?limit=<n>`
测试历史列表(按 id 倒序)。`limit` 默认 100,最大 1000。
### `GET /api/tests`
测试历史列表,支持**分页 + 搜索 + 筛选 + 排序**:
| 参数 | 说明 |
|------|------|
| `page` | 页码,默认 1 |
| `page_size` | 每页条数,默认 20,最大 100 |
| `q` | 关键词搜索(匹配 id/名称/模型/提供商/状态/时间) |
| `status` | 状态筛选,逗号多选:`done,error,canceled,running` |
| `provider` | 提供商筛选:`openai` / `anthropic` / `google` |
| `sort` | 排序字段:`id`/`created_at`/`name`/`provider`/`model`/`status`/`samples`/`ttft`/`prefill`/`decode`/`total_ms`,默认 `id` |
| `order` | 排序方向:`asc` / `desc`,默认 `desc`(数值字段空值恒排最后) |
**响应:**
```json
[
{ "id": 9, "created_at": "2026-08-23 18:52:00", "status": "done",
{ "ok": true, "items": [ { "id": 9, "created_at": "2026-08-23 18:52:00", "status": "done",
"provider": "openai", "model": "unsloth/Qwen3.8-27B-Q4_K_M",
"name": "Qwen3 不同上下文长度速度对比",
"error": "",
"summary": { "samples_ok": 2, "samples_total": 2, "avg_ttft_ms": 1808.7, ... } }
]
"summary": { "samples_ok": 2, "samples_total": 2, "avg_ttft_ms": 1808.7, ... } } ],
"total": 31, "page": 1, "page_size": 20, "total_pages": 2 }
```
### `DELETE /api/tests`
一键清空全部测试历史(测试记录 + 采样指标 + 日志),不可恢复。响应 `{ "ok": true }`
### `GET /api/tests/export`
一键导出全部历史记录为 JSON 文件下载(`Content-Disposition: attachment`,文件名带时间戳)。每条含基本字段 + 配置(**API Key 打码为 `******`**)+ 生成参数 + 汇总指标,不含逐条采样与完整日志。
### `POST /api/tests/<id>/cancel`
停止正在运行的测试。响应 `{ "ok": true, "msg": "正在停止..." }`
@@ -353,7 +368,12 @@ curl -X POST $BASE/api/tests -H 'Content-Type: application/json' -d '{
}'
# 查询测试列表 / 详情
curl "$BASE/api/tests?limit=10"
curl "$BASE/api/tests?page=1&page_size=20"
curl "$BASE/api/tests?q=Qwen&status=done,error&sort=decode&order=desc&page=1&page_size=10"
# 一键清空全部历史(不可恢复)
curl -X DELETE $BASE/api/tests
# 一键导出全部历史(JSON 文件下载)
curl -OJ $BASE/api/tests/export
curl $BASE/api/tests/9
# 画图数据(CSV
+12 -4
View File
@@ -4,7 +4,7 @@
- **访问地址:** `http://<IP>:16097/`
- **技术栈:** Python 3 + Flask + SQLite(纯 REST,无额外依赖)
- **版本:** v2.5.0
- **版本:** v2.7.0
---
@@ -30,7 +30,8 @@
### 📊 指标与结果
- 实时指标卡:首字延迟、预填充速度、解码速度、上文/输出 tokens、总耗时
- **整体统计(平均/最小/最大)**:详情弹窗与 Excel 汇总展示首字延迟、预填充速度、解码速度、总耗时的平均/最小/最大
- **推理型模型兼容**:支持 Qwen3 / DeepSeek 等思维链模型(`reasoning_content` / `thinking` / `thought`),思维过程计入输出,不会误报“未收到输出”
- **推理型模型兼容**:支持 Qwen3 / DeepSeek 等思维链模型(`reasoning_content` / `reasoning` / `thinking` / `thought`),思维过程计入输出,不会误报“未收到输出”
- **解码速度防荒谬值**:兼容 vLLM/Qwen3.8-27B 用 `reasoning` 字段流式输出思维链(此前未识别导致 output_chars=0、解码时间≈0、解码速度飙到数百万 tok/s,已修复);另防“幻影 token”——usage 报了输出 token 但流里没收到任何文本时(如推理用满 max_tokens 没正文),解码速度标记为不可测(—)而非荒谬数字
- **采样失败不中断**:单次采样失败会记录并继续,不会让整个测试半途终止;全部失败才标记 error
- 实时控制台日志:校准、预热、每次采样明细全程可追溯
- **每次完整测试**支持:
@@ -39,6 +40,11 @@
- **文件下载 Excelxlsx)**:历史记录「Excel」按钮或详情弹窗「导出 Excel」,包含 汇总 / 采样明细 / 日志 三个 Sheet
- **文件下载 JSON**:详情弹窗「导出 JSON」
- 测试历史留存(含测试名称),可随时刷新、查看、导出、删除
- **📥 一键导出全部历史**:历史卡片「导出全部」按钮,把全部测试记录(配置 Key 打码 + 生成参数 + 汇总指标)导出为一个 JSON 文件下载(`GET /api/tests/export`
- **📄 历史分页 + 筛选 + 搜索 + 排序**:历史列表支持分页(10/20/50/100 条每页)、多条件筛选(状态多选、提供商)、关键词搜索(编号/名称/模型/提供商/状态/时间)、点击表头升降排序(#/时间/名称/提供商/模型/采样/首字/预填充/解码/状态,数值列空值恒排最后);提供「✕ 重置」一键恢复默认
- **🗑 一键清空历史**:历史卡片右上角「清空历史」二次确认后,删除全部测试记录/采样/日志(不可恢复)
- **🗜 左侧配置块可折叠**:大模型接口配置 / 速度测试配置 / 评测站同步 三个卡片点击标题即可折叠/展开(箭头指示),状态本地持久化
- **⬆⬇ 一键置顶/置底**:右下角固定悬浮按钮(亮蓝渐变高对比配色,易于发现),平滑滚动到页面顶部/底部
- **⚖️ 多测试结果对比**:测试历史表格勾选多个测试(可全选),点「⚖️ 对比所选」弹出对比面板:
- **指标对比表**:时间 / 名称 / 模型 / 并发档 / 采样 / 首字 / 预填充 / 解码 / 单流均解码 / 输出 / 总耗时 并排展示(点击测试名可跳详情)
- **柱状图**:各测试预填充(空心柱)vs 解码(实心柱)速度对比
@@ -100,7 +106,9 @@ pip install -r requirements.txt
| GET/POST | `/api/configs` | 配置列表 / 新增配置 |
| GET/PUT/DELETE | `/api/configs/<id>` | 单个配置 / 更新(局部)/ 删除 |
| POST | `/api/configs/test` | 测试连接 |
| GET/POST | `/api/tests?limit=n` | 测试历史 / 启动测试(异步) |
| GET/POST | `/api/tests?page=&page_size=&q=&status=&provider=&sort=&order=` | 测试历史(分页/搜索/筛选/排序)/ 启动测试(异步) |
| DELETE | `/api/tests` | 一键清空全部测试历史(含采样与日志) |
| GET | `/api/tests/export` | 一键导出全部历史记录(JSON 文件下载,API Key 打码) |
| GET | `/api/tests/<id>` | 测试详情(含 runs / logs / summary / 按长度分组) |
| GET | `/api/tests/<id>/logs?after=<id>` | 增量日志(前端轮询用) |
| POST | `/api/tests/<id>/cancel` | 停止测试 |
@@ -178,4 +186,4 @@ llm-speed-tester/
## Git
- **仓库:** `hz4th_coder/llm-speed-tester`
- **版本:** v2.5.0模型实时获取:接口配置区「📋 查看模型」按钮,Base URL+API Key 下实时拉取模型列表(GET /models),搜索/点击填入,仍支持手动输入
- **版本:** v2.7.0一键导出全部历史(JSON);左侧三块配置卡片可折叠展开(状态持久化);置顶/置底按钮高亮蓝渐变配色
+61 -5
View File
@@ -3,6 +3,8 @@
import io
import json
import subprocess
import time
from datetime import datetime
import requests
from flask import Flask, jsonify, request, send_file, send_from_directory
@@ -111,13 +113,17 @@ def list_models_api():
@app.route("/api/configs/test", methods=["POST"])
def test_config():
cfg = _fill_defaults(request.get_json(force=True) or {})
body = request.get_json(force=True) or {}
cfg = _fill_defaults(body)
if not cfg.get("api_key"):
return jsonify({"ok": False, "error": "请填写 API Key"}), 400
try:
# 连接测试:只要流式请求成功返回(哪怕正文为空/只有思维链)都算连通
# 慢接口长预处理可能长时间等不到首字,这里直接用请求里带的超时(默认 30 分钟),避免误判接口不通
m = call_stream(cfg, "你好,请简要回答:1+1=",
{"max_tokens": 32, "avoid_cache": False})
{"max_tokens": 32, "avoid_cache": False,
"read_timeout": body.get("read_timeout"),
"connect_timeout": body.get("connect_timeout")})
note = ""
if not (m.get("output_tokens") or m.get("output_chars")):
note = "(连接正常,但本次未返回正文内容,可能为推理型模型)"
@@ -146,13 +152,63 @@ def start_test():
return jsonify({"ok": True, "id": tid})
@app.route("/api/tests/export")
def export_tests():
"""一键导出全部历史记录(JSON 文件下载,API Key 打码)"""
data = db.export_tests()
payload = {
"ok": True,
"count": len(data),
"exported_at": datetime.now().strftime("%Y-%m-%d %H:%M:%S"),
"tests": data,
}
resp = app.response_class(json.dumps(payload, ensure_ascii=False),
mimetype="application/json")
resp.headers["Content-Disposition"] = (
'attachment; filename="llm_speed_tests_%s.json"'
% datetime.now().strftime("%Y%m%d_%H%M%S"))
return resp
@app.route("/api/tests", methods=["GET"])
def list_tests():
"""历史记录列表:支持关键词搜索 q、状态 status(逗号多选)、提供商 provider、
字段排序 sort/order、分页 page/page_size"""
try:
limit = int(request.args.get("limit", 100))
page = max(1, int(request.args.get("page", 1)))
page_size = min(100, max(1, int(request.args.get("page_size", 20))))
except ValueError:
limit = 100
return jsonify(db.list_tests(max(1, min(limit, 1000))))
page, page_size = 1, 20
q = (request.args.get("q") or "").strip()
status = (request.args.get("status") or "").strip()
provider = (request.args.get("provider") or "").strip()
sort = (request.args.get("sort") or "id").strip()
order = (request.args.get("order") or "desc").strip().lower()
if order not in ("asc", "desc"):
order = "desc"
data = db.query_tests(q=q, status=status, provider=provider,
sort=sort, order=order, page=page, page_size=page_size)
return jsonify({"ok": True, **data})
@app.route("/api/tests", methods=["DELETE"])
def clear_tests():
"""一键清空全部测试历史(含采样指标与日志)。
先停止所有运行中的测试线程,避免清空后它们继续写入孤儿数据;
再由 db.clear_tests 做物理删除(重置自增 + VACUUM 瘦身),真正清空、不可恢复。"""
# 1) 停止所有运行中的测试,并等待其退出
alive = [r for r in RUNNERS.values() if r.is_alive()]
for r in alive:
r.request_cancel()
if alive:
deadline = time.time() + 5
for r in alive:
if r.is_alive():
r.join(timeout=max(0.1, deadline - time.time()))
RUNNERS.clear()
# 2) 物理清空数据库
db.clear_tests()
return jsonify({"ok": True})
@app.route("/api/tests/<int:tid>", methods=["GET"])
+4 -2
View File
@@ -10,9 +10,11 @@ DATA_DIR = os.path.join(BASE_DIR, "data")
LOG_DIR = os.path.join(BASE_DIR, "logs")
DB_PATH = os.path.join(DATA_DIR, "llm_speed_tester.db")
# 流式请求超时:连接 60s,两次数据包间隔最长 300s推理型/长上文模型也够用
# 流式请求超时:连接 60s,两次数据包间隔最长 1800s30 分钟
# 注意:慢接口长上下文预处理(预填充)期间服务端可能长时间不返回任何字节,
# 等待首字受 STREAM_READ_TIMEOUT 限制;如接口更慢可在「速度测试配置」里按次调大请求超时。
CONNECT_TIMEOUT = 60
STREAM_READ_TIMEOUT = 300
STREAM_READ_TIMEOUT = 1800
# data-chart-tool 图表服务地址(用它的 /api/chart 画折线图:预填充左轴虚线 / 解码右轴实线)
CHART_API_BASE = "http://127.0.0.1:16016"
+105 -3
View File
@@ -2,6 +2,7 @@
"""SQLite 存储:提供商配置 / 测试记录 / 每次采样指标 / 全量日志"""
import os
import json
import math
import sqlite3
import threading
import time
@@ -204,20 +205,93 @@ def get_test(tid: int):
conn.close()
def list_tests(limit=100):
# 历史列表排序字段 → 取值函数(数值字段取 summary 聚合值,None 恒排最后)
SORT_KEYS = {
"id": lambda t: t["id"],
"created_at": lambda t: t["created_at"],
"name": lambda t: (t.get("name") or "").lower(),
"provider": lambda t: (t.get("provider") or "").lower(),
"model": lambda t: (t.get("model") or "").lower(),
"status": lambda t: t.get("status") or "",
"samples": lambda t: (t.get("summary") or {}).get("samples_ok"),
"ttft": lambda t: (t.get("summary") or {}).get("avg_ttft_ms"),
"prefill": lambda t: (t.get("summary") or {}).get("avg_prefill_speed"),
"decode": lambda t: (t.get("summary") or {}).get("avg_decode_speed"),
"total_ms": lambda t: (t.get("summary") or {}).get("avg_total_ms"),
}
def query_tests(q="", status="", provider="", sort="id", order="desc",
page=1, page_size=20):
"""历史记录:关键词搜索 + 状态/提供商筛选 + 字段排序 + 分页。
返回 {"items": [...], "total": N, "page": p, "page_size": s, "total_pages": n}"""
with _lock:
conn = _connect()
try:
where, args = [], []
if q:
like = "%" + q + "%"
where.append("(id LIKE ? OR name LIKE ? OR model LIKE ? OR provider LIKE ?"
" OR status LIKE ? OR created_at LIKE ?)")
args += [like] * 6
if status:
sts = [s.strip() for s in status.split(",") if s.strip()]
if sts:
where.append("status IN (%s)" % ",".join("?" * len(sts)))
args += sts
if provider:
where.append("provider = ?")
args.append(provider)
wsql = ("WHERE " + " AND ".join(where)) if where else ""
rows = conn.execute(
"SELECT id,created_at,status,provider,model,name,summary_json,gen_cfg_json,error "
"FROM tests ORDER BY id DESC LIMIT ?", (limit,)).fetchall()
"FROM tests " + wsql + " ORDER BY id DESC", args).fetchall()
out = []
for r in rows:
d = dict(r)
d["summary"] = json.loads(d.pop("summary_json") or "{}")
d["gen"] = json.loads(d.pop("gen_cfg_json") or "{}")
out.append(d)
return out
# Python 侧排序:None 值恒排最后(无论升降序)
keyfn = SORT_KEYS.get(sort) or SORT_KEYS["id"]
present = [t for t in out if keyfn(t) is not None]
absent = [t for t in out if keyfn(t) is None]
present.sort(key=keyfn, reverse=(order == "desc"))
out = present + absent
total = len(out)
total_pages = max(1, math.ceil(total / page_size))
page = min(max(1, page), total_pages)
start = (page - 1) * page_size
return {
"items": out[start:start + page_size],
"total": total,
"page": page,
"page_size": page_size,
"total_pages": total_pages,
}
finally:
conn.close()
def clear_tests():
"""一键清空全部历史(测试记录 + 采样指标 + 日志)。
真正清空:删除全部行 + 重置自增序列 + WAL 检查点 + VACUUM 物理回收文件空间,
数据不可恢复(不是只从列表里移除,文件里也不再残留)。"""
with _lock:
conn = _connect()
try:
conn.execute("DELETE FROM test_runs")
conn.execute("DELETE FROM logs")
conn.execute("DELETE FROM tests")
# 重置自增序列:清空后新测试 ID 从 1 重新开始
conn.execute("DELETE FROM sqlite_sequence WHERE name IN ('tests','test_runs','logs')")
conn.commit()
# WAL 检查点截断 + VACUUM:把文件里已删除的数据页物理抹掉、文件瘦身
conn.execute("PRAGMA wal_checkpoint(TRUNCATE)")
conn.execute("VACUUM")
finally:
conn.close()
@@ -229,11 +303,39 @@ def delete_test(tid: int):
conn.execute("DELETE FROM tests WHERE id=?", (tid,))
conn.execute("DELETE FROM test_runs WHERE test_id=?", (tid,))
conn.execute("DELETE FROM logs WHERE test_id=?", (tid,))
# 全部删光时顺带重置自增序列
n = conn.execute("SELECT COUNT(*) FROM tests").fetchone()[0]
if n == 0:
conn.execute("DELETE FROM sqlite_sequence WHERE name IN ('tests','test_runs','logs')")
conn.commit()
finally:
conn.close()
def export_tests():
"""导出全部历史:每条含 基本字段 + 配置(Key打码) + 生成参数 + 摘要指标,
不含逐条采样与完整日志(避免文件过大)"""
with _lock:
conn = _connect()
try:
rows = conn.execute(
"SELECT id,created_at,status,provider,model,name,config_json,gen_cfg_json,summary_json,error "
"FROM tests ORDER BY id").fetchall()
out = []
for r in rows:
d = dict(r)
cfg = json.loads(d.pop("config_json") or "{}")
if cfg.get("api_key"):
cfg["api_key"] = "******"
d["config"] = cfg
d["gen"] = json.loads(d.pop("gen_cfg_json") or "{}")
d["summary"] = json.loads(d.pop("summary_json") or "{}")
out.append(d)
return out
finally:
conn.close()
# ───────────────────────── 采样指标 ─────────────────────────
def add_run(tid: int, run_index: int, metrics: dict, error: str = "", context_length: int = 0):
+46 -5
View File
@@ -63,6 +63,10 @@ def _metrics(start, first_token_at, end, prompt_tokens, output_tokens,
total_ms = (end - start) * 1000
prefill = (prompt_tokens / (ttft_ms / 1000)) if prompt_tokens and ttft_ms > 0 else None
decode = (output_tokens / (decode_ms / 1000)) if output_tokens and decode_ms > 0 else None
# 防“幻影 token”:usage 报了输出 token,但整个流没收到任何文本(如推理全用满 max_tokens
# 或提供商只回 usage 不正文),此时解码时间无意义,不报出数百万的荒谬速度
if output_tokens and decode is not None and not output_chars:
decode = None
return {
"prompt_tokens": int(prompt_tokens or 0),
"output_tokens": int(output_tokens or 0),
@@ -112,7 +116,7 @@ def stream_openai(cfg, prompt, gen, log, should_stop=None):
if should_stop and should_stop():
raise StopRequested()
resp = requests.post(url, json=build_payload(), headers=headers, stream=True,
timeout=(config.CONNECT_TIMEOUT, config.STREAM_READ_TIMEOUT))
timeout=_timeouts(gen))
if resp.status_code == 200:
break
err = resp.text[:400]
@@ -132,8 +136,9 @@ def stream_openai(cfg, prompt, gen, log, should_stop=None):
event_count += 1
if obj.get("choices"):
delta = obj["choices"][0].get("delta") or {}
# 兼容推理型模型:Qwen3/DeepSeek 思维链在 reasoning_content
piece = delta.get("content") or delta.get("reasoning_content") or ""
# 兼容推理型模型思维链DeepSeek reasoning_content,部分 vLLM(Qwen3.8-27B-FP8) 用 reasoningAnthropic/Gemini 分别在各自适配器处理
piece = (delta.get("content") or delta.get("reasoning_content")
or delta.get("reasoning") or "")
if piece:
if first_token_at is None:
first_token_at = time.time()
@@ -147,6 +152,10 @@ def stream_openai(cfg, prompt, gen, log, should_stop=None):
cached_tokens = details.get("cached_tokens") or 0
except StopRequested:
raise
except requests.exceptions.ReadTimeout:
raise ProviderError(_read_timeout_err(gen))
except requests.exceptions.ConnectTimeout:
raise ProviderError(_connect_timeout_err(gen))
except Exception as e:
raise ProviderError("流式请求异常: %s" % e)
finally:
@@ -166,6 +175,30 @@ def _bad_stream_options(err: str):
or "additional properties" in err)
def _timeouts(gen):
"""从测试参数 gen 里取超时配置(秒),未配置则用全局默认。
返回 (connect, read) 元组,供 requests timeout 使用。"""
try:
connect = float(gen.get("connect_timeout") or config.CONNECT_TIMEOUT)
except (TypeError, ValueError):
connect = config.CONNECT_TIMEOUT
try:
read = float(gen.get("read_timeout") or config.STREAM_READ_TIMEOUT)
except (TypeError, ValueError):
read = config.STREAM_READ_TIMEOUT
return (max(connect, 5), max(read, 10))
def _read_timeout_err(gen):
"""读超时的友好报错:提示这是等待数据超时,可调大请求超时"""
return ("等待响应超时:超过 %.0f 秒未收到数据(接口预处理/首字耗时过长或网络过慢),"
"请在「速度测试配置」中调大“请求超时(秒)”" % _timeouts(gen)[1])
def _connect_timeout_err(gen):
return "连接超时:超过 %.0f 秒未建立连接(请检查地址/网络)" % _timeouts(gen)[0]
def _iter_json(resp):
"""解析 SSE data: 行,逐个返回 JSON 对象"""
for raw in resp.iter_lines(decode_unicode=True):
@@ -201,7 +234,7 @@ def stream_anthropic(cfg, prompt, gen, log, should_stop=None):
if should_stop and should_stop():
raise StopRequested()
resp = requests.post(url, json=payload, headers=headers, stream=True,
timeout=(config.CONNECT_TIMEOUT, config.STREAM_READ_TIMEOUT))
timeout=_timeouts(gen))
if resp.status_code != 200:
err = resp.text[:400]
resp.close()
@@ -228,6 +261,10 @@ def stream_anthropic(cfg, prompt, gen, log, should_stop=None):
output_tokens = usage.get("output_tokens") or output_tokens
except StopRequested:
raise
except requests.exceptions.ReadTimeout:
raise ProviderError(_read_timeout_err(gen))
except requests.exceptions.ConnectTimeout:
raise ProviderError(_connect_timeout_err(gen))
except Exception as e:
raise ProviderError("流式请求异常: %s" % e)
finally:
@@ -267,7 +304,7 @@ def stream_google(cfg, prompt, gen, log, should_stop=None):
if should_stop and should_stop():
raise StopRequested()
resp = requests.post(url, params=params, json=payload, headers=headers, stream=True,
timeout=(config.CONNECT_TIMEOUT, config.STREAM_READ_TIMEOUT))
timeout=_timeouts(gen))
if resp.status_code != 200:
err = resp.text[:400]
resp.close()
@@ -294,6 +331,10 @@ def stream_google(cfg, prompt, gen, log, should_stop=None):
cached_tokens = um.get("cachedContentTokenCount") or 0
except StopRequested:
raise
except requests.exceptions.ReadTimeout:
raise ProviderError(_read_timeout_err(gen))
except requests.exceptions.ConnectTimeout:
raise ProviderError(_connect_timeout_err(gen))
except Exception as e:
raise ProviderError("流式请求异常: %s" % e)
finally:
+65 -2
View File
@@ -43,17 +43,44 @@ body {
/* ── 布局 ── */
.layout { display: grid; grid-template-columns: 340px 1fr; gap: 16px; padding: 16px 24px; align-items: start; }
.config-panel { display: flex; flex-direction: column; gap: 16px; position: sticky; top: 72px; }
.config-panel {
display: flex; flex-direction: column; gap: 16px;
position: sticky; top: 72px;
/* 左侧独立滚动:面板固定视口高度,鼠标在左侧滚动时只翻左侧几个块,不带动右侧 */
max-height: calc(100vh - 88px);
overflow-y: auto;
overscroll-behavior: contain;
padding-right: 6px;
scrollbar-width: thin;
scrollbar-color: var(--border) transparent;
}
.config-panel::-webkit-scrollbar { width: 8px; }
.config-panel::-webkit-scrollbar-thumb { background: var(--border); border-radius: 4px; }
.config-panel::-webkit-scrollbar-thumb:hover { background: var(--accent); }
.main { display: flex; flex-direction: column; gap: 16px; min-width: 0; }
.card {
background: var(--panel); border: 1px solid var(--border);
border-radius: 12px; padding: 16px;
}
/* ── 可折叠卡片(左侧三块) ── */
.card h2 { font-size: 15px; margin-bottom: 14px; }
.collapse-head {
display: flex; align-items: center; justify-content: space-between;
cursor: pointer; user-select: none;
}
.collapse-head:hover { color: var(--accent); }
.collapse-arrow {
font-size: 12px; color: var(--muted); transition: transform .15s;
}
.card.collapsed .collapse-arrow { transform: rotate(-90deg); }
.card.collapsed .collapse-body { display: none; }
/* ── 卡片头部 ── */
.card-head { display: flex; align-items: center; justify-content: space-between; margin-bottom: 14px; }
.card-head h2 { margin-bottom: 0; }
/* ── 表单 ── */
.field { margin-bottom: 12px; }
.field label { display: block; font-size: 12px; color: var(--muted); margin-bottom: 5px; }
@@ -184,6 +211,17 @@ body {
.console .ln.sys { color: #5b6b8c; font-style: italic; }
/* ── 历史表 ── */
.hist-filters {
display: flex; flex-wrap: wrap; gap: 6px; align-items: center;
margin-bottom: 10px;
}
.hist-filters select {
background: var(--panel2); color: var(--text); border: 1px solid var(--border);
border-radius: 8px; padding: 7px 10px; font-size: 12.5px; outline: none;
}
.hist-filters select:focus { border-color: var(--accent); }
.hist-filters .icon-input { min-width: 160px; }
.table-wrap { overflow-x: auto; }
table.history { width: 100%; border-collapse: collapse; font-size: 13px; }
.history th, .history td { padding: 8px 10px; text-align: left; border-bottom: 1px solid var(--border); white-space: nowrap; }
@@ -192,6 +230,17 @@ table.history { width: 100%; border-collapse: collapse; font-size: 13px; }
.history td.num { font-family: var(--mono); }
.history .chk-col { width: 34px; text-align: center; }
.history .chk-col input { width: 15px; height: 15px; cursor: pointer; accent-color: var(--accent); }
.history th.sortable { cursor: pointer; user-select: none; }
.history th.sortable:hover { color: var(--accent); }
.history th.sortable.asc::after { content: " ▲"; font-size: 10px; }
.history th.sortable.desc::after { content: " ▼"; font-size: 10px; }
/* 历史分页条 */
.hist-pager {
display: flex; align-items: center; justify-content: center; gap: 14px;
margin-top: 12px; padding-top: 10px; border-top: 1px solid var(--border);
}
.hist-pager .btn:disabled { opacity: .4; cursor: not-allowed; }
.status-pill { padding: 2px 10px; border-radius: 12px; font-size: 11px; }
.status-pill.running { background: rgba(79,140,255,.15); color: var(--accent); }
.status-pill.done { background: rgba(34,197,139,.15); color: var(--accent2); }
@@ -257,8 +306,22 @@ pre.json-box {
padding: 10px; font: 11.5px/1.6 var(--mono); overflow: auto; max-height: 260px; color: #a8f0d0;
}
/* ── 右下角一键置顶/置底(高对比配色,易于发现) ── */
.scroll-fab {
position: fixed; right: 20px; bottom: 20px;
display: flex; flex-direction: column; gap: 8px; z-index: 60;
}
.fab {
width: 44px; height: 44px; border-radius: 50%; border: none;
background: linear-gradient(135deg, #5a92ff, #2f6bff);
color: #fff; font-size: 18px; cursor: pointer;
box-shadow: 0 6px 18px rgba(47,107,255,.55); transition: .15s; line-height: 1;
}
.fab:hover { filter: brightness(1.15); transform: translateY(-1px); }
.fab:active { transform: translateY(0); }
@media (max-width: 1000px) {
.layout { grid-template-columns: 1fr; }
.config-panel { position: static; }
.config-panel { position: static; max-height: none; overflow: visible; }
.metrics { grid-template-columns: repeat(3, 1fr); }
}
+66 -9
View File
@@ -19,8 +19,9 @@
<main class="layout">
<!-- 左侧:配置 -->
<aside class="config-panel">
<section class="card">
<h2>🔌 大模型接口配置</h2>
<section class="card collapsible" id="sec-interface">
<h2 class="collapse-head">🔌 大模型接口配置<span class="collapse-arrow"></span></h2>
<div class="collapse-body">
<div class="field">
<label>提供商类型</label>
<select id="cfg-provider">
@@ -75,10 +76,12 @@
</div>
<button class="btn block" id="btn-test-conn">🔍 测试连接</button>
<div class="conn-result" id="conn-result" hidden></div>
</div>
</section>
<section class="card">
<h2>🚀 速度测试配置</h2>
<section class="card collapsible" id="sec-test">
<h2 class="collapse-head">🚀 速度测试配置<span class="collapse-arrow"></span></h2>
<div class="collapse-body">
<div class="field">
<label>测试名称(主题)</label>
<input id="gen-name" type="text" placeholder="如:DeepSeek-V4 不同上下文长度速度对比">
@@ -122,13 +125,20 @@
<span class="slider"></span>
</label>
</div>
<div class="field">
<label>请求超时(秒)<span class="hint-inline">等待首字/预处理上限,接口慢就调大</span></label>
<div class="icon-input"><span class="icon"></span><input id="gen-timeout" type="number" min="30" step="30" value="1800"></div>
<div class="hint">连接测试与速度采样共用:超过此时长仍未收到数据才判定超时(默认 1800 秒 = 30 分钟)。测慢接口/超长上下文时建议调大,如 3600。</div>
</div>
<div class="btn-group">
<button class="btn primary block" id="btn-start">▶ 开始测试</button>
<button class="btn danger block" id="btn-cancel" disabled>■ 停止</button>
</div>
</div>
</section>
<section class="card">
<h2>📤 评测站同步</h2>
<section class="card collapsible" id="sec-eval">
<h2 class="collapse-head">📤 评测站同步<span class="collapse-arrow"></span></h2>
<div class="collapse-body">
<div class="field">
<label>评测站地址</label>
<div class="icon-input"><span class="icon">🌐</span><input id="eval-url" placeholder="http://127.0.0.1:16066"></div>
@@ -147,6 +157,7 @@
<button class="btn block" id="btn-eval-test">🔗 测试连接</button>
<div class="conn-result" id="eval-result" hidden></div>
<div class="hint" style="margin-top:6px">测试完成后,在历史列表或详情里点「📤 发送到评测站」,一键把结果发布到模型评测网站(端口 16066)对应账号下。</div>
</div>
</section>
</aside>
@@ -175,23 +186,63 @@
<div class="card">
<div class="card-head">
<h2>🗂 测试历史</h2>
<h2>🗂 测试历史 <span class="hint" id="hist-total"></span></h2>
<div class="log-actions">
<span class="hint" id="hist-selected"></span>
<button class="btn small primary" id="btn-compare">⚖️ 对比所选</button>
<button class="btn small" id="btn-export-history" title="一键导出全部历史记录(JSON">📥 导出全部</button>
<button class="btn small danger" id="btn-clear-history" title="一键清空全部历史记录(不可恢复)">🗑 清空历史</button>
<button class="btn small" id="btn-refresh-history">刷新</button>
</div>
</div>
<div class="hist-filters">
<div class="icon-input" style="flex:2;min-width:160px"><span class="icon">🔍</span><input id="hist-q" placeholder="搜索:编号/名称/模型/提供商/状态/时间"></div>
<select id="hist-status" title="按状态筛选">
<option value="">状态:全部</option>
<option value="running">测试中</option>
<option value="done">完成</option>
<option value="error">出错</option>
<option value="canceled">已取消</option>
</select>
<select id="hist-provider" title="按提供商筛选">
<option value="">提供商:全部</option>
<option value="openai">OpenAI 兼容</option>
<option value="anthropic">Anthropic</option>
<option value="google">Google Gemini</option>
</select>
<select id="hist-size" title="每页条数">
<option value="10">10 条/页</option>
<option value="20" selected>20 条/页</option>
<option value="50">50 条/页</option>
<option value="100">100 条/页</option>
</select>
<button class="btn small" id="hist-reset" title="清空筛选条件并回到第一页">✕ 重置</button>
</div>
<div class="table-wrap">
<table class="history" id="history">
<thead><tr>
<th class="chk-col"><input type="checkbox" id="hist-check-all" title="全选"></th>
<th>#</th><th>时间</th><th>名称</th><th>提供商</th><th>模型</th><th>并发</th><th>采样</th>
<th>首字 ms</th><th>预填充 tok/s</th><th>解码 tok/s</th><th>状态</th><th>操作</th>
<th class="sortable" data-sort="id" title="点击排序">#</th>
<th class="sortable" data-sort="created_at" title="点击排序">时间</th>
<th class="sortable" data-sort="name" title="点击排序">名称</th>
<th class="sortable" data-sort="provider" title="点击排序">提供商</th>
<th class="sortable" data-sort="model" title="点击排序">模型</th>
<th>并发</th>
<th class="sortable" data-sort="samples" title="点击排序">采样</th>
<th class="sortable" data-sort="ttft" title="点击排序">首字 ms</th>
<th class="sortable" data-sort="prefill" title="点击排序">预填充 tok/s</th>
<th class="sortable" data-sort="decode" title="点击排序">解码 tok/s</th>
<th class="sortable" data-sort="status" title="点击排序">状态</th>
<th>操作</th>
</tr></thead>
<tbody></tbody>
</table>
</div>
<div class="hist-pager">
<button class="btn small" id="hist-prev">◀ 上一页</button>
<span class="hint" id="hist-page-info"></span>
<button class="btn small" id="hist-next">下一页 ▶</button>
</div>
</div>
</section>
</main>
@@ -223,6 +274,12 @@
</div>
</div>
<!-- 右下角:一键置顶 / 置底 -->
<div class="scroll-fab">
<button class="fab" id="fab-top" title="回到顶部"></button>
<button class="fab" id="fab-bottom" title="滚到底部"></button>
</div>
<script src="/js/app.js"></script>
</body>
</html>
+183 -32
View File
@@ -69,6 +69,7 @@ function currentGen() {
concurrency_levels: [...concurrencyLevelsActive].sort((a, b) => a - b),
avoid_cache: $("#gen-avoid-cache").checked,
warmup: $("#gen-warmup").checked,
read_timeout: parseInt($("#gen-timeout").value) || 1800,
};
}
@@ -201,9 +202,11 @@ function loadSelectedConfig() {
async function testConnection() {
const cfg = currentConfig();
if (!cfg.api_key) { showConn(false, "请先填写 API Key"); return; }
// 慢接口长预处理可能等首字很久,连接测试也带上请求超时,避免误判接口不通
cfg.read_timeout = parseInt($("#gen-timeout").value) || 1800;
const box = $("#conn-result");
box.hidden = false; box.className = "conn-result";
box.textContent = "⏳ 正在测试连接...";
box.textContent = "⏳ 正在测试连接(等待首字,最长约 " + cfg.read_timeout + " 秒)...";
try {
const r = await api("/api/configs/test", "POST", cfg);
if (r.ok) {
@@ -254,7 +257,8 @@ async function listModels() {
const box = $("#model-result");
box.hidden = false;
box.className = "conn-result fail";
box.textContent = "❌ 获取模型失败:" + e.message;
box.textContent = "❌ 接口不通,请检查 Base URL / API Key 配置后重试";
console.warn("[listModels] 获取模型失败(已隐藏原始错误):", e);
}
}
@@ -422,6 +426,7 @@ function stopTest() {
/* ───────────────────────── 测试历史 ───────────────────────── */
let histSelected = new Set(); // 勾选用于对比的测试 id
const histState = { page: 1, pageSize: 20, q: "", status: "", provider: "", sort: "id", order: "desc" };
function updateHistSelectedLabel() {
const el = $("#hist-selected");
@@ -429,43 +434,74 @@ function updateHistSelectedLabel() {
}
async function loadHistory() {
const list = await api("/api/tests");
const p = new URLSearchParams({
page: histState.page, page_size: histState.pageSize,
q: histState.q, status: histState.status, provider: histState.provider,
sort: histState.sort, order: histState.order,
});
let d;
try {
d = await api("/api/tests?" + p.toString());
} catch (e) { return; }
if (!d || !d.ok) return;
renderHistory(d);
}
function renderHistory(d) {
const list = d.items || [];
const tb = $("#history tbody");
tb.innerHTML = "";
const selAll = $("#hist-check-all");
if (selAll) selAll.checked = false;
const hasFilter = histState.q || histState.status || histState.provider;
$("#hist-total").textContent = d.total ? `(共 ${d.total} 条)` : "";
if (!list.length) {
tb.innerHTML = '<tr><td colspan="13" style="color:var(--muted);text-align:center">暂无测试记录</td></tr>';
tb.innerHTML = `<tr><td colspan="13" style="color:var(--muted);text-align:center">暂无${hasFilter ? "匹配的" : ""}测试记录${hasFilter ? "(试试调整筛选条件)" : ""}</td></tr>`;
updateHistSelectedLabel();
return;
}
for (const t of list) {
const s = t.summary || {};
const g = t.gen || {};
const cls = s.concurrency_levels || g.concurrency_levels || [1];
const tr = document.createElement("tr");
tr.innerHTML = `
<td class="chk-col"><input type="checkbox" class="hist-chk" data-id="${t.id}" ${histSelected.has(t.id) ? "checked" : ""}></td>
<td>#${t.id}</td>
<td>${esc(t.created_at)}</td>
<td title="${esc(t.name || "")}">${esc(t.name || "—")}</td>
<td>${esc(PROVIDER_LABEL[t.provider] || t.provider)}</td>
<td>${esc(t.model)}</td>
<td class="num">${esc(cls.join("/"))}</td>
<td class="num">${fmt(s.samples_ok)}/${fmt(s.samples_total)}</td>
<td class="num">${fmt(s.avg_ttft_ms)}</td>
<td class="num">${fmt(s.avg_prefill_speed)}</td>
<td class="num">${fmt(s.avg_decode_speed)}</td>
<td><span class="status-pill ${esc(t.status)}">${STATUS_LABEL[t.status] || t.status}</span></td>
<td>
<button class="btn small" data-view="${t.id}">查看</button>
<button class="btn small" data-send="${t.id}" title="发送到评测站对应账号下">📤 发送</button>
<button class="btn small" data-xlsx="${t.id}">Excel</button>
<button class="btn small danger" data-del="${t.id}">删除</button>
</td>`;
tb.appendChild(tr);
} else {
for (const t of list) {
const s = t.summary || {};
const g = t.gen || {};
const cls = s.concurrency_levels || g.concurrency_levels || [1];
const tr = document.createElement("tr");
tr.innerHTML = `
<td class="chk-col"><input type="checkbox" class="hist-chk" data-id="${t.id}" ${histSelected.has(t.id) ? "checked" : ""}></td>
<td>#${t.id}</td>
<td>${esc(t.created_at)}</td>
<td title="${esc(t.name || "")}">${esc(t.name || "—")}</td>
<td>${esc(PROVIDER_LABEL[t.provider] || t.provider)}</td>
<td>${esc(t.model)}</td>
<td class="num">${esc(cls.join("/"))}</td>
<td class="num">${fmt(s.samples_ok)}/${fmt(s.samples_total)}</td>
<td class="num">${fmt(s.avg_ttft_ms)}</td>
<td class="num">${fmt(s.avg_prefill_speed)}</td>
<td class="num">${fmt(s.avg_decode_speed)}</td>
<td><span class="status-pill ${esc(t.status)}">${STATUS_LABEL[t.status] || t.status}</span></td>
<td>
<button class="btn small" data-view="${t.id}">查看</button>
<button class="btn small" data-send="${t.id}" title="发送到评测站对应账号下">📤 发送</button>
<button class="btn small" data-xlsx="${t.id}">Excel</button>
<button class="btn small danger" data-del="${t.id}">删除</button>
</td>`;
tb.appendChild(tr);
}
}
updateHistSelectedLabel();
// 分页信息
const totalPages = Math.max(1, d.total_pages || 1);
$("#hist-page-info").textContent = `${d.page} / ${totalPages} 页 · 共 ${d.total}`;
$("#hist-prev").disabled = d.page <= 1;
$("#hist-next").disabled = d.page >= totalPages;
histState.page = d.page;
updateSortIndicators();
}
function updateSortIndicators() {
$$("#history thead th.sortable").forEach((th) => {
th.classList.remove("asc", "desc");
if (th.dataset.sort === histState.sort) th.classList.add(histState.order);
});
}
/* ───────────────────────── 多测试对比 ───────────────────────── */
@@ -1064,8 +1100,72 @@ function bind() {
$("#btn-clear-console").addEventListener("click", clearConsole);
$("#btn-export-log").addEventListener("click", exportCurrentLog);
$("#btn-refresh-history").addEventListener("click", loadHistory);
$("#btn-refresh-history").addEventListener("click", () => { histState.page = 1; loadHistory(); });
$("#btn-compare").addEventListener("click", openCompare);
$("#btn-export-history").addEventListener("click", exportAllHistory);
$("#btn-clear-history").addEventListener("click", clearAllHistory);
// 左侧三块:点击标题折叠/展开(状态存 localStorage
const COLLAPSE_KEY = "llmst.collapsed";
let collapsedSections = new Set();
try { collapsedSections = new Set(JSON.parse(localStorage.getItem(COLLAPSE_KEY) || "[]")); } catch (e) {}
document.querySelectorAll(".card.collapsible").forEach((sec) => {
if (collapsedSections.has(sec.id)) sec.classList.add("collapsed");
const head = sec.querySelector(".collapse-head");
if (head) head.addEventListener("click", () => {
const on = sec.classList.toggle("collapsed");
if (on) collapsedSections.add(sec.id); else collapsedSections.delete(sec.id);
try { localStorage.setItem(COLLAPSE_KEY, JSON.stringify([...collapsedSections])); } catch (e) {}
});
});
// 历史:筛选 / 搜索 / 每页条数 / 分页 / 重置
let histQTimer = null;
$("#hist-q").addEventListener("input", () => {
clearTimeout(histQTimer);
histQTimer = setTimeout(() => {
histState.q = $("#hist-q").value.trim();
histState.page = 1;
loadHistory();
}, 300);
});
$("#hist-status").addEventListener("change", () => {
histState.status = $("#hist-status").value;
histState.page = 1;
loadHistory();
});
$("#hist-provider").addEventListener("change", () => {
histState.provider = $("#hist-provider").value;
histState.page = 1;
loadHistory();
});
$("#hist-size").addEventListener("change", () => {
histState.pageSize = parseInt($("#hist-size").value) || 20;
histState.page = 1;
loadHistory();
});
$("#hist-prev").addEventListener("click", () => {
if (histState.page > 1) { histState.page--; loadHistory(); }
});
$("#hist-next").addEventListener("click", () => {
if (!$("#hist-next").disabled) { histState.page++; loadHistory(); }
});
$("#hist-reset").addEventListener("click", resetHistoryFilters);
// 历史:点击表头升降排序
$("#history thead").addEventListener("click", (e) => {
const th = e.target.closest("th.sortable");
if (!th) return;
const key = th.dataset.sort;
if (histState.sort === key) {
histState.order = histState.order === "asc" ? "desc" : "asc";
} else {
histState.sort = key;
histState.order = (key === "id" || key === "created_at") ? "desc" : "asc";
}
histState.page = 1;
loadHistory();
});
$("#history tbody").addEventListener("click", (e) => {
const v = e.target.closest("[data-view]");
@@ -1108,6 +1208,57 @@ function bind() {
$("#dt-send-eval").addEventListener("click", () => { if (window.__detail) sendToEval(window.__detail.id); });
document.addEventListener("keydown", (e) => { if (e.key === "Escape") { hideModelPick(); closeDetail(); closeCompare(); } });
// 右下角一键置顶 / 置底
$("#fab-top").addEventListener("click", () => window.scrollTo({ top: 0, behavior: "smooth" }));
$("#fab-bottom").addEventListener("click", () =>
window.scrollTo({ top: document.documentElement.scrollHeight, behavior: "smooth" }));
}
/* ───────────────────────── 历史筛选工具 ───────────────────────── */
function resetHistoryFilters() {
histState.q = ""; histState.status = ""; histState.provider = "";
histState.sort = "id"; histState.order = "desc";
histState.page = 1;
$("#hist-q").value = "";
$("#hist-status").value = "";
$("#hist-provider").value = "";
loadHistory();
}
async function clearAllHistory() {
if (!confirm("确定清空全部测试历史?\n所有测试记录、采样数据与日志将被永久删除,无法恢复!")) return;
if (!confirm("再次确认:真的要清空全部历史记录吗?")) return;
const r = await api("/api/tests", "DELETE");
if (r.ok) {
histSelected.clear();
histState.page = 1;
toast("✅ 历史记录已全部清空");
loadHistory();
updateHistSelectedLabel();
} else {
toast("❌ 清空失败:" + (r.error || "未知错误"));
}
}
async function exportAllHistory() {
try {
const resp = await fetch("/api/tests/export");
if (!resp.ok) throw new Error("HTTP " + resp.status);
const blob = await resp.blob();
const cd = resp.headers.get("Content-Disposition") || "";
const m = cd.match(/filename\"?=?\"?([^";]+)\"?/);
const name = m ? m[1] : "llm_speed_tests.json";
const url = URL.createObjectURL(blob);
const a = document.createElement("a");
a.href = url; a.download = name;
document.body.appendChild(a); a.click(); a.remove();
URL.revokeObjectURL(url);
toast("✅ 已导出全部历史(共 " + (blob.size / 1024).toFixed(1) + " KB");
} catch (e) {
toast("❌ 导出失败:" + e.message);
}
}
/* ───────────────────────── 初始化 ───────────────────────── */
+13 -2
View File
@@ -77,6 +77,9 @@ class TestRunner(threading.Thread):
% (" / ".join(str(x) for x in lengths), max_tokens,
" / ".join(str(x) for x in concurrency_levels), n,
"" if warmup else "", "" if avoid_cache else ""))
to = self.gen.get("read_timeout") or "默认(1800)"
self.log("INFO", "请求超时: 连接 %s s | 等待首字/预处理 %s s(接口慢可在左侧调大)"
% (self.gen.get("connect_timeout") or 60, to))
ratio = self._calibrate()
self.ratio = ratio
@@ -157,7 +160,9 @@ class TestRunner(threading.Thread):
"""
from concurrent.futures import ThreadPoolExecutor
gen_opt = {"max_tokens": max_tokens, "avoid_cache": avoid_cache}
gen_opt = {"max_tokens": max_tokens, "avoid_cache": avoid_cache,
"connect_timeout": self.gen.get("connect_timeout"),
"read_timeout": self.gen.get("read_timeout")}
def worker(idx):
prompt = self._finalize_prompt(base_prompt) # 每流独立随机前缀,避免共享缓存
@@ -200,6 +205,10 @@ class TestRunner(threading.Thread):
total_ms = max((batch_end - batch_start) * 1000.0, 0.1)
prefill = (total_prompt / (ttft_ms / 1000.0)) if total_prompt else None
decode = (total_output / (decode_ms / 1000.0)) if total_output else None
# 防“幻影 token”:所有流都没收到任何流式文本但 usage 报了输出 token(如推理用满 max_tokens),
# 整批解码时间无意义,不报数百万的荒谬解码速度
if total_output and decode is not None and not total_ochars:
decode = None
agg = {
"concurrency": concurrency,
@@ -236,7 +245,9 @@ class TestRunner(threading.Thread):
self.log("INFO", "正在校准 token/字符 比例(发送小探测请求)...")
try:
m = lp.call_stream(self.cfg, probe,
{"max_tokens": 8, "avoid_cache": False},
{"max_tokens": 8, "avoid_cache": False,
"connect_timeout": self.gen.get("connect_timeout"),
"read_timeout": self.gen.get("read_timeout")},
log=lambda lv, msg: self.log(lv, msg),
should_stop=self.should_stop)
pt = m.get("prompt_tokens") or 0