diff --git a/README.md b/README.md index e94f771..19bd6d8 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,8 @@ -# 大模型API中转系统 +# 大模型API中转系统 (LLM Proxy) -> 兼容OpenAI API格式的多提供商代理系统，支持优先级自动切换 +> 兼容OpenAI API格式的多提供商代理系统，支持能力(Capability)路由、优先级自动切换 + +**版本：v2.1.0** ## 功能特点 @@ -9,21 +11,27 @@ - 按优先级自动选择可用提供商 - 故障自动切换到备用提供商 -### 📡 OpenAI API 兼容 -- 完全兼容 OpenAI API 格式 -- 支持 Chat Completions API -- 支持 Embeddings API -- 支持流式和非流式响应 +### 🎯 能力路由（v2.1.0 新增） +每个模型可标记**能力标签**，AUTO配置**固定绑定一个功能类型**并自动筛选具备该能力的模型： -### 🎯 智能路由 -- `auto` 模型自动选择可用提供商 +| 能力 | 说明 | 端点 | +|------|------|------| +| `text` 文本推理 | 纯文本对话 | `/v1/chat/completions` | +| `vision` 视觉能力 | 多模态图像理解 | `/v1/chat/completions`（消息含图片自动路由） | +| `audio_out` 语音输出 | TTS 语音合成 | `/v1/audio/speech` | +| `audio_in` 语音输入 | ASR 语音识别 | `/v1/audio/transcriptions` | +| `image_gen` 图片生成 | 文生图 | `/v1/images/generations` | +| `video_gen` 视频生成 | 文生视频 | `/v1/video/generations` | + +### 📡 OpenAI API 兼容 +- 完全兼容 OpenAI API 格式（Chat / Embeddings / Images / Audio） +- 支持流式和非流式响应 - 支持模型别名映射 -- 请求参数自动适配 ### 🛡️ 高可用 - 自动健康检查 -- 错误计数与熔断 -- 自动重试机制 +- 错误计数与熔断（连续失败3次熔断，**冷却期后自动恢复**） +- 自动重试（失败切换到下一个托管同一模型的提供商，**不改变用户请求的模型**） ## 快速开始 @@ -36,13 +44,18 @@ pip install -r requirements.txt ### 启动服务 ```bash +./start.sh # 后台启动（PID 管理） +./start.sh stop # 停止 +./start.sh status # 状态 +# 或前台运行 python app.py ``` ### 访问地址 ``` -http://localhost:19007 +前台API: http://localhost:16003/v1/chat/completions +后台管理: http://localhost:16003/admin ``` ## API 使用 @@ -50,9 +63,9 @@ http://localhost:19007 ### Chat Completions ```bash -curl http://localhost:19007/v1/chat/completions \ +curl http://localhost:16003/v1/chat/completions \ -H "Content-Type: application/json" \ - -H "Authorization: Bearer any-key" \ + -H "Authorization: Bearer ***" \ -d '{ "model": "auto", "messages": [{"role": "user", "content": "Hello!"}], @@ -60,27 +73,62 @@ curl http://localhost:19007/v1/chat/completions \ }' ``` -### 列出模型 +### 列出模型（含能力标签） ```bash -curl http://localhost:19007/v1/models +curl http://localhost:16003/v1/models ``` ### 流式响应 ```bash -curl http://localhost:19007/v1/chat/completions \ +curl http://localhost:16003/v1/chat/completions \ -H "Content-Type: application/json" \ -d '{ - "model": "qwen3.5-4b", - "messages": [{"role": "user", "content": "Hello!"}], + "model": "auto-text", + "messages": [{"role": "user", "content": "讲个笑话"}], "stream": true }' ``` +### 图片生成 + +```bash +curl http://localhost:16003/v1/images/generations \ + -H "Content-Type: application/json" \ + -d '{ + "model": "auto-image", + "prompt": "a cute cat, cartoon style", + "size": "1024x1024" + }' +``` + +## Auto 配置（按能力固定功能） + +每个 Auto 配置固定一个**功能类型**，调用时 `model="配置名称"` 即自动选择具备该能力的模型： + +| 配置名称 | 功能 | 说明 | +|----------|------|------| +| `auto` | 文本推理 | 默认，按优先级自动选择 | +| `auto-text` | 文本推理 | 纯文本模型 | +| `auto-vision` | 视觉能力 | 多模态视觉模型 | +| `auto-image` | 图片生成 | 生图模型 | +| `auto-voice-out` | 语音输出 | TTS 模型 | +| `auto-voice-in` | 语音输入 | ASR 模型 | +| `auto-video` | 视频生成 | 视频模型 | + +可在后台「Auto配置」页创建/修改，从模型管理中选择具备对应能力的模型。 + +## 模型管理 + +后台「模型管理」页： +- 每个模型可配置能力标签（文本/视觉/语音/生图/生视频） +- 支持添加/编辑/删除模型、设置默认模型 +- 支持模型别名管理（`qwen`→`unsloth/Qwen3.8-27B-Q6_K` 等） + ## 配置说明 -编辑 `config/settings.py`: +默认配置在 `config/settings.py`，运行时配置存于 `data/config.json`（可在后台修改）： ```python UPSTREAM_PROVIDERS = [ @@ -89,20 +137,35 @@ UPSTREAM_PROVIDERS = [ "priority": 1, # 优先级，数字越小越高 "base_url": "https://api.example.com/v1", "api_key": "sk-xxx", - "models": ["model-1", "model-2"], + "capabilities": ["text", "vision"], # 提供商能力（新模型默认继承） + "models": [ + {"name": "model-1", "capabilities": ["text"]}, + {"name": "model-2", "capabilities": ["text", "vision"]}, + ], "default_model": "model-1", - "timeout": 120, + "timeout": 180, "enabled": True, }, ] ``` +### 内置提供商 + +| 提供商 | 能力 | 模型 | +|--------|------|------| +| Local Qwen | 文本+视觉 | `unsloth/Qwen3.8-27B-Q6_K`, `unsloth/Qwen3.8-27B-Q4_K_M` | +| SiliconFlow LLM | 文本 | `deepseek-ai/DeepSeek-V4-Flash`, `meituan-longcat/LongCat-2.0` | +| Autodl | 文本+视觉 | `qwen3.6-plus`, `GLM-5.3-flash` | +| Autodl Image | 图片生成 | `Qwen-Image` | + ### 模型别名 ```python MODEL_ALIASES = { - "auto": "auto", # 自动选择 - "gpt-4": "actual-model", # 别名映射 + "auto": "auto", + "qwen": "unsloth/Qwen3.8-27B-Q6_K", + "deepseek": "deepseek-ai/DeepSeek-V4-Flash", + ... } ``` @@ -113,119 +176,37 @@ MODEL_ALIASES = { | `/` | GET | 服务信息 | | `/v1/chat/completions` | POST | 聊天完成 | | `/v1/embeddings` | POST | 文本嵌入 | +| `/v1/images/generations` | POST | 图片生成 | +| `/v1/audio/speech` | POST | 语音合成 | +| `/v1/audio/transcriptions` | POST | 语音识别 | +| `/v1/video/generations` | POST | 视频生成 | | `/v1/models` | GET | 模型列表 | | `/health` | GET | 健康检查 | | `/status` | GET | 详细状态 | +| `/admin` | GET | 后台管理 | -## 使用示例 - -### Python (OpenAI SDK) - -```python -from openai import OpenAI - -client = OpenAI( - base_url="http://localhost:19007/v1", - api_key="any-key" -) - -response = client.chat.completions.create( - model="auto", - messages=[ - {"role": "user", "content": "你好！"} - ] -) - -print(response.choices[0].message.content) -``` - -### 流式响应 - -```python -stream = client.chat.completions.create( - model="qwen3.5-4b", - messages=[{"role": "user", "content": "讲个笑话"}], - stream=True -) - -for chunk in stream: - if chunk.choices[0].delta.content: - print(chunk.choices[0].delta.content, end="") -``` - -## 优先级机制 - -当使用 `model="auto"` 时： +## 优先级与熔断机制 +当使用 `model="auto"` 系列时： 1. 按配置的优先级顺序选择提供商 -2. 跳过不可用的提供商 -3. 请求失败自动切换到下一个提供商 -4. 连续失败3次的提供商暂时标记为不可用 - -## 监控 - -### 健康检查 - -```bash -curl http://localhost:19007/health -``` - -### 详细状态 - -```bash -curl http://localhost:19007/status -``` +2. 跳过不可用/不具备对应能力的提供商 +3. 请求失败自动切换到下一个托管同一模型的提供商（保持请求模型不变） +4. 连续失败3次的提供商被熔断，冷却期（默认60秒）后自动半开恢复 ## 项目结构 ``` llm-proxy/ -├── app.py # 主程序 +├── app.py # 主程序（前台API + 后台管理，单端口） +├── start.sh # 启动/停止脚本 ├── requirements.txt # 依赖 ├── config/ -│ └── settings.py # 配置 -├── logs/ # 日志目录 -└── README.md +│ └── settings.py # 默认配置（提供商/能力/别名/auto） +├── data/ # 运行时配置与数据（config.json/stats.json/chats.json） +├── logs/ # 日志目录 +└── templates/ # 后台管理页面 ``` -## 后台管理 - -后台管理系统提供可视化监控和配置查看。 - -启动后台： - -```bash -python admin/app.py -``` - -后台地址: http://localhost:19008 - -### 功能模块 - -| 模块 | 功能 | -|------|------| -| 仪表盘 | 统计数据、提供商状态、调用流程 | -| 提供商管理 | 查看提供商、测试连接、详情 | -| 模型管理 | 模型别名、目标模型、提供商映射 | -| 日志查看 | 实时日志、自动刷新 | -| 系统配置 | 配置查看 | - -## 版本历史 - -### v0.2.0 (2026-04-08) -- 新增后台管理系统 -- 仪表盘统计 -- 提供商管理 -- 模型管理 -- 日志查看 - -### v0.1.0 (2026-04-08) -- 初始版本 -- 多提供商支持 -- OpenAI API 兼容 -- 优先级自动切换 -- 流式响应支持 - ## License -MIT \ No newline at end of file +MIT diff --git a/admin/app.py b/admin/app.py deleted file mode 100644 index d7a0601..0000000 --- a/admin/app.py +++ /dev/null @@ -1,792 +0,0 @@ -""" -大模型API中转系统 - 后台管理系统 -支持动态添加、编辑、删除提供商和优先级调整 -""" - -from flask import Flask, render_template, jsonify, request -from flask_cors import CORS -import json -import time -import uuid -from datetime import datetime -from pathlib import Path -import sys -import requests - -# 添加父目录到路径 -sys.path.insert(0, str(Path(__file__).parent.parent)) -from config.settings import ( - DEFAULT_PROVIDERS, DEFAULT_MODEL_ALIASES, DEFAULT_AUTO_PROFILES, - SERVER_CONFIG, LOG_CONFIG, RETRY_CONFIG, - load_config, save_config, get_providers, - get_provider, add_provider, update_provider, - delete_provider, update_priority, get_model_aliases, - update_model_alias, get_auto_profiles, get_auto_profile, - add_auto_profile, update_auto_profile, delete_auto_profile -) - -app = Flask(__name__) -CORS(app) - -# 数据目录 -DATA_DIR = Path(__file__).parent.parent / 'data' -DATA_DIR.mkdir(exist_ok=True) -STATS_FILE = DATA_DIR / 'stats.json' -CONFIG_FILE = DATA_DIR / 'config.json' -LOGS_DIR = Path(__file__).parent.parent / 'logs' - -# 提供商状态缓存 -provider_status = {} - -def refresh_provider_status(): - """刷新提供商状态""" - providers = get_providers() - for provider in providers: - if provider['name'] not in provider_status: - provider_status[provider['name']] = { - 'available': True, - 'last_check': None, - 'error_count': 0, - 'last_error': None, - 'request_count': 0, - 'success_count': 0, - 'total_tokens': 0, - } - -def load_stats(): - """加载统计数据""" - if STATS_FILE.exists(): - return json.loads(STATS_FILE.read_text(encoding='utf-8')) - return { - 'total_requests': 0, - 'total_success': 0, - 'total_errors': 0, - 'total_tokens': 0, - 'requests_today': 0, - 'providers': {}, - 'last_updated': None - } - -def save_stats(stats): - """保存统计数据""" - stats['last_updated'] = datetime.now().isoformat() - STATS_FILE.write_text(json.dumps(stats, ensure_ascii=False, indent=2), encoding='utf-8') - -# ============ 页面路由 ============ - -@app.route('/') -def index(): - return render_template('index.html') - -@app.route('/providers') -def providers_page(): - return render_template('providers.html') - -@app.route('/models') -def models_page(): - return render_template('models.html') - -@app.route('/logs') -def logs_page(): - return render_template('logs.html') - -@app.route('/config') -def config_page(): - return render_template('config.html') - -@app.route('/chat') -def chat_page(): - return render_template('chat.html') - -@app.route('/auto-profiles') -def auto_profiles_page(): - return render_template('auto-profiles.html') - -# ============ API路由 ============ - -@app.route('/api/stats') -def api_stats(): - """获取统计数据""" - stats = load_stats() - providers = get_providers() - refresh_provider_status() - - # 统计提供商状态 - available_count = sum(1 for p in providers if provider_status.get(p['name'], {}).get('available', True)) - - return jsonify({ - 'total_requests': stats.get('total_requests', 0), - 'total_success': stats.get('total_success', 0), - 'total_errors': stats.get('total_errors', 0), - 'total_tokens': stats.get('total_tokens', 0), - 'providers_count': len(providers), - 'available_providers': available_count, - 'models_count': len(get_model_aliases()), - 'uptime': time.time(), - }) - -@app.route('/api/providers') -def api_providers(): - """获取提供商列表""" - providers = get_providers() - refresh_provider_status() - stats = load_stats() - providers_data = [] - - for provider in sorted(providers, key=lambda x: x['priority']): - p_stats = stats.get('providers', {}).get(provider['name'], {}) - p_status = provider_status.get(provider['name'], {}) - - providers_data.append({ - 'id': provider.get('id', provider['name'].lower().replace(' ', '-')), - 'name': provider['name'], - 'priority': provider['priority'], - 'enabled': provider['enabled'], - 'available': p_status.get('available', True), - 'base_url': provider['base_url'], - 'api_key': provider['api_key'], - 'models': provider['models'], - 'default_model': provider['default_model'], - 'timeout': provider.get('timeout', 120), - 'request_count': p_stats.get('request_count', 0), - 'success_count': p_stats.get('success_count', 0), - 'error_count': p_status.get('error_count', 0), - 'last_error': p_status.get('last_error'), - 'last_check': p_status.get('last_check'), - }) - - return jsonify(providers_data) - -@app.route('/api/providers/', methods=['GET']) -def api_provider_detail(provider_id): - """获取提供商详情""" - provider = get_provider(provider_id) - - if not provider: - return jsonify({'error': 'Provider not found'}), 404 - - stats = load_stats() - p_stats = stats.get('providers', {}).get(provider['name'], {}) - p_status = provider_status.get(provider['name'], {}) - - return jsonify({ - **provider, - 'status': { - 'available': p_status.get('available', True), - 'error_count': p_status.get('error_count', 0), - 'last_error': p_status.get('last_error'), - 'request_count': p_stats.get('request_count', 0), - 'success_count': p_stats.get('success_count', 0), - } - }) - -@app.route('/api/providers', methods=['POST']) -def api_add_provider(): - """添加新提供商""" - data = request.get_json() - - if not data: - return jsonify({'error': 'Invalid request body'}), 400 - - # 验证必填字段 - required = ['name', 'base_url', 'api_key', 'models'] - for field in required: - if not data.get(field): - return jsonify({'error': f'Missing required field: {field}'}), 400 - - # 构建提供商数据 - providers = get_providers() - max_priority = max([p['priority'] for p in providers]) if providers else 0 - - new_provider = { - 'id': data.get('id') or data['name'].lower().replace(' ', '-').replace('.', '-'), - 'name': data['name'], - 'priority': data.get('priority', max_priority + 1), - 'base_url': data['base_url'].rstrip('/'), - 'api_key': data['api_key'], - 'models': data['models'] if isinstance(data['models'], list) else data['models'].split(','), - 'default_model': data.get('default_model', data['models'][0] if isinstance(data['models'], list) else data['models'].split(',')[0]), - 'timeout': data.get('timeout', 120), - 'enabled': data.get('enabled', True), - } - - # 添加到配置 - result = add_provider(new_provider) - - # 初始化状态 - provider_status[result['name']] = { - 'available': True, - 'last_check': None, - 'error_count': 0, - 'last_error': None, - } - - return jsonify({'success': True, 'provider': result}) - -@app.route('/api/providers/', methods=['PUT']) -def api_update_provider(provider_id): - """更新提供商""" - data = request.get_json() - - if not data: - return jsonify({'error': 'Invalid request body'}), 400 - - # 处理models字段 - if 'models' in data and isinstance(data['models'], str): - data['models'] = [m.strip() for m in data['models'].split(',') if m.strip()] - - result = update_provider(provider_id, data) - - if not result: - return jsonify({'error': 'Provider not found'}), 404 - - return jsonify({'success': True, 'provider': result}) - -@app.route('/api/providers/', methods=['DELETE']) -def api_delete_provider(provider_id): - """删除提供商""" - result = delete_provider(provider_id) - - if not result: - return jsonify({'error': 'Provider not found'}), 404 - - # 清理状态 - providers = get_providers() - for p in providers: - if p.get('id') == provider_id: - if p['name'] in provider_status: - del provider_status[p['name']] - break - - return jsonify({'success': True}) - -@app.route('/api/providers/priority', methods=['POST']) -def api_update_priority(): - """更新优先级顺序（拖拽排序）""" - data = request.get_json() - - if not data or 'order' not in data: - return jsonify({'error': 'Missing order field'}), 400 - - # order 是提供商ID列表，按新顺序排列 - provider_ids = data['order'] - result = update_priority(provider_ids) - - return jsonify({'success': True, 'providers': result}) - -@app.route('/api/providers//toggle', methods=['POST']) -def api_toggle_provider(provider_id): - """切换提供商启用状态""" - provider = get_provider(provider_id) - - if not provider: - return jsonify({'error': 'Provider not found'}), 404 - - new_enabled = not provider.get('enabled', True) - result = update_provider(provider_id, {'enabled': new_enabled}) - - return jsonify({'success': True, 'enabled': new_enabled}) - -@app.route('/api/providers//test', methods=['POST']) -def api_test_provider(provider_id): - """测试提供商连接""" - provider = get_provider(provider_id) - - if not provider: - return jsonify({'success': False, 'error': 'Provider not found'}), 404 - - try: - # 测试模型列表接口 - url = f"{provider['base_url'].rstrip('/')}/models" - headers = {"Authorization": f"Bearer {provider['api_key']}"} - - response = requests.get(url, headers=headers, timeout=10) - - if response.status_code == 200: - provider_status[provider['name']] = { - 'available': True, - 'last_check': datetime.now().isoformat(), - 'error_count': 0, - 'last_error': None, - } - # 尝试解析返回的模型列表 - models_data = [] - try: - resp_json = response.json() - models_data = resp_json.get('data', []) - except: - pass - - return jsonify({ - 'success': True, - 'message': 'Connection successful', - 'models_count': len(models_data) - }) - else: - provider_status[provider['name']] = { - 'available': False, - 'last_check': datetime.now().isoformat(), - 'error_count': provider_status.get(provider['name'], {}).get('error_count', 0) + 1, - 'last_error': f'HTTP {response.status_code}', - } - return jsonify({ - 'success': False, - 'error': f'HTTP {response.status_code}: {response.text[:200]}' - }) - - except Exception as e: - provider_status[provider['name']] = { - 'available': False, - 'last_check': datetime.now().isoformat(), - 'error_count': provider_status.get(provider['name'], {}).get('error_count', 0) + 1, - 'last_error': str(e), - } - return jsonify({'success': False, 'error': str(e)}) - -@app.route('/api/models') -def api_models(): - """获取模型列表""" - providers = get_providers() - aliases = get_model_aliases() - - models_list = [] - added = set() - - # 添加auto - models_list.append({ - 'alias': 'auto', - 'target': 'auto', - 'description': '自动选择可用模型（按优先级）' - }) - added.add('auto') - - # 从提供商获取模型 - for provider in sorted(providers, key=lambda x: x['priority']): - for model in provider['models']: - if model not in added: - models_list.append({ - 'alias': model, - 'target': model, - 'provider': provider['name'], - 'priority': provider['priority'], - }) - added.add(model) - - # 添加别名 - for alias, target in aliases.items(): - if alias != 'auto' and alias not in added: - # 找到目标模型对应的提供商 - provider_name = None - for p in providers: - if target in p['models']: - provider_name = p['name'] - break - - models_list.append({ - 'alias': alias, - 'target': target, - 'provider': provider_name, - }) - - return jsonify(models_list) - -@app.route('/api/logs') -def api_logs(): - """获取日志""" - log_file = LOGS_DIR / 'proxy.log' - - lines = [] - if log_file.exists(): - content = log_file.read_text(encoding='utf-8') - lines = content.strip().split('\n')[-100:] # 最近100条 - - return jsonify({ - 'logs': lines, - 'total_lines': len(lines) - }) - -@app.route('/api/config') -def api_config(): - """获取配置""" - providers = get_providers() - aliases = get_model_aliases() - - return jsonify({ - 'providers': [{ - 'id': p.get('id', p['name'].lower().replace(' ', '-')), - 'name': p['name'], - 'priority': p['priority'], - 'base_url': p['base_url'], - 'models': p['models'], - 'timeout': p.get('timeout', 120), - 'enabled': p['enabled'], - } for p in providers], - 'model_aliases': aliases, - 'retry_config': RETRY_CONFIG, - 'server_config': { - 'port': SERVER_CONFIG['port'], - } - }) - -@app.route('/api/reload', methods=['POST']) -def api_reload_config(): - """通知主服务重新加载配置""" - # 这个接口可以被主服务调用以重新加载配置 - # 这里只返回成功，实际重载由主服务自己处理 - return jsonify({'success': True, 'message': 'Config saved, restart main service to apply'}) - - -# ============ 对话功能 ============ - -CHATS_FILE = DATA_DIR / 'chats.json' - -def load_chats(): - """加载对话数据""" - if CHATS_FILE.exists(): - return json.loads(CHATS_FILE.read_text(encoding='utf-8')) - return {'chats': []} - -def save_chats(data): - """保存对话数据""" - CHATS_FILE.write_text(json.dumps(data, ensure_ascii=False, indent=2), encoding='utf-8') - - -@app.route('/api/chat/models') -def api_chat_models(): - """获取可用模型列表""" - providers = get_providers() - profiles = get_auto_profiles() - - models = [] - added = set() - - # 添加所有auto配置 - for name, profile in profiles.items(): - if name not in added: - models.append({ - 'id': name, - 'description': profile.get('description', 'Auto-select') - }) - added.add(name) - - # 添加提供商的模型 - for provider in providers: - if not provider['enabled']: - continue - for model in provider['models']: - if model not in added: - models.append({ - 'id': model, - 'description': provider['name'] - }) - added.add(model) - - return jsonify(models) - - -@app.route('/api/chat/list') -def api_chat_list(): - """获取对话列表""" - data = load_chats() - - chats = [] - for chat in data.get('chats', []): - chats.append({ - 'id': chat['id'], - 'title': chat.get('title', '新对话'), - 'model': chat.get('model', 'auto'), - 'message_count': len(chat.get('messages', [])), - 'created_at': chat.get('created_at'), - 'updated_at': chat.get('updated_at') - }) - - # 按更新时间倒序 - chats.sort(key=lambda x: x.get('updated_at', ''), reverse=True) - - return jsonify(chats) - - -@app.route('/api/chat/') -def api_chat_detail(chat_id): - """获取对话详情""" - data = load_chats() - - for chat in data.get('chats', []): - if chat['id'] == chat_id: - return jsonify(chat) - - return jsonify({'error': 'Chat not found'}), 404 - - -@app.route('/api/chat/send', methods=['POST']) -def api_chat_send(): - """发送消息""" - req = request.get_json() - - user_message = req.get('message', '') - model = req.get('model', 'auto') - chat_id = req.get('chat_id') - - if not user_message: - return jsonify({'error': 'Message is required'}), 400 - - data = load_chats() - - # 查找或创建对话 - chat = None - if chat_id: - for c in data['chats']: - if c['id'] == chat_id: - chat = c - break - - if not chat: - chat_id = str(uuid.uuid4())[:8] - chat = { - 'id': chat_id, - 'title': '新对话', - 'model': model, - 'messages': [], - 'created_at': datetime.now().isoformat(), - 'updated_at': datetime.now().isoformat() - } - data['chats'].append(chat) - - # 添加用户消息 - chat['messages'].append({ - 'role': 'user', - 'content': user_message, - 'time': datetime.now().isoformat() - }) - - # 调用LLM API - try: - proxy_url = f"http://localhost:{SERVER_CONFIG['port']}/v1/chat/completions" - - # 构建消息历史 - messages = [] - for msg in chat['messages'][-20:]: # 最多保留20条历史 - messages.append({ - 'role': msg['role'], - 'content': msg['content'] - }) - - response = requests.post(proxy_url, json={ - 'model': model, - 'messages': messages, - 'stream': False - }, timeout=120) - - if response.status_code == 200: - result = response.json() - assistant_message = result['choices'][0]['message']['content'] - used_model = result.get('model', model) - - # 添加助手消息 - chat['messages'].append({ - 'role': 'assistant', - 'content': assistant_message, - 'model': used_model, - 'time': datetime.now().isoformat() - }) - - # 更新标题（如果是第一条消息） - if len(chat['messages']) == 2: - chat['title'] = user_message[:30] + ('...' if len(user_message) > 30 else '') - - chat['updated_at'] = datetime.now().isoformat() - save_chats(data) - - return jsonify({ - 'success': True, - 'chat_id': chat_id, - 'response': assistant_message, - 'model': used_model, - 'title': chat['title'] - }) - else: - error_msg = response.json().get('error', {}).get('message', 'Unknown error') - return jsonify({'error': error_msg}), response.status_code - - except Exception as e: - return jsonify({'error': str(e)}), 500 - - -@app.route('/api/chat/', methods=['DELETE']) -def api_delete_chat(chat_id): - """删除对话""" - data = load_chats() - data['chats'] = [c for c in data['chats'] if c['id'] != chat_id] - save_chats(data) - return jsonify({'success': True}) - - -@app.route('/api/chat//clear', methods=['POST']) -def api_clear_chat(chat_id): - """清空对话消息""" - data = load_chats() - - for chat in data['chats']: - if chat['id'] == chat_id: - chat['messages'] = [] - chat['updated_at'] = datetime.now().isoformat() - save_chats(data) - return jsonify({'success': True}) - - return jsonify({'error': 'Chat not found'}), 404 - -@app.route('/api/requests/recent') -def api_recent_requests(): - """获取最近请求记录""" - # 模拟数据 - return jsonify([]) - - -# ============ Auto配置管理 ============ - -@app.route('/api/auto-profiles') -def api_auto_profiles(): - """获取所有Auto配置""" - profiles = get_auto_profiles() - providers = get_providers() - - result = [] - for name, profile in profiles.items(): - # 解析允许的提供商信息 - allowed_providers = profile.get('providers', ['*']) - provider_details = [] - - if '*' in allowed_providers: - provider_details = [{'id': '*', 'name': '所有启用的提供商'}] - else: - for p in providers: - if p.get('id') in allowed_providers or p['name'] in allowed_providers: - provider_details.append({ - 'id': p.get('id'), - 'name': p['name'], - 'priority': p['priority'] - }) - - result.append({ - 'name': name, - 'display_name': profile.get('name', name), - 'description': profile.get('description', ''), - 'strategy': profile.get('strategy', 'priority'), - 'providers': allowed_providers, - 'provider_details': provider_details, - }) - - return jsonify(result) - - -@app.route('/api/auto-profiles/', methods=['GET']) -def api_auto_profile_detail(profile_name): - """获取单个Auto配置详情""" - profile = get_auto_profile(profile_name) - - if not profile: - return jsonify({'error': 'Profile not found'}), 404 - - providers = get_providers() - allowed_providers = profile.get('providers', ['*']) - provider_details = [] - - if '*' in allowed_providers: - provider_details = [{'id': '*', 'name': '所有启用的提供商', 'selected': True}] - for p in providers: - provider_details.append({ - 'id': p.get('id'), - 'name': p['name'], - 'priority': p['priority'], - 'selected': True - }) - else: - for p in providers: - selected = p.get('id') in allowed_providers or p['name'] in allowed_providers - provider_details.append({ - 'id': p.get('id'), - 'name': p['name'], - 'priority': p['priority'], - 'selected': selected - }) - - return jsonify({ - 'name': profile_name, - 'display_name': profile.get('name', profile_name), - 'description': profile.get('description', ''), - 'strategy': profile.get('strategy', 'priority'), - 'providers': allowed_providers, - 'provider_details': provider_details, - }) - - -@app.route('/api/auto-profiles', methods=['POST']) -def api_add_auto_profile(): - """添加新的Auto配置""" - data = request.get_json() - - if not data or not data.get('name'): - return jsonify({'error': 'Missing profile name'}), 400 - - profile_name = data['name'].lower().replace(' ', '-').replace('.', '-') - - if profile_name in get_auto_profiles(): - return jsonify({'error': 'Profile already exists'}), 400 - - profile_data = { - 'name': data.get('display_name', data['name']), - 'description': data.get('description', ''), - 'providers': data.get('providers', ['*']), - 'strategy': data.get('strategy', 'priority'), - } - - result = add_auto_profile(profile_name, profile_data) - - return jsonify({'success': True, 'profile': {profile_name: profile_data}}) - - -@app.route('/api/auto-profiles/', methods=['PUT']) -def api_update_auto_profile(profile_name): - """更新Auto配置""" - data = request.get_json() - - if not data: - return jsonify({'error': 'Invalid request body'}), 400 - - profile_data = {} - if 'display_name' in data: - profile_data['name'] = data['display_name'] - if 'description' in data: - profile_data['description'] = data['description'] - if 'providers' in data: - profile_data['providers'] = data['providers'] - if 'strategy' in data: - profile_data['strategy'] = data['strategy'] - - result = update_auto_profile(profile_name, profile_data) - - if not result: - return jsonify({'error': 'Profile not found'}), 404 - - return jsonify({'success': True, 'profile': result}) - - -@app.route('/api/auto-profiles/', methods=['DELETE']) -def api_delete_auto_profile(profile_name): - """删除Auto配置""" - result = delete_auto_profile(profile_name) - - if not result: - return jsonify({'error': 'Cannot delete default auto profile or profile not found'}), 400 - - return jsonify({'success': True}) - -if __name__ == '__main__': - print("=" * 50) - print("大模型API中转系统 - 后台管理") - print("=" * 50) - print(f"访问地址: http://localhost:19008") - print(f"前台地址: http://localhost:19007") - print("=" * 50) - - app.run(host='0.0.0.0', port=19008, debug=True) \ No newline at end of file diff --git a/admin/static/img/favicon.svg b/admin/static/img/favicon.svg deleted file mode 100644 index ae3cdd1..0000000 --- a/admin/static/img/favicon.svg +++ /dev/null @@ -1,12 +0,0 @@ - - - - - - - - - - - - \ No newline at end of file diff --git a/admin/templates/auto-profiles.html b/admin/templates/auto-profiles.html deleted file mode 100644 index edcbb9b..0000000 --- a/admin/templates/auto-profiles.html +++ /dev/null @@ -1,366 +0,0 @@ - - - - - - Auto配置管理 - LLM Proxy - - - - - - -
- - -
- -
-
-

Auto配置管理

-

创建自定义的自动选择模式，指定候选提供商和优先级

-
- -
- - -
- - Auto配置：创建自定义的模型自动选择模式。例如 model="auto-fast" 只选择响应快的提供商，model="auto-cheap" 只选择便宜的提供商。 -
- - -
-

加载中...

-
-
-
- - - - - - - - \ No newline at end of file diff --git a/admin/templates/chat.html b/admin/templates/chat.html deleted file mode 100644 index 74d2d9f..0000000 --- a/admin/templates/chat.html +++ /dev/null @@ -1,416 +0,0 @@ - - - - - - 对话 - LLM Proxy - - - - - - - -
- - - - -
- -
-
-

历史对话

- -
-
-

加载中...

-
-
- - -
- -
-
- - 新对话 -
-
- -
-
- - -
-
- -

开始新对话

-
-
- - -
-
- - -
-
-
-
-
- - - - \ No newline at end of file diff --git a/admin/templates/config.html b/admin/templates/config.html deleted file mode 100644 index 4ad91fd..0000000 --- a/admin/templates/config.html +++ /dev/null @@ -1,124 +0,0 @@ - - - - - - 系统配置 - LLM Proxy - - - - - - -
- - -
-

系统配置

- -
-

加载中...

-
-
-
- - - - \ No newline at end of file diff --git a/admin/templates/index.html b/admin/templates/index.html deleted file mode 100644 index 59886d2..0000000 --- a/admin/templates/index.html +++ /dev/null @@ -1,232 +0,0 @@ - - - - - - LLM Proxy - 后台管理 - - - - - - -
- - - - -
- -
-
-
-
-

总请求数

-

-

-
-
- -
-
-
- -
-
-
-

成功率

-

-

-
-
- -
-
-
- -
-
-
-

提供商状态

-

-

-
-
- -
-
-
- -
-
-
-

支持模型

-

-

-
-
- -
-
-
-
- - -
-
-

- - 提供商状态 -

-
-

加载中...

-
-
- -
-

- - 调用流程 -

-
-
- 1 -
-

客户端请求

-

POST /v1/chat/completions

-
-
-
- 2 -
-

解析模型名

-

model="auto" → 按优先级选择

-
-
-
- 3 -
-

转发请求

-

调用高优先级提供商

-
-
-
- 4 -
-

故障切换

-

失败自动切换备用提供商

-
-
-
-
-
- - -
-

- - 快速操作 -

-
- - 管理提供商 - - - 查看API状态 - - - 查看日志 - -
-
-
-
- - - - \ No newline at end of file diff --git a/admin/templates/logs.html b/admin/templates/logs.html deleted file mode 100644 index 760b2eb..0000000 --- a/admin/templates/logs.html +++ /dev/null @@ -1,99 +0,0 @@ - - - - - - 日志查看 - LLM Proxy - - - - - - -
- - -
-
-

日志查看

- -
- -
-
-

加载中...

-
-
-
-
- - - - \ No newline at end of file diff --git a/admin/templates/models.html b/admin/templates/models.html deleted file mode 100644 index b03ab17..0000000 --- a/admin/templates/models.html +++ /dev/null @@ -1,98 +0,0 @@ - - - - - - 模型管理 - LLM Proxy - - - - - - -
- - -
-

模型管理

- -
- - - - - - - - - - - - - -
模型别名目标模型提供商优先级说明
加载中...
-
-
-
- - - - \ No newline at end of file diff --git a/admin/templates/providers.html b/admin/templates/providers.html deleted file mode 100644 index f34abd1..0000000 --- a/admin/templates/providers.html +++ /dev/null @@ -1,610 +0,0 @@ - - - - - - 提供商管理 - LLM Proxy - - - - - - -
- - -
- -
-
-

提供商管理

-

拖拽卡片调整优先级顺序（auto模式选择顺序）

-
- -
- - -
-

加载中...

-
- - -
- - Auto模式优先级： 当使用 model="auto" 时，系统会按优先级顺序依次尝试可用的提供商。拖拽上方卡片可调整顺序。 -
-
-
- - - - - - - - - - \ No newline at end of file diff --git a/app.py b/app.py index 96657f0..2cd23b9 100644 --- a/app.py +++ b/app.py @@ -1,11 +1,11 @@ """ 大模型API中转系统 -v2.0.0 - 合并后台管理到单端口 +v2.1.0 - 能力(Capability)体系 + 模型管理 + 生图/语音/视频端点 兼容OpenAI API格式，支持多上游提供商优先级调度 -端口: 19007 -前台API: http://localhost:19007/v1/chat/completions -后台管理: http://localhost:19007/admin +端口: 16003 +前台API: http://localhost:16003/v1/chat/completions +后台管理: http://localhost:16003/admin """ from flask import Flask, request, jsonify, Response, stream_with_context, render_template @@ -13,6 +13,7 @@ from flask_cors import CORS import requests import json import time +import random import logging from datetime import datetime, date from pathlib import Path @@ -23,39 +24,108 @@ import uuid # 添加配置路径 sys.path.insert(0, str(Path(__file__).parent)) from config.settings import ( - get_providers, get_model_aliases, get_auto_profiles, SERVER_CONFIG, - LOG_CONFIG, RETRY_CONFIG, - load_config, save_config, get_provider, add_provider, update_provider, - delete_provider, update_priority, update_model_alias, - get_auto_profile, add_auto_profile, update_auto_profile, delete_auto_profile, + get_providers, get_model_aliases, get_auto_profiles, get_auto_profile, + SERVER_CONFIG, LOG_CONFIG, RETRY_CONFIG, CAPABILITY_DEFS, + load_config, save_config, get_provider, add_provider, update_provider, + delete_provider, update_priority, update_model_alias, delete_model_alias, + add_auto_profile, update_auto_profile, delete_auto_profile, DEFAULT_PROVIDERS, DEFAULT_MODEL_ALIASES, DEFAULT_AUTO_PROFILES ) app = Flask(__name__, template_folder='templates') CORS(app) +VERSION = "2.1.0" + # 数据目录和统计文件 DATA_DIR = Path(__file__).parent / 'data' DATA_DIR.mkdir(exist_ok=True) STATS_FILE = DATA_DIR / 'stats.json' CHATS_FILE = DATA_DIR / 'chats.json' -LOGS_DIR = Path(__file__).parent / 'logs' +LOGS_DIR = Path(__file__).parent / (LOG_CONFIG.get('log_dir', 'logs')) LOGS_DIR.mkdir(exist_ok=True) # 统计锁（避免并发写入冲突） stats_lock = threading.Lock() +chats_lock = threading.Lock() # 提供商状态缓存 provider_status = {} # 配置缓存时间（秒） -CONFIG_CACHE_TTL = 5 +CONFIG_CACHE_TTL = 3 _last_config_load = 0 _cached_providers = [] _cached_aliases = {} _cached_auto_profiles = {} +# ============ 能力/模型 工具函数 ============ + +def normalize_models(models, default_caps): + """规范化模型列表（支持字符串 或 {name,capabilities} 两种格式）""" + result = [] + for m in models or []: + if isinstance(m, str): + result.append({'name': m, 'capabilities': list(default_caps)}) + elif isinstance(m, dict): + caps = m.get('capabilities') or list(default_caps) + result.append({'name': m.get('name', ''), 'capabilities': caps}) + return [m for m in result if m['name']] + + +def provider_models(provider): + """获取提供商的有效模型列表（dict）""" + default_caps = provider.get('capabilities', ['text']) + return normalize_models(provider.get('models', []), default_caps) + + +def provider_model_names(provider): + """提供商支持的模型名列表""" + return [m['name'] for m in provider_models(provider)] + + +def model_capabilities(provider, model_name): + """获取某模型在该提供商下的能力""" + for m in provider_models(provider): + if m['name'] == model_name: + return m.get('capabilities') or provider.get('capabilities', ['text']) + return provider.get('capabilities', ['text']) + + +def model_has_capability(provider, model_name, capability): + """判断模型是否具备某能力""" + if capability == 'all' or not capability: + return True + caps = model_capabilities(provider, model_name) + return capability in caps + + +def provider_supports_capability(provider, capability): + """判断提供商下是否有模型具备某能力""" + if capability == 'all' or not capability: + return True + return any(model_has_capability(provider, m['name'], capability) for m in provider_models(provider)) + + +def provider_capability_models(provider, capability): + """返回提供商下具备某能力的所有模型名""" + return [m['name'] for m in provider_models(provider) if model_has_capability(provider, m['name'], capability)] + + +def find_default_model_for_capability(provider, capability): + """找提供商下具备某能力的最佳模型（优先 default_model）""" + default_model = provider.get('default_model', '') + if default_model and default_model in provider_model_names(provider) and model_has_capability(provider, default_model, capability): + return default_model + for m in provider_models(provider): + if model_has_capability(provider, m['name'], capability): + return m['name'] + return None + + +# ============ 统计 ============ + def load_stats(): """加载统计数据""" if STATS_FILE.exists(): @@ -87,21 +157,21 @@ def increment_stats(model, provider_name, success=False, tokens=0, error=None): with stats_lock: stats = load_stats() today = date.today().isoformat() - + if stats.get('date') != today: stats['date'] = today stats['requests_today'] = 0 - + stats['total_requests'] += 1 stats['requests_today'] += 1 - + if model not in stats['requests_by_model']: stats['requests_by_model'][model] = {'count': 0, 'success': 0, 'tokens': 0} stats['requests_by_model'][model]['count'] += 1 if success: stats['requests_by_model'][model]['success'] += 1 stats['requests_by_model'][model]['tokens'] += tokens - + if provider_name not in stats['providers']: stats['providers'][provider_name] = {'requests': 0, 'success': 0, 'errors': 0, 'tokens': 0} stats['providers'][provider_name]['requests'] += 1 @@ -113,21 +183,23 @@ def increment_stats(model, provider_name, success=False, tokens=0, error=None): else: stats['providers'][provider_name]['errors'] += 1 stats['total_errors'] += 1 - + save_stats(stats) +# ============ 配置缓存 ============ + def refresh_config(): """动态刷新配置""" - global _last_config_load, _cached_providers, _cached_aliases, _cached_auto_profiles, provider_status - + global _last_config_load, _cached_providers, _cached_aliases, _cached_auto_profiles + current_time = time.time() if current_time - _last_config_load > CONFIG_CACHE_TTL: _cached_providers = get_providers() _cached_aliases = get_model_aliases() _cached_auto_profiles = get_auto_profiles() _last_config_load = current_time - + for provider in _cached_providers: if provider['name'] not in provider_status: provider_status[provider['name']] = { @@ -154,137 +226,221 @@ def refresh_provider_status(): } -def load_chats(): - """加载对话数据""" - if CHATS_FILE.exists(): - return json.loads(CHATS_FILE.read_text(encoding='utf-8')) - return {'chats': []} +# ============ 熔断器（带冷却恢复） ============ - -def save_chats(data): - """保存对话数据""" - CHATS_FILE.write_text(json.dumps(data, ensure_ascii=False, indent=2), encoding='utf-8') - - -# 配置日志 -logging.basicConfig( - level=logging.INFO, - format='%(asctime)s - %(name)s - %(levelname)s - %(message)s', - handlers=[ - logging.FileHandler(LOGS_DIR / 'proxy.log', encoding='utf-8'), - logging.StreamHandler() - ] -) -logger = logging.getLogger(__name__) - -# 初始化 -refresh_config() - - -# ============ 提供商选择逻辑 ============ - -def get_provider_for_model(model_name): - """根据模型名获取提供商""" - refresh_config() - - resolved_model = _cached_aliases.get(model_name, model_name) - - if resolved_model == 'auto' or resolved_model.startswith('auto-'): - return get_available_provider_for_auto(resolved_model) - - sorted_providers = sorted(_cached_providers, key=lambda x: x['priority']) - - for provider in sorted_providers: - if not provider['enabled']: - continue - if not provider_status.get(provider['name'], {}).get('available', True): - continue - if resolved_model in provider['models']: - return provider, resolved_model - - for provider in sorted_providers: - if not provider['enabled']: - continue - if not provider_status.get(provider['name'], {}).get('available', True): - continue - for m in provider['models']: - if resolved_model.lower() in m.lower() or m.lower() in resolved_model.lower(): - return provider, m - - return None, None - - -def get_available_provider_for_auto(auto_name='auto', exclude_providers=None): - """获取auto模式下的可用提供商""" - refresh_config() - - exclude_providers = exclude_providers or [] - profile = _cached_auto_profiles.get(auto_name, _cached_auto_profiles.get('auto', {})) - allowed_providers = profile.get('providers', ['*']) - - sorted_providers = sorted(_cached_providers, key=lambda x: x['priority']) - candidates = [] - - for provider in sorted_providers: - if not provider['enabled']: - continue - if provider['name'] in exclude_providers: - continue - if not provider_status.get(provider['name'], {}).get('available', True): - continue - - if '*' in allowed_providers: - candidates.append(provider) - elif provider.get('id') in allowed_providers or provider['name'] in allowed_providers: - candidates.append(provider) - - if not candidates: - if sorted_providers: - return sorted_providers[0], sorted_providers[0]['default_model'] - return None, None - - return candidates[0], candidates[0]['default_model'] +def is_provider_available(provider_name): + """判断提供商是否可用（含熔断冷却自动恢复）""" + status = provider_status.get(provider_name) + if not status: + return True + if status.get('available', True): + return True + # 熔断中：检查冷却期是否已过，过了则半开恢复一次 + open_at = status.get('circuit_open_at') + if open_at: + cooldown = RETRY_CONFIG.get('cooldown_seconds', 60) + if time.time() - open_at > cooldown: + status['error_count'] = 0 + status['available'] = True + status['circuit_open_at'] = None + logger.info(f"Provider {provider_name} circuit breaker recovered (cooldown passed)") + return True + return False def mark_provider_error(provider_name, error): - """标记提供商错误""" + """标记提供商错误（连续3次熔断）""" if provider_name in provider_status: - provider_status[provider_name]['error_count'] += 1 - provider_status[provider_name]['last_error'] = str(error) - provider_status[provider_name]['last_check'] = datetime.now() - - if provider_status[provider_name]['error_count'] >= 3: - provider_status[provider_name]['available'] = False - logger.warning(f"Provider {provider_name} marked as unavailable") + status = provider_status[provider_name] + status['error_count'] = status.get('error_count', 0) + 1 + status['last_error'] = str(error) + status['last_check'] = datetime.now() + + if status['error_count'] >= 3: + status['available'] = False + status['circuit_open_at'] = time.time() + logger.warning(f"Provider {provider_name} marked as unavailable (circuit open)") def mark_provider_success(provider_name): - """标记提供商成功""" + """标记提供商成功（复位熔断）""" if provider_name in provider_status: provider_status[provider_name]['error_count'] = 0 provider_status[provider_name]['available'] = True + provider_status[provider_name]['circuit_open_at'] = None provider_status[provider_name]['last_check'] = datetime.now() -def proxy_request(provider, model, request_data, stream=False): - """转发请求到上游提供商""" - url = f"{provider['base_url'].rstrip('/')}/chat/completions" - +# ============ 提供商/模型路由 ============ + +def resolve_model_name(model_name): + """解析模型别名""" + refresh_config() + return _cached_aliases.get(model_name, model_name) + + +def is_auto_model(model_name): + """判断是否为auto类模型（auto 或 auto-xxx 配置）""" + return model_name == 'auto' or model_name.startswith('auto-') + + +def sorted_providers(): + """按优先级排序的启用提供商""" + refresh_config() + return sorted(_cached_providers, key=lambda x: x['priority']) + + +def find_provider_for_model(model_name, capability=None, exclude=None): + """精确查找托管某模型的提供商（保持模型名不变，仅用于切换时的同模型替换）""" + exclude = exclude or set() + for provider in sorted_providers(): + if not provider['enabled']: + continue + if provider['name'] in exclude: + continue + if not is_provider_available(provider['name']): + continue + if model_name in provider_model_names(provider): + if model_has_capability(provider, model_name, capability): + return provider + return None + + +def get_auto_provider(profile_name='auto', capability=None, exclude=None): + """获取auto模式下可用的提供商与模型（按能力过滤）""" + refresh_config() + exclude = exclude or set() + profile = _cached_auto_profiles.get(profile_name, _cached_auto_profiles.get('auto', {})) + req_cap = profile.get('capability') or capability or 'text' + allowed_providers = profile.get('providers', ['*']) + strategy = profile.get('strategy', 'priority') + + candidates = [] + for provider in sorted_providers(): + if not provider['enabled']: + continue + if provider['name'] in exclude: + continue + if not is_provider_available(provider['name']): + continue + if not ('*' in allowed_providers or provider.get('id') in allowed_providers or provider['name'] in allowed_providers): + continue + chosen_model = find_default_model_for_capability(provider, req_cap) + if chosen_model: + candidates.append((provider, chosen_model)) + + if not candidates: + return None, None + if strategy == 'random': + return random.choice(candidates) + return candidates[0] + + +def get_provider_for_model(model_name, capability=None): + """根据模型名获取提供商与解析后的模型名""" + resolved_model = resolve_model_name(model_name) + + if is_auto_model(resolved_model): + return get_auto_provider(resolved_model, capability) + + # 精确匹配：提供商托管该模型 + provider = find_provider_for_model(resolved_model, capability) + if provider: + return provider, resolved_model + + # 兜底：匹配提供商 default_model + for provider in sorted_providers(): + if not provider['enabled']: + continue + if not is_provider_available(provider['name']): + continue + if provider.get('default_model') == resolved_model: + if model_has_capability(provider, resolved_model, capability): + return provider, resolved_model + + # 最后：模糊匹配（仅当请求模型是提供商某个模型名的子串或反向，避免误路由） + for provider in sorted_providers(): + if not provider['enabled']: + continue + if not is_provider_available(provider['name']): + continue + for m in provider_model_names(provider): + if resolved_model.lower() == m.lower(): + return provider, m + + return None, None + + +def detect_capability(data): + """根据请求内容判断所需能力（含图片则 vision，否则 text）""" + messages = data.get('messages', []) if isinstance(data, dict) else [] + for msg in messages: + content = msg.get('content') + if isinstance(content, list): + for part in content: + if isinstance(part, dict) and part.get('type') in ('image_url', 'image', 'input_image'): + return 'vision' + return 'text' + + +# ============ 上游转发 ============ + +def build_headers(provider, content_type='application/json', extra=None): headers = { - "Content-Type": "application/json", - "Authorization": f"Bearer {provider['api_key']}" + "Authorization": f"Bearer {provider['api_key']}", + "Content-Type": content_type, } - + if extra: + headers.update(extra) + return headers + + +def proxy_chat(provider, model, request_data, stream=False): + """转发聊天请求到上游""" + url = f"{provider['base_url'].rstrip('/')}/chat/completions" data = request_data.copy() data['model'] = model - + try: if stream: - response = requests.post(url, headers=headers, json=data, stream=True, timeout=provider.get('timeout', 120)) - return response - else: - response = requests.post(url, headers=headers, json=data, timeout=provider.get('timeout', 120)) - return response + return requests.post(url, headers=build_headers(provider), json=data, stream=True, timeout=provider.get('timeout', 120)) + return requests.post(url, headers=build_headers(provider), json=data, timeout=provider.get('timeout', 120)) + except requests.exceptions.Timeout: + mark_provider_error(provider['name'], "Timeout") + raise Exception(f"Provider {provider['name']} timeout") + except requests.exceptions.ConnectionError: + mark_provider_error(provider['name'], "Connection error") + raise Exception(f"Provider {provider['name']} connection error") + except Exception as e: + mark_provider_error(provider['name'], str(e)) + raise + + +def proxy_json(provider, path, request_data, timeout=None): + """通用JSON转发""" + url = f"{provider['base_url'].rstrip('/')}/{path.lstrip('/')}" + timeout = timeout or provider.get('timeout', 120) + try: + return requests.post(url, headers=build_headers(provider), json=request_data, timeout=timeout) + except requests.exceptions.Timeout: + mark_provider_error(provider['name'], "Timeout") + raise Exception(f"Provider {provider['name']} timeout") + except requests.exceptions.ConnectionError: + mark_provider_error(provider['name'], "Connection error") + raise Exception(f"Provider {provider['name']} connection error") + except Exception as e: + mark_provider_error(provider['name'], str(e)) + raise + + +def proxy_raw(provider, path, timeout=None): + """通用原始体转发（用于 multipart 表单，如语音识别）""" + url = f"{provider['base_url'].rstrip('/')}/{path.lstrip('/')}" + timeout = timeout or provider.get('timeout', 180) + data = request.get_data() + headers = build_headers(provider, content_type=request.headers.get('Content-Type', 'application/octet-stream')) + try: + return requests.post(url, headers=headers, data=data, timeout=timeout) except requests.exceptions.Timeout: mark_provider_error(provider['name'], "Timeout") raise Exception(f"Provider {provider['name']} timeout") @@ -307,6 +463,22 @@ def stream_response(response): yield b'data: {"error": "' + str(e).encode() + b'"}\n\n' +# ============ 日志 ============ + +logging.basicConfig( + level=logging.INFO, + format='%(asctime)s - %(name)s - %(levelname)s - %(message)s', + handlers=[ + logging.FileHandler(LOGS_DIR / 'proxy.log', encoding='utf-8'), + logging.StreamHandler() + ] +) +logger = logging.getLogger(__name__) + +# 初始化配置与提供商状态 +refresh_config() +refresh_provider_status() + # ============ 前台 API 路由 ============ @app.route('/') @@ -314,27 +486,32 @@ def index(): """首页""" return jsonify({ "name": "LLM Proxy", - "version": "2.0.0", - "description": "OpenAI-compatible LLM API Proxy", + "version": VERSION, + "description": "OpenAI-compatible LLM API Proxy with capability routing", "endpoints": { "chat": "/v1/chat/completions", "models": "/v1/models", "embeddings": "/v1/embeddings", + "images": "/v1/images/generations", + "audio_speech": "/v1/audio/speech", + "audio_transcriptions": "/v1/audio/transcriptions", + "video": "/v1/video/generations", "health": "/health", "status": "/status", "admin": "/admin" - } + }, + "capabilities": CAPABILITY_DEFS, }) @app.route('/v1/models', methods=['GET']) def list_models(): - """列出可用模型""" + """列出可用模型（含auto配置与能力标签）""" refresh_config() - + models_list = [] added_models = set() - + for profile_name, profile in _cached_auto_profiles.items(): if profile_name not in added_models: models_list.append({ @@ -342,23 +519,27 @@ def list_models(): "object": "model", "created": int(time.time()), "owned_by": "proxy", - "description": profile.get('description', 'Auto-select available model') + "kind": "auto", + "capability": profile.get('capability', 'text'), + "description": f"[{CAPABILITY_DEFS.get(profile.get('capability', 'text'), profile.get('capability', 'text'))}] {profile.get('description', '')}" }) added_models.add(profile_name) - + for provider in _cached_providers: if not provider['enabled']: continue - for model in provider['models']: - if model not in added_models: + for m in provider_models(provider): + if m['name'] not in added_models: models_list.append({ - "id": model, + "id": m['name'], "object": "model", "created": int(time.time()), "owned_by": provider['name'], + "kind": "model", + "capabilities": m['capabilities'], }) - added_models.add(model) - + added_models.add(m['name']) + return jsonify({"object": "list", "data": models_list}) @@ -369,38 +550,39 @@ def chat_completions(): request_provider = None request_success = False request_tokens = 0 - + try: data = request.get_json() - + if not data: increment_stats('unknown', 'unknown', success=False, error='Invalid request body') return jsonify({"error": "Invalid request body"}), 400 - + model = data.get('model', 'auto') stream = data.get('stream', False) request_model = model - - provider, resolved_model = get_provider_for_model(model) - + capability = detect_capability(data) + + provider, resolved_model = get_provider_for_model(model, capability) + if not provider: increment_stats(model, 'unknown', success=False, error=f'No provider for model: {model}') - return jsonify({"error": {"message": f"No available provider for model: {model}", "type": "invalid_request_error"}}), 400 - + return jsonify({"error": {"message": f"No available provider for model: {model} (capability: {capability})", "type": "invalid_request_error"}}), 400 + request_provider = provider['name'] - logger.info(f"Request: model={model} -> provider={provider['name']}, resolved_model={resolved_model}, stream={stream}") - + logger.info(f"Request: model={model} -> provider={provider['name']}, resolved_model={resolved_model}, stream={stream}, capability={capability}") + last_error = None - tried_providers = [] - + tried_providers = set() + for attempt in range(RETRY_CONFIG['max_retries']): try: - response = proxy_request(provider, resolved_model, data, stream) - + response = proxy_chat(provider, resolved_model, data, stream) + if response.status_code == 200: mark_provider_success(provider['name']) request_success = True - + if stream: increment_stats(model, provider['name'], success=True, tokens=0) return Response( @@ -414,42 +596,55 @@ def chat_completions(): request_tokens = usage.get('total_tokens', 0) increment_stats(model, provider['name'], success=True, tokens=request_tokens) return jsonify(result) - + else: error_info = response.json() if response.headers.get('content-type', '').startswith('application/json') else {"error": response.text} last_error = error_info logger.warning(f"Provider {provider['name']} returned {response.status_code}: {error_info}") mark_provider_error(provider['name'], f"HTTP {response.status_code}") - tried_providers.append(provider['name']) - - next_provider, next_model = get_available_provider_for_auto('auto', exclude_providers=tried_providers) - if next_provider and next_provider['name'] not in tried_providers: + tried_providers.add(provider['name']) + + # 切换到下一个能托管同一模型的提供商（保持模型名不变） + next_provider = find_provider_for_model(resolved_model, capability, exclude=tried_providers) + if next_provider: logger.info(f"Switching to next provider: {next_provider['name']}") + provider = next_provider + request_provider = provider['name'] + time.sleep(RETRY_CONFIG['retry_delay']) + continue + + increment_stats(model, provider['name'], success=False, error=str(last_error)) + return jsonify(error_info), response.status_code + + except Exception as e: + last_error = str(e) + logger.error(f"Attempt {attempt + 1} failed: {e}") + tried_providers.add(provider['name']) + + # 非auto模型：优先切换到能托管同一模型的其他提供商 + next_provider = find_provider_for_model(resolved_model, capability, exclude=tried_providers) + if next_provider: + provider = next_provider + request_provider = provider['name'] + time.sleep(RETRY_CONFIG['retry_delay']) + continue + + # auto模型：按auto配置切换到下一个候选（模型可随能力重选） + if is_auto_model(model): + next_provider, next_model = get_auto_provider(model, capability, exclude=tried_providers) + if next_provider: provider = next_provider resolved_model = next_model request_provider = provider['name'] time.sleep(RETRY_CONFIG['retry_delay']) continue - - increment_stats(model, provider['name'], success=False, error=str(last_error)) - return jsonify(error_info), response.status_code - - except Exception as e: - last_error = str(e) - logger.error(f"Attempt {attempt + 1} failed: {e}") - tried_providers.append(provider['name']) - - next_provider, next_model = get_available_provider_for_auto('auto', exclude_providers=tried_providers) - if next_provider and next_provider['name'] not in tried_providers: - provider = next_provider - resolved_model = next_model - request_provider = provider['name'] - time.sleep(RETRY_CONFIG['retry_delay']) - continue - + + # 没有更多可切换提供商，立即结束（不再重试同一个失败提供商） + break + increment_stats(model, request_provider or 'unknown', success=False, error=str(last_error)) return jsonify({"error": {"message": f"All providers failed. Last error: {last_error}", "type": "api_error"}}), 503 - + except Exception as e: logger.error(f"Unexpected error: {e}") increment_stats(request_model or 'unknown', request_provider or 'unknown', success=False, error=str(e)) @@ -458,30 +653,134 @@ def chat_completions(): @app.route('/v1/embeddings', methods=['POST']) def embeddings(): - """嵌入API""" + """嵌入API（按模型路由，不再固定第一个提供商）""" refresh_config() - + try: data = request.get_json() - - if _cached_providers: - provider = _cached_providers[0] - url = f"{provider['base_url'].rstrip('/')}/embeddings" - headers = {"Content-Type": "application/json", "Authorization": f"Bearer {provider['api_key']}"} - response = requests.post(url, headers=headers, json=data, timeout=60) - return jsonify(response.json()), response.status_code + if not data: + return jsonify({"error": "Invalid request body"}), 400 + + model = data.get('model') + provider = None + + if model: + provider = find_provider_for_model(model) + if not provider: + # 按名字匹配（如 embedding 模型名可能不在托管列表） + for p in sorted_providers(): + if p['enabled'] and is_provider_available(p['name']): + provider = p + break else: - return jsonify({"error": "No providers available"}), 503 + for p in sorted_providers(): + if p['enabled'] and is_provider_available(p['name']): + provider = p + break + + if not provider: + return jsonify({"error": "No available provider for embeddings"}), 503 + + response = proxy_json(provider, 'embeddings', data, timeout=60) + return jsonify(response.json()), response.status_code except Exception as e: return jsonify({"error": str(e)}), 500 +def _generic_capability_endpoint(capability, path, error_msg="No available provider"): + """通用能力端点路由""" + refresh_config() + try: + data = request.get_json() if request.is_json else {} + if not isinstance(data, dict): + data = {} + model = data.get('model', 'auto') + + if is_auto_model(model): + provider, resolved_model = get_auto_provider(model, capability) + else: + resolved_model = resolve_model_name(model) + provider = find_provider_for_model(resolved_model, capability) + if not provider: + # 允许提供商默认模型具备该能力 + for p in sorted_providers(): + if not p['enabled'] or not is_provider_available(p['name']): + continue + if p.get('default_model') == resolved_model and model_has_capability(p, resolved_model, capability): + provider = p + break + + if not provider: + return jsonify({"error": {"message": f"{error_msg} (capability: {capability})", "type": "invalid_request_error"}}), 400 + + data['model'] = resolved_model if resolved_model else data.get('model') + response = proxy_json(provider, path, data) + return Response( + response.content, + status=response.status_code, + content_type=response.headers.get('Content-Type', 'application/json') + ) + except Exception as e: + return jsonify({"error": str(e)}), 500 + + +@app.route('/v1/images/generations', methods=['POST']) +def images_generations(): + """图片生成API（OpenAI兼容）""" + return _generic_capability_endpoint('image_gen', 'images/generations', "No image generation provider available") + + +@app.route('/v1/audio/speech', methods=['POST']) +def audio_speech(): + """语音合成API（TTS，OpenAI兼容）""" + return _generic_capability_endpoint('audio_out', 'audio/speech', "No TTS provider available") + + +@app.route('/v1/audio/transcriptions', methods=['POST']) +def audio_transcriptions(): + """语音识别API（ASR，OpenAI兼容，multipart转发）""" + refresh_config() + try: + model = request.form.get('model', 'auto') + + if is_auto_model(model): + provider, resolved_model = get_auto_provider(model, 'audio_in') + else: + resolved_model = resolve_model_name(model) + provider = find_provider_for_model(resolved_model, 'audio_in') + if not provider: + for p in sorted_providers(): + if not p['enabled'] or not is_provider_available(p['name']): + continue + if p.get('default_model') == resolved_model and model_has_capability(p, resolved_model, 'audio_in'): + provider = p + break + + if not provider: + return jsonify({"error": {"message": "No ASR provider available (capability: audio_in)", "type": "invalid_request_error"}}), 400 + + response = proxy_raw(provider, 'audio/transcriptions') + return Response( + response.content, + status=response.status_code, + content_type=response.headers.get('Content-Type', 'application/json') + ) + except Exception as e: + return jsonify({"error": str(e)}), 500 + + +@app.route('/v1/video/generations', methods=['POST']) +def video_generations(): + """视频生成API""" + return _generic_capability_endpoint('video_gen', 'video/generations', "No video generation provider available") + + @app.route('/health', methods=['GET']) def health(): """健康检查""" - available_count = sum(1 for s in provider_status.values() if s['available']) + available_count = sum(1 for name, s in provider_status.items() if is_provider_available(name)) total_count = len(provider_status) - + return jsonify({ "status": "healthy" if available_count > 0 else "degraded", "providers": {"available": available_count, "total": total_count}, @@ -493,7 +792,7 @@ def health(): def status(): """详细状态""" refresh_config() - + providers_detail = [] for provider in _cached_providers: status_info = provider_status.get(provider['name'], {}) @@ -501,17 +800,19 @@ def status(): "name": provider['name'], "priority": provider['priority'], "enabled": provider['enabled'], - "available": status_info.get('available', True), + "available": is_provider_available(provider['name']), "error_count": status_info.get('error_count', 0), "last_error": status_info.get('last_error'), - "models": provider['models'], + "capabilities": provider.get('capabilities', ['text']), + "models": [{'name': m['name'], 'capabilities': m['capabilities']} for m in provider_models(provider)], }) - + return jsonify({ - "version": "2.0.0", + "version": VERSION, "uptime": time.time(), "providers": providers_detail, "model_aliases": _cached_aliases, + "auto_profiles": _cached_auto_profiles, }) @@ -524,6 +825,10 @@ def list_engines(): @app.route('/v1/engines//completions', methods=['POST']) def engine_completions(model): data = request.get_json() + if data is None: + data = {} + if not isinstance(data, dict): + return jsonify({"error": {"message": "Invalid request body", "type": "invalid_request_error"}}), 400 data['model'] = model return chat_completions() @@ -536,36 +841,44 @@ def admin_index(): @app.route('/admin/providers') -def admin_providers(): +def admin_providers_page(): return render_template('providers.html') @app.route('/admin/models') -def admin_models(): +def admin_models_page(): return render_template('models.html') @app.route('/admin/logs') -def admin_logs(): +def admin_logs_page(): return render_template('logs.html') @app.route('/admin/config') -def admin_config(): +def admin_config_page(): return render_template('config.html') @app.route('/admin/chat') -def admin_chat(): +def admin_chat_page(): return render_template('chat.html') @app.route('/admin/auto-profiles') -def admin_auto_profiles(): +def admin_auto_profiles_page(): return render_template('auto-profiles.html') -# ============ 后台管理 API 路由 ============ +# ============ 后台管理 API：能力 ============ + +@app.route('/api/admin/capabilities') +def api_admin_capabilities(): + """获取能力定义""" + return jsonify(CAPABILITY_DEFS) + + +# ============ 后台管理 API：统计 ============ @app.route('/api/admin/stats') def api_admin_stats(): @@ -573,51 +886,67 @@ def api_admin_stats(): stats = load_stats() providers = get_providers() refresh_provider_status() - - available_count = sum(1 for p in providers if provider_status.get(p['name'], {}).get('available', True)) - + + available_count = sum(1 for p in providers if is_provider_available(p['name'])) + return jsonify({ 'total_requests': stats.get('total_requests', 0), 'total_success': stats.get('total_success', 0), 'total_errors': stats.get('total_errors', 0), 'total_tokens': stats.get('total_tokens', 0), + 'requests_today': stats.get('requests_today', 0), 'providers_count': len(providers), 'available_providers': available_count, - 'models_count': len(get_model_aliases()), + 'models_count': sum(len(provider_model_names(p)) for p in providers), 'uptime': time.time(), }) +# ============ 后台管理 API：提供商 ============ + +def _provider_public(p): + """提供商对外数据结构（含能力与模型详情）""" + return { + 'id': p.get('id', p['name'].lower().replace(' ', '-')), + 'name': p['name'], + 'priority': p['priority'], + 'enabled': p.get('enabled', True), + 'available': is_provider_available(p['name']), + 'base_url': p['base_url'], + 'api_key': p['api_key'], + 'capabilities': p.get('capabilities', ['text']), + 'models': [{'name': m['name'], 'capabilities': m['capabilities']} for m in provider_models(p)], + 'default_model': p.get('default_model', ''), + 'timeout': p.get('timeout', 120), + 'request_count': 0, + 'success_count': 0, + 'error_count': 0, + 'last_error': None, + 'last_check': None, + } + + @app.route('/api/admin/providers') def api_admin_providers(): """获取提供商列表""" providers = get_providers() refresh_provider_status() stats = load_stats() + providers_data = [] - for provider in sorted(providers, key=lambda x: x['priority']): p_stats = stats.get('providers', {}).get(provider['name'], {}) p_status = provider_status.get(provider['name'], {}) - - providers_data.append({ - 'id': provider.get('id', provider['name'].lower().replace(' ', '-')), - 'name': provider['name'], - 'priority': provider['priority'], - 'enabled': provider['enabled'], - 'available': p_status.get('available', True), - 'base_url': provider['base_url'], - 'api_key': provider['api_key'], - 'models': provider['models'], - 'default_model': provider['default_model'], - 'timeout': provider.get('timeout', 120), + item = _provider_public(provider) + item.update({ 'request_count': p_stats.get('requests', 0), 'success_count': p_stats.get('success', 0), 'error_count': p_status.get('error_count', 0), 'last_error': p_status.get('last_error'), 'last_check': p_status.get('last_check'), }) - + providers_data.append(item) + return jsonify(providers_data) @@ -625,100 +954,137 @@ def api_admin_providers(): def api_admin_provider_detail(provider_id): """获取提供商详情""" provider = get_provider(provider_id) - + if not provider: return jsonify({'error': 'Provider not found'}), 404 - + stats = load_stats() p_stats = stats.get('providers', {}).get(provider['name'], {}) p_status = provider_status.get(provider['name'], {}) - - return jsonify({ - **provider, - 'status': { - 'available': p_status.get('available', True), - 'error_count': p_status.get('error_count', 0), - 'last_error': p_status.get('last_error'), - 'request_count': p_stats.get('requests', 0), - 'success_count': p_stats.get('success', 0), - } - }) + + item = _provider_public(provider) + item['status'] = { + 'available': is_provider_available(provider['name']), + 'error_count': p_status.get('error_count', 0), + 'last_error': p_status.get('last_error'), + 'request_count': p_stats.get('requests', 0), + 'success_count': p_stats.get('success', 0), + } + return jsonify(item) + + +def _sanitize_provider_input(data): + """清洗提供商输入""" + new_provider = {} + if 'id' in data: + new_provider['id'] = data['id'] + if 'name' in data: + new_provider['name'] = data['name'].strip() + if 'priority' in data: + try: + new_provider['priority'] = int(data['priority']) + except: + pass + if 'base_url' in data: + new_provider['base_url'] = data['base_url'].rstrip('/') + if 'api_key' in data: + new_provider['api_key'] = data['api_key'] + if 'timeout' in data: + try: + new_provider['timeout'] = int(data['timeout']) + except: + pass + if 'enabled' in data: + new_provider['enabled'] = bool(data['enabled']) + if 'capabilities' in data: + caps = data['capabilities'] + if isinstance(caps, str): + caps = [c.strip() for c in caps.split(',') if c.strip()] + new_provider['capabilities'] = list(caps) + if 'models' in data: + models = data['models'] + if isinstance(models, str): + models = [m.strip() for m in models.split(',') if m.strip()] + # 统一为 dict 形式，新模型默认继承提供商能力 + default_caps = new_provider.get('capabilities') or data.get('capabilities', ['text']) + norm = [] + for m in models: + if isinstance(m, dict): + norm.append({'name': m['name'], 'capabilities': m.get('capabilities', list(default_caps))}) + else: + norm.append({'name': m, 'capabilities': list(default_caps)}) + new_provider['models'] = norm + if 'default_model' in data and data['default_model']: + new_provider['default_model'] = data['default_model'] + return new_provider @app.route('/api/admin/providers', methods=['POST']) def api_admin_add_provider(): """添加新提供商""" data = request.get_json() - + if not data: return jsonify({'error': 'Invalid request body'}), 400 - + required = ['name', 'base_url', 'api_key', 'models'] for field in required: if not data.get(field): return jsonify({'error': f'Missing required field: {field}'}), 400 - + providers = get_providers() max_priority = max([p['priority'] for p in providers]) if providers else 0 - - new_provider = { - 'id': data.get('id') or data['name'].lower().replace(' ', '-').replace('.', '-'), - 'name': data['name'], - 'priority': data.get('priority', max_priority + 1), - 'base_url': data['base_url'].rstrip('/'), - 'api_key': data['api_key'], - 'models': data['models'] if isinstance(data['models'], list) else data['models'].split(','), - 'default_model': data.get('default_model', data['models'][0] if isinstance(data['models'], list) else data['models'].split(',')[0]), - 'timeout': data.get('timeout', 120), - 'enabled': data.get('enabled', True), - } - + + new_provider = _sanitize_provider_input(data) + new_provider.setdefault('id', data['name'].lower().replace(' ', '-').replace('.', '-')) + new_provider.setdefault('priority', max_priority + 1) + new_provider.setdefault('enabled', True) + new_provider.setdefault('capabilities', ['text']) + if not new_provider.get('default_model'): + names = [m['name'] for m in new_provider.get('models', [])] + new_provider['default_model'] = names[0] if names else '' + new_provider['priority'] = data.get('priority', max_priority + 1) + result = add_provider(new_provider) - + provider_status[result['name']] = { 'available': True, 'last_check': None, 'error_count': 0, 'last_error': None, } - - return jsonify({'success': True, 'provider': result}) + + return jsonify({'success': True, 'provider': _provider_public(result)}) @app.route('/api/admin/providers/', methods=['PUT']) def api_admin_update_provider(provider_id): """更新提供商""" data = request.get_json() - + if not data: return jsonify({'error': 'Invalid request body'}), 400 - - if 'models' in data and isinstance(data['models'], str): - data['models'] = [m.strip() for m in data['models'].split(',') if m.strip()] - - result = update_provider(provider_id, data) - + + result = update_provider(provider_id, _sanitize_provider_input(data)) + if not result: return jsonify({'error': 'Provider not found'}), 404 - - return jsonify({'success': True, 'provider': result}) + + return jsonify({'success': True, 'provider': _provider_public(result)}) @app.route('/api/admin/providers/', methods=['DELETE']) def api_admin_delete_provider(provider_id): """删除提供商""" - result = delete_provider(provider_id) - - if not result: + provider = get_provider(provider_id) + if not provider: return jsonify({'error': 'Provider not found'}), 404 - - providers = get_providers() - for p in providers: - if p.get('id') == provider_id: - if p['name'] in provider_status: - del provider_status[p['name']] - break - + + result = delete_provider(provider_id) + + if provider['name'] in provider_status: + del provider_status[provider['name']] + return jsonify({'success': True}) @@ -726,13 +1092,13 @@ def api_admin_delete_provider(provider_id): def api_admin_update_priority(): """更新优先级顺序""" data = request.get_json() - + if not data or 'order' not in data: return jsonify({'error': 'Missing order field'}), 400 - + provider_ids = data['order'] result = update_priority(provider_ids) - + return jsonify({'success': True, 'providers': result}) @@ -740,30 +1106,31 @@ def api_admin_update_priority(): def api_admin_toggle_provider(provider_id): """切换提供商启用状态""" provider = get_provider(provider_id) - + if not provider: return jsonify({'error': 'Provider not found'}), 404 - + new_enabled = not provider.get('enabled', True) result = update_provider(provider_id, {'enabled': new_enabled}) - + return jsonify({'success': True, 'enabled': new_enabled}) @app.route('/api/admin/providers//test', methods=['POST']) def api_admin_test_provider(provider_id): - """测试提供商连接""" + """测试提供商连接（优先 /models，失败则用聊天探测，适配 new-api 等无 /models 的服务）""" provider = get_provider(provider_id) - + if not provider: return jsonify({'success': False, 'error': 'Provider not found'}), 404 - + + # 方式1: GET /models try: url = f"{provider['base_url'].rstrip('/')}/models" headers = {"Authorization": f"Bearer {provider['api_key']}"} - + response = requests.get(url, headers=headers, timeout=10) - + if response.status_code == 200: provider_status[provider['name']] = { 'available': True, @@ -777,17 +1144,44 @@ def api_admin_test_provider(provider_id): models_data = resp_json.get('data', []) except: pass - + return jsonify({'success': True, 'message': 'Connection successful', 'models_count': len(models_data)}) - else: + except Exception as e: + pass # 继续尝试聊天探测 + + # 方式2: 能力探测（用 default_model 发一条最小请求，按能力选择探测端点） + try: + default_model = provider.get('default_model') or '' + probe_model = default_model if default_model in provider_model_names(provider) else (provider_model_names(provider) or [''])[0] + if probe_model: + caps = model_capabilities(provider, probe_model) + if 'image_gen' in caps: + # 生图提供商：探测 /images/generations + url = f"{provider['base_url'].rstrip('/')}/images/generations" + headers = {"Authorization": f"Bearer {provider['api_key']}", "Content-Type": "application/json"} + payload = {"model": probe_model, "prompt": "a tiny red dot", "n": 1} + probe_name = 'image generation' + else: + url = f"{provider['base_url'].rstrip('/')}/chat/completions" + headers = {"Authorization": f"Bearer {provider['api_key']}", "Content-Type": "application/json"} + payload = {"model": probe_model, "messages": [{"role": "user", "content": "hi"}], "max_tokens": 1, "stream": False} + probe_name = 'chat' + resp = requests.post(url, headers=headers, json=payload, timeout=20) + if resp.status_code in (200, 201): + provider_status[provider['name']] = { + 'available': True, + 'last_check': datetime.now().isoformat(), + 'error_count': 0, + 'last_error': None, + } + return jsonify({'success': True, 'message': f'Connection successful ({probe_name} probe, model={probe_model})', 'models_count': None}) provider_status[provider['name']] = { 'available': False, 'last_check': datetime.now().isoformat(), 'error_count': provider_status.get(provider['name'], {}).get('error_count', 0) + 1, - 'last_error': f'HTTP {response.status_code}', + 'last_error': f'HTTP {resp.status_code}', } - return jsonify({'success': False, 'error': f'HTTP {response.status_code}: {response.text[:200]}'}) - + return jsonify({'success': False, 'error': f'HTTP {resp.status_code}: {resp.text[:200]}'}) except Exception as e: provider_status[provider['name']] = { 'available': False, @@ -797,52 +1191,329 @@ def api_admin_test_provider(provider_id): } return jsonify({'success': False, 'error': str(e)}) + provider_status[provider['name']] = { + 'available': False, + 'last_check': datetime.now().isoformat(), + 'error_count': provider_status.get(provider['name'], {}).get('error_count', 0) + 1, + 'last_error': 'Connection failed', + } + return jsonify({'success': False, 'error': 'Connection failed'}) + + +# ============ 后台管理 API：模型管理 ============ @app.route('/api/admin/models') def api_admin_models(): - """获取模型列表""" + """获取所有模型（按提供商分组，含能力）""" providers = get_providers() aliases = get_model_aliases() - - models_list = [] - added = set() - - models_list.append({'alias': 'auto', 'target': 'auto', 'description': '自动选择可用模型'}) - added.add('auto') - - for provider in sorted(providers, key=lambda x: x['priority']): - for model in provider['models']: - if model not in added: - models_list.append({ - 'alias': model, - 'target': model, - 'provider': provider['name'], - 'priority': provider['priority'], - }) - added.add(model) - - for alias, target in aliases.items(): - if alias != 'auto' and alias not in added: - provider_name = None - for p in providers: - if target in p['models']: - provider_name = p['name'] - break - models_list.append({'alias': alias, 'target': target, 'provider': provider_name}) - - return jsonify(models_list) + result = [] + for provider in sorted(providers, key=lambda x: x['priority']): + default_model = provider.get('default_model', '') + model_aliases = {} + for alias, target in aliases.items(): + if alias == 'auto' or alias.startswith('auto-'): + continue + if target in provider_model_names(provider): + model_aliases.setdefault(target, []).append(alias) + for m in provider_models(provider): + result.append({ + 'name': m['name'], + 'capabilities': m['capabilities'], + 'provider_id': provider.get('id'), + 'provider_name': provider['name'], + 'provider_priority': provider['priority'], + 'provider_enabled': provider.get('enabled', True), + 'is_default': m['name'] == default_model, + 'aliases': model_aliases.get(m['name'], []), + }) + + return jsonify(result) + + +@app.route('/api/admin/models', methods=['POST']) +def api_admin_add_model(): + """向提供商添加模型""" + data = request.get_json() + + if not data: + return jsonify({'error': 'Invalid request body'}), 400 + + provider_id = data.get('provider_id') + name = (data.get('name') or '').strip() + if not provider_id or not name: + return jsonify({'error': 'provider_id and name are required'}), 400 + + provider = get_provider(provider_id) + if not provider: + return jsonify({'error': 'Provider not found'}), 404 + + caps = data.get('capabilities') or provider.get('capabilities', ['text']) + if isinstance(caps, str): + caps = [c.strip() for c in caps.split(',') if c.strip()] + + if name in provider_model_names(provider): + return jsonify({'error': 'Model already exists in this provider'}), 400 + + models = provider.get('models', []) + models.append({'name': name, 'capabilities': list(caps)}) + result = update_provider(provider_id, {'models': models}) + + return jsonify({'success': True, 'model': {'name': name, 'capabilities': list(caps)}}) + + +@app.route('/api/admin/models/', methods=['PUT']) +def api_admin_update_model(provider_id): + """更新模型（能力、名称）——模型名放 body，避免路径斜杠问题""" + data = request.get_json() + + if not data: + return jsonify({'error': 'Invalid request body'}), 400 + + old_name = data.get('name') or data.get('old_name') + if not old_name: + return jsonify({'error': 'Model name is required'}), 400 + + provider = get_provider(provider_id) + if not provider: + return jsonify({'error': 'Provider not found'}), 404 + + models = provider.get('models', []) + found = False + for m in models: + mname = m['name'] if isinstance(m, dict) else m + if mname == old_name: + found = True + if 'new_name' in data and data['new_name']: + m['name'] = data['new_name'] + if 'capabilities' in data: + caps = data['capabilities'] + if isinstance(caps, str): + caps = [c.strip() for c in caps.split(',') if c.strip()] + m['capabilities'] = list(caps) + break + + if not found: + return jsonify({'error': 'Model not found'}), 404 + + # 同步 default_model + if provider.get('default_model') == old_name and 'new_name' in data and data['new_name']: + provider['default_model'] = data['new_name'] + + result = update_provider(provider_id, {'models': models, 'default_model': provider.get('default_model', '')}) + return jsonify({'success': True, 'provider': _provider_public(result)}) + + +@app.route('/api/admin/models/', methods=['DELETE']) +def api_admin_delete_model(provider_id): + """删除模型——模型名放 query，避免路径斜杠问题""" + name = request.args.get('name', '') + if not name: + return jsonify({'error': 'Model name is required (query param name=)'}), 400 + + provider = get_provider(provider_id) + if not provider: + return jsonify({'error': 'Provider not found'}), 404 + + models = [m for m in provider.get('models', []) if (m['name'] if isinstance(m, dict) else m) != name] + + update_data = {'models': models} + if provider.get('default_model') == name: + names = [m['name'] for m in provider_models({'models': models, 'capabilities': provider.get('capabilities', ['text'])})] + update_data['default_model'] = names[0] if names else '' + update_provider(provider_id, update_data) + + return jsonify({'success': True}) + + +@app.route('/api/admin/models//default', methods=['POST']) +def api_admin_set_default_model(provider_id): + """设置默认模型""" + data = request.get_json() or {} + name = data.get('name', '') + if not name: + return jsonify({'error': 'Model name is required'}), 400 + + provider = get_provider(provider_id) + if not provider: + return jsonify({'error': 'Provider not found'}), 404 + + if name not in provider_model_names(provider): + return jsonify({'error': 'Model not found in provider'}), 404 + + result = update_provider(provider_id, {'default_model': name}) + return jsonify({'success': True, 'default_model': name}) + + +# ============ 后台管理 API：模型别名 ============ + +@app.route('/api/admin/aliases') +def api_admin_aliases(): + """获取模型别名""" + return jsonify(get_model_aliases()) + + +@app.route('/api/admin/aliases', methods=['POST']) +def api_admin_add_alias(): + """添加/更新模型别名""" + data = request.get_json() + + if not data or not data.get('alias') or not data.get('target'): + return jsonify({'error': 'alias and target are required'}), 400 + + alias = data['alias'].strip() + target = data['target'].strip() + + if alias != 'auto' and alias.startswith('auto-'): + return jsonify({'error': 'auto- 前缀保留给Auto配置'}), 400 + + result = update_model_alias(alias, target) + return jsonify({'success': True, 'aliases': result}) + + +@app.route('/api/admin/aliases/', methods=['DELETE']) +def api_admin_delete_alias(alias): + """删除模型别名""" + result = delete_model_alias(alias) + if not result: + return jsonify({'error': 'Cannot delete alias or alias not found'}), 400 + return jsonify({'success': True}) + + +# ============ 后台管理 API：Auto配置 ============ + +def _auto_profile_public(name, profile): + """Auto配置对外结构（含能力与匹配模型预览）""" + providers = get_providers() + allowed_providers = profile.get('providers', ['*']) + capability = profile.get('capability', 'text') + + provider_details = [] + matched_models = [] + for p in sorted(providers, key=lambda x: x['priority']): + if not ('*' in allowed_providers or p.get('id') in allowed_providers or p['name'] in allowed_providers): + continue + models = provider_capability_models(p, capability) + if models: + provider_details.append({'id': p.get('id'), 'name': p['name'], 'priority': p['priority'], 'selected': True, 'matched_models': models}) + matched_models.extend(models) + + return { + 'name': name, + 'display_name': profile.get('name', name), + 'description': profile.get('description', ''), + 'capability': capability, + 'capability_label': CAPABILITY_DEFS.get(capability, capability), + 'strategy': profile.get('strategy', 'priority'), + 'providers': allowed_providers, + 'provider_details': provider_details, + 'matched_models': matched_models, + } + + +@app.route('/api/admin/auto-profiles') +def api_admin_auto_profiles(): + """获取所有Auto配置""" + profiles = get_auto_profiles() + result = [_auto_profile_public(name, profile) for name, profile in profiles.items()] + return jsonify(result) + + +@app.route('/api/admin/auto-profiles/', methods=['GET']) +def api_admin_auto_profile_detail(profile_name): + """获取单个Auto配置详情""" + profile = get_auto_profile(profile_name) + + if not profile: + return jsonify({'error': 'Profile not found'}), 404 + + return jsonify(_auto_profile_public(profile_name, profile)) + + +@app.route('/api/admin/auto-profiles', methods=['POST']) +def api_admin_add_auto_profile(): + """添加新的Auto配置""" + data = request.get_json() + + if not data or not data.get('name'): + return jsonify({'error': 'Missing profile name'}), 400 + + profile_name = data['name'].lower().replace(' ', '-').replace('.', '-') + + if profile_name in get_auto_profiles(): + return jsonify({'error': 'Profile already exists'}), 400 + + capability = data.get('capability', 'text') + if capability not in CAPABILITY_DEFS: + return jsonify({'error': f'Invalid capability: {capability}. Valid: {list(CAPABILITY_DEFS.keys())}'}), 400 + + profile_data = { + 'name': data.get('display_name', data['name']), + 'description': data.get('description', ''), + 'capability': capability, + 'providers': data.get('providers', ['*']), + 'strategy': data.get('strategy', 'priority'), + } + + result = add_auto_profile(profile_name, profile_data) + + return jsonify({'success': True, 'profile': _auto_profile_public(profile_name, profile_data)}) + + +@app.route('/api/admin/auto-profiles/', methods=['PUT']) +def api_admin_update_auto_profile(profile_name): + """更新Auto配置""" + data = request.get_json() + + if not data: + return jsonify({'error': 'Invalid request body'}), 400 + + profile_data = {} + if 'display_name' in data: + profile_data['name'] = data['display_name'] + if 'description' in data: + profile_data['description'] = data['description'] + if 'capability' in data: + if data['capability'] not in CAPABILITY_DEFS: + return jsonify({'error': f'Invalid capability: {data["capability"]}'}), 400 + profile_data['capability'] = data['capability'] + if 'providers' in data: + profile_data['providers'] = data['providers'] + if 'strategy' in data: + profile_data['strategy'] = data['strategy'] + + result = update_auto_profile(profile_name, profile_data) + + if not result: + return jsonify({'error': 'Profile not found'}), 404 + + return jsonify({'success': True, 'profile': _auto_profile_public(profile_name, result)}) + + +@app.route('/api/admin/auto-profiles/', methods=['DELETE']) +def api_admin_delete_auto_profile(profile_name): + """删除Auto配置""" + result = delete_auto_profile(profile_name) + + if not result: + return jsonify({'error': 'Cannot delete default auto profile or profile not found'}), 400 + + return jsonify({'success': True}) + + +# ============ 后台管理 API：日志/配置 ============ @app.route('/api/admin/logs') def api_admin_logs(): """获取日志""" log_file = LOGS_DIR / 'proxy.log' - + lines = [] if log_file.exists(): content = log_file.read_text(encoding='utf-8') - lines = content.strip().split('\n')[-100:] - + lines = content.strip().split('\n')[-200:] + return jsonify({'logs': lines, 'total_lines': len(lines)}) @@ -851,172 +1522,68 @@ def api_admin_config(): """获取配置""" providers = get_providers() aliases = get_model_aliases() - + return jsonify({ 'providers': [{ 'id': p.get('id', p['name'].lower().replace(' ', '-')), 'name': p['name'], 'priority': p['priority'], 'base_url': p['base_url'], - 'models': p['models'], + 'capabilities': p.get('capabilities', ['text']), + 'models': [{'name': m['name'], 'capabilities': m['capabilities']} for m in provider_models(p)], + 'default_model': p.get('default_model', ''), 'timeout': p.get('timeout', 120), 'enabled': p['enabled'], } for p in providers], 'model_aliases': aliases, + 'auto_profiles': get_auto_profiles(), 'retry_config': RETRY_CONFIG, + 'capabilities': CAPABILITY_DEFS, 'server_config': {'port': SERVER_CONFIG['port']} }) -# ============ Auto配置管理 ============ +# ============ 后台管理 API：对话 ============ -@app.route('/api/admin/auto-profiles') -def api_admin_auto_profiles(): - """获取所有Auto配置""" - profiles = get_auto_profiles() - providers = get_providers() - - result = [] - for name, profile in profiles.items(): - allowed_providers = profile.get('providers', ['*']) - provider_details = [] - - if '*' in allowed_providers: - provider_details = [{'id': '*', 'name': '所有启用的提供商'}] - else: - for p in providers: - if p.get('id') in allowed_providers or p['name'] in allowed_providers: - provider_details.append({'id': p.get('id'), 'name': p['name'], 'priority': p['priority']}) - - result.append({ - 'name': name, - 'display_name': profile.get('name', name), - 'description': profile.get('description', ''), - 'strategy': profile.get('strategy', 'priority'), - 'providers': allowed_providers, - 'provider_details': provider_details, - }) - - return jsonify(result) +def load_chats(): + """加载对话数据""" + if CHATS_FILE.exists(): + try: + return json.loads(CHATS_FILE.read_text(encoding='utf-8')) + except: + pass + return {'chats': []} -@app.route('/api/admin/auto-profiles/', methods=['GET']) -def api_admin_auto_profile_detail(profile_name): - """获取单个Auto配置详情""" - profile = get_auto_profile(profile_name) - - if not profile: - return jsonify({'error': 'Profile not found'}), 404 - - providers = get_providers() - allowed_providers = profile.get('providers', ['*']) - provider_details = [] - - if '*' in allowed_providers: - provider_details = [{'id': '*', 'name': '所有启用的提供商', 'selected': True}] - for p in providers: - provider_details.append({'id': p.get('id'), 'name': p['name'], 'priority': p['priority'], 'selected': True}) - else: - for p in providers: - selected = p.get('id') in allowed_providers or p['name'] in allowed_providers - provider_details.append({'id': p.get('id'), 'name': p['name'], 'priority': p['priority'], 'selected': selected}) - - return jsonify({ - 'name': profile_name, - 'display_name': profile.get('name', profile_name), - 'description': profile.get('description', ''), - 'strategy': profile.get('strategy', 'priority'), - 'providers': allowed_providers, - 'provider_details': provider_details, - }) +def save_chats(data): + """保存对话数据""" + CHATS_FILE.write_text(json.dumps(data, ensure_ascii=False, indent=2), encoding='utf-8') -@app.route('/api/admin/auto-profiles', methods=['POST']) -def api_admin_add_auto_profile(): - """添加新的Auto配置""" - data = request.get_json() - - if not data or not data.get('name'): - return jsonify({'error': 'Missing profile name'}), 400 - - profile_name = data['name'].lower().replace(' ', '-').replace('.', '-') - - if profile_name in get_auto_profiles(): - return jsonify({'error': 'Profile already exists'}), 400 - - profile_data = { - 'name': data.get('display_name', data['name']), - 'description': data.get('description', ''), - 'providers': data.get('providers', ['*']), - 'strategy': data.get('strategy', 'priority'), - } - - result = add_auto_profile(profile_name, profile_data) - - return jsonify({'success': True, 'profile': {profile_name: profile_data}}) - - -@app.route('/api/admin/auto-profiles/', methods=['PUT']) -def api_admin_update_auto_profile(profile_name): - """更新Auto配置""" - data = request.get_json() - - if not data: - return jsonify({'error': 'Invalid request body'}), 400 - - profile_data = {} - if 'display_name' in data: - profile_data['name'] = data['display_name'] - if 'description' in data: - profile_data['description'] = data['description'] - if 'providers' in data: - profile_data['providers'] = data['providers'] - if 'strategy' in data: - profile_data['strategy'] = data['strategy'] - - result = update_auto_profile(profile_name, profile_data) - - if not result: - return jsonify({'error': 'Profile not found'}), 404 - - return jsonify({'success': True, 'profile': result}) - - -@app.route('/api/admin/auto-profiles/', methods=['DELETE']) -def api_admin_delete_auto_profile(profile_name): - """删除Auto配置""" - result = delete_auto_profile(profile_name) - - if not result: - return jsonify({'error': 'Cannot delete default auto profile or profile not found'}), 400 - - return jsonify({'success': True}) - - -# ============ 对话功能 ============ - @app.route('/api/admin/chat/models') def api_admin_chat_models(): - """获取可用模型列表""" + """获取可用模型列表（含auto配置，标注能力）""" providers = get_providers() profiles = get_auto_profiles() - + models = [] added = set() - + for name, profile in profiles.items(): if name not in added: - models.append({'id': name, 'description': profile.get('description', 'Auto-select')}) + cap = profile.get('capability', 'text') + models.append({'id': name, 'description': f"[{CAPABILITY_DEFS.get(cap, cap)}] {profile.get('description', 'Auto-select')}"}) added.add(name) - + for provider in providers: if not provider['enabled']: continue - for model in provider['models']: - if model not in added: - models.append({'id': model, 'description': provider['name']}) - added.add(model) - + for m in provider_models(provider): + if m['name'] not in added: + caps = ','.join(CAPABILITY_DEFS.get(c, c) for c in m['capabilities']) + models.append({'id': m['name'], 'description': f"{provider['name']} ({caps})"}) + added.add(m['name']) + return jsonify(models) @@ -1024,7 +1591,7 @@ def api_admin_chat_models(): def api_admin_chat_list(): """获取对话列表""" data = load_chats() - + chats = [] for chat in data.get('chats', []): chats.append({ @@ -1035,9 +1602,9 @@ def api_admin_chat_list(): 'created_at': chat.get('created_at'), 'updated_at': chat.get('updated_at') }) - + chats.sort(key=lambda x: x.get('updated_at', ''), reverse=True) - + return jsonify(chats) @@ -1045,11 +1612,11 @@ def api_admin_chat_list(): def api_admin_chat_detail(chat_id): """获取对话详情""" data = load_chats() - + for chat in data.get('chats', []): if chat['id'] == chat_id: return jsonify(chat) - + return jsonify({'error': 'Chat not found'}), 404 @@ -1057,83 +1624,92 @@ def api_admin_chat_detail(chat_id): def api_admin_chat_send(): """发送消息""" req = request.get_json() - + user_message = req.get('message', '') model = req.get('model', 'auto') chat_id = req.get('chat_id') - + if not user_message: return jsonify({'error': 'Message is required'}), 400 - - data = load_chats() - - chat = None - if chat_id: - for c in data['chats']: - if c['id'] == chat_id: - chat = c - break - - if not chat: - chat_id = str(uuid.uuid4())[:8] - chat = { - 'id': chat_id, - 'title': '新对话', - 'model': model, - 'messages': [], - 'created_at': datetime.now().isoformat(), - 'updated_at': datetime.now().isoformat() - } - data['chats'].append(chat) - - chat['messages'].append({ - 'role': 'user', - 'content': user_message, - 'time': datetime.now().isoformat() - }) - + + with chats_lock: + data = load_chats() + + chat = None + if chat_id: + for c in data['chats']: + if c['id'] == chat_id: + chat = c + break + + if not chat: + chat_id = str(uuid.uuid4())[:8] + chat = { + 'id': chat_id, + 'title': '新对话', + 'model': model, + 'messages': [], + 'created_at': datetime.now().isoformat(), + 'updated_at': datetime.now().isoformat() + } + data['chats'].append(chat) + + # 先保存用户消息，避免上游失败时丢数据 + chat['messages'].append({ + 'role': 'user', + 'content': user_message, + 'time': datetime.now().isoformat() + }) + save_chats(data) + try: proxy_url = f"http://localhost:{SERVER_CONFIG['port']}/v1/chat/completions" - + messages = [] for msg in chat['messages'][-20:]: messages.append({'role': msg['role'], 'content': msg['content']}) - + response = requests.post(proxy_url, json={ 'model': model, 'messages': messages, 'stream': False - }, timeout=120) - + }, timeout=180) + if response.status_code == 200: result = response.json() assistant_message = result['choices'][0]['message']['content'] used_model = result.get('model', model) - - chat['messages'].append({ - 'role': 'assistant', - 'content': assistant_message, - 'model': used_model, - 'time': datetime.now().isoformat() - }) - - if len(chat['messages']) == 2: - chat['title'] = user_message[:30] + ('...' if len(user_message) > 30 else '') - - chat['updated_at'] = datetime.now().isoformat() - save_chats(data) - + + with chats_lock: + data = load_chats() + for c in data['chats']: + if c['id'] == chat_id: + c['messages'].append({ + 'role': 'assistant', + 'content': assistant_message, + 'model': used_model, + 'time': datetime.now().isoformat() + }) + if len(c['messages']) == 2: + c['title'] = user_message[:30] + ('...' if len(user_message) > 30 else '') + c['updated_at'] = datetime.now().isoformat() + break + save_chats(data) + return jsonify({ 'success': True, 'chat_id': chat_id, 'response': assistant_message, 'model': used_model, - 'title': chat['title'] + 'title': chat.get('title', '新对话') }) else: - error_msg = response.json().get('error', {}).get('message', 'Unknown error') + try: + error_msg = response.json().get('error', {}).get('message', 'Unknown error') + except: + error_msg = f'HTTP {response.status_code}' return jsonify({'error': error_msg}), response.status_code - + except Exception as e: return jsonify({'error': str(e)}), 500 @@ -1141,44 +1717,51 @@ def api_admin_chat_send(): @app.route('/api/admin/chat/', methods=['DELETE']) def api_admin_delete_chat(chat_id): """删除对话""" - data = load_chats() - data['chats'] = [c for c in data['chats'] if c['id'] != chat_id] - save_chats(data) + with chats_lock: + data = load_chats() + data['chats'] = [c for c in data['chats'] if c['id'] != chat_id] + save_chats(data) return jsonify({'success': True}) @app.route('/api/admin/chat//clear', methods=['POST']) def api_admin_clear_chat(chat_id): """清空对话消息""" - data = load_chats() - - for chat in data['chats']: - if chat['id'] == chat_id: - chat['messages'] = [] - chat['updated_at'] = datetime.now().isoformat() - save_chats(data) - return jsonify({'success': True}) - + with chats_lock: + data = load_chats() + + for chat in data['chats']: + if chat['id'] == chat_id: + chat['messages'] = [] + chat['updated_at'] = datetime.now().isoformat() + save_chats(data) + return jsonify({'success': True}) + return jsonify({'error': 'Chat not found'}), 404 if __name__ == '__main__': refresh_config() - + print("=" * 60) - print("大模型API中转系统 v2.0.0") + print(f"大模型API中转系统 v{VERSION}") print("=" * 60) print(f"API地址: http://localhost:{SERVER_CONFIG['port']}") print(f"后台管理: http://localhost:{SERVER_CONFIG['port']}/admin") print("=" * 60) print("上游提供商:") for p in sorted(_cached_providers, key=lambda x: x['priority']): - print(f" [{p['priority']}] {p['name']}: {p['base_url']}") - print(f" 模型: {', '.join(p['models'])}") + caps = ','.join(CAPABILITY_DEFS.get(c, c) for c in p.get('capabilities', ['text'])) + print(f" [{p['priority']}] {p['name']}: {p['base_url']} ({caps})") + for m in provider_models(p): + mcaps = ','.join(CAPABILITY_DEFS.get(c, c) for c in m['capabilities']) + marker = " [默认]" if m['name'] == p.get('default_model') else "" + print(f" - {m['name']} ({mcaps}){marker}") print("=" * 60) - + app.run( host=SERVER_CONFIG['host'], port=SERVER_CONFIG['port'], - debug=SERVER_CONFIG['debug'] - ) \ No newline at end of file + debug=SERVER_CONFIG['debug'], + threaded=True + ) diff --git a/config/settings.py b/config/settings.py index 54c5650..b36b5bb 100644 --- a/config/settings.py +++ b/config/settings.py @@ -1,5 +1,14 @@ """ 大模型API中转系统配置 - 支持动态修改 +v2.1.0 - 能力(Capability)体系: 每个模型标记能力，AUTO配置按能力绑定 + +能力类型: + text 文本推理 + vision 视觉能力 (多模态输入) + audio_out 语音输出 (TTS) + audio_in 语音输入 (ASR) + image_gen 图片生成 + video_gen 视频生成 """ import json @@ -8,28 +17,76 @@ from pathlib import Path # 配置文件路径 CONFIG_FILE = Path(__file__).parent.parent / 'data' / 'config.json' +# 能力定义 +CAPABILITY_DEFS = { + "text": "文本推理", + "vision": "视觉能力", + "audio_out": "语音输出", + "audio_in": "语音输入", + "image_gen": "图片生成", + "video_gen": "视频生成", +} + # 默认上游模型配置 +# capabilities: 该提供商下所有模型的默认能力（模型管理页可对单个模型覆盖） DEFAULT_PROVIDERS = [ { "id": "local-qwen", "name": "Local Qwen", "priority": 1, - "base_url": "http://192.168.2.5:1234/v1", - "api_key": "sk-lm-fuP5tGU8:Hi7YU87jHyDP6Ay8Tl2j", - "models": ["qwen3.5-4b", "qwen3.5", "qwen"], - "default_model": "qwen3.5-4b", - "timeout": 120, + "base_url": "http://121.40.164.32:18003/v1", + "api_key": "sk-xxxx", + "models": [ + {"name": "unsloth/Qwen3.8-27B-Q6_K", "capabilities": ["text", "vision"]}, + {"name": "unsloth/Qwen3.8-27B-Q4_K_M", "capabilities": ["text", "vision"]}, + ], + "default_model": "unsloth/Qwen3.8-27B-Q6_K", + "capabilities": ["text", "vision"], + "timeout": 180, "enabled": True, }, { - "id": "siliconflow-deepseek", - "name": "SiliconFlow DeepSeek", + "id": "siliconflow-llm", + "name": "SiliconFlow LLM", "priority": 2, "base_url": "https://api.siliconflow.cn/v1", "api_key": "sk-fhpoexpptvjghpnphtaxbkhjwulzovoqfffbckcfscjmwhcg", - "models": ["Pro/deepseek-ai/DeepSeek-V3.2", "deepseek-v3", "deepseek"], - "default_model": "Pro/deepseek-ai/DeepSeek-V3.2", - "timeout": 120, + "models": [ + {"name": "deepseek-ai/DeepSeek-V4-Flash", "capabilities": ["text"]}, + {"name": "meituan-longcat/LongCat-2.0", "capabilities": ["text"]}, + ], + "default_model": "deepseek-ai/DeepSeek-V4-Flash", + "capabilities": ["text"], + "timeout": 180, + "enabled": True, + }, + { + "id": "autodl", + "name": "Autodl", + "priority": 3, + "base_url": "https://www.autodl.art/api/v1", + "api_key": "F9MBfolzuapqTsD4KmUf9qen720rXvUZ3Sp3IrWiCTukqonx", + "models": [ + {"name": "qwen3.6-plus", "capabilities": ["text", "vision"]}, + {"name": "GLM-5.3-flash", "capabilities": ["text", "vision"]}, + ], + "default_model": "GLM-5.3-flash", + "capabilities": ["text", "vision"], + "timeout": 180, + "enabled": True, + }, + { + "id": "autodl-image", + "name": "Autodl Image", + "priority": 4, + "base_url": "https://www.autodl.art/api/v1", + "api_key": "F9MBfolzuapqTsD4KmUf9qen720rXvUZ3Sp3IrWiCTukqonx", + "models": [ + {"name": "Qwen-Image", "capabilities": ["image_gen"]}, + ], + "default_model": "Qwen-Image", + "capabilities": ["image_gen"], + "timeout": 180, "enabled": True, }, ] @@ -37,24 +94,80 @@ DEFAULT_PROVIDERS = [ # 默认模型别名 DEFAULT_MODEL_ALIASES = { "auto": "auto", - "qwen": "qwen3.5-4b", - "qwen3.5": "qwen3.5-4b", - "qwen3.5-4b": "qwen3.5-4b", - "deepseek": "Pro/deepseek-ai/DeepSeek-V3.2", - "deepseek-v3": "Pro/deepseek-ai/DeepSeek-V3.2", - "deepseek-v3.2": "Pro/deepseek-ai/DeepSeek-V3.2", + "auto-text": "auto-text", + "auto-vision": "auto-vision", + "auto-image": "auto-image", + "auto-voice-out": "auto-voice-out", + "auto-voice-in": "auto-voice-in", + "auto-video": "auto-video", + "auto-vlm": "auto-vision", # 兼容旧配置 + "qwen": "unsloth/Qwen3.8-27B-Q6_K", + "qwen3.8": "unsloth/Qwen3.8-27B-Q6_K", + "local": "unsloth/Qwen3.8-27B-Q6_K", + "deepseek": "deepseek-ai/DeepSeek-V4-Flash", + "deepseek-v4-flash": "deepseek-ai/DeepSeek-V4-Flash", + "longcat": "meituan-longcat/LongCat-2.0", + "glm": "GLM-5.3-flash", + "qwen3.6-plus": "qwen3.6-plus", + "gpt-4": "GLM-5.3-flash", } -# 默认Auto配置（自定义候选模型和优先级） +# 默认Auto配置：每个auto配置固定绑定一个能力 +# capability: 固定此auto的功能类型（取自 CAPABILITY_DEFS） +# providers: 候选提供商（* 表示所有启用的、且具备该能力模型的提供商） DEFAULT_AUTO_PROFILES = { "auto": { "name": "默认Auto", - "description": "所有启用的提供商按优先级自动选择", - "providers": ["*"], # * 表示所有启用的提供商 - "strategy": "priority", # priority | random | round-robin - } + "description": "文本推理 - 自动选择可用提供商", + "capability": "text", + "providers": ["*"], + "strategy": "priority", + }, + "auto-text": { + "name": "文本推理", + "description": "纯文本推理，自动选择文本模型", + "capability": "text", + "providers": ["*"], + "strategy": "priority", + }, + "auto-vision": { + "name": "视觉能力", + "description": "多模态视觉理解，自动选择视觉模型", + "capability": "vision", + "providers": ["*"], + "strategy": "priority", + }, + "auto-image": { + "name": "图片生成", + "description": "文生图，自动选择生图模型", + "capability": "image_gen", + "providers": ["*"], + "strategy": "priority", + }, + "auto-voice-out": { + "name": "语音输出", + "description": "语音合成(TTS)，自动选择语音输出模型", + "capability": "audio_out", + "providers": ["*"], + "strategy": "priority", + }, + "auto-voice-in": { + "name": "语音输入", + "description": "语音识别(ASR)，自动选择语音输入模型", + "capability": "audio_in", + "providers": ["*"], + "strategy": "priority", + }, + "auto-video": { + "name": "视频生成", + "description": "文生视频，自动选择视频生成模型", + "capability": "video_gen", + "providers": ["*"], + "strategy": "priority", + }, } + def load_config(): """加载配置""" if CONFIG_FILE.exists(): @@ -68,16 +181,19 @@ def load_config(): "model_aliases": DEFAULT_MODEL_ALIASES, } + def save_config(config): """保存配置""" CONFIG_FILE.parent.mkdir(parents=True, exist_ok=True) CONFIG_FILE.write_text(json.dumps(config, ensure_ascii=False, indent=2), encoding='utf-8') + def get_providers(): """获取提供商列表""" config = load_config() return config.get("providers", DEFAULT_PROVIDERS) + def get_provider(provider_id): """获取单个提供商""" providers = get_providers() @@ -86,25 +202,27 @@ def get_provider(provider_id): return p return None + def add_provider(provider): """添加提供商""" config = load_config() providers = config.get("providers", []) - + # 生成ID if not provider.get("id"): provider["id"] = provider["name"].lower().replace(" ", "-").replace(".", "-") - + providers.append(provider) config["providers"] = providers save_config(config) return provider + def update_provider(provider_id, data): """更新提供商""" config = load_config() providers = config.get("providers", []) - + for i, p in enumerate(providers): if p.get("id") == provider_id: providers[i] = {**p, **data} @@ -113,6 +231,7 @@ def update_provider(provider_id, data): return providers[i] return None + def delete_provider(provider_id): """删除提供商""" config = load_config() @@ -122,26 +241,29 @@ def delete_provider(provider_id): save_config(config) return True + def update_priority(provider_ids): """更新优先级顺序""" config = load_config() providers = config.get("providers", []) - + # 按新顺序设置优先级 for i, pid in enumerate(provider_ids): for p in providers: if p.get("id") == pid: p["priority"] = i + 1 - + config["providers"] = providers save_config(config) return providers + def get_model_aliases(): """获取模型别名""" config = load_config() return config.get("model_aliases", DEFAULT_MODEL_ALIASES) + def update_model_alias(alias, target): """更新模型别名""" config = load_config() @@ -151,16 +273,33 @@ def update_model_alias(alias, target): save_config(config) return aliases + +def delete_model_alias(alias): + """删除模型别名""" + if alias == "auto": + return False # 不能删除默认的auto + config = load_config() + aliases = config.get("model_aliases", {}) + if alias in aliases: + del aliases[alias] + config["model_aliases"] = aliases + save_config(config) + return True + return False + + def get_auto_profiles(): """获取Auto配置列表""" config = load_config() return config.get("auto_profiles", DEFAULT_AUTO_PROFILES) + def get_auto_profile(profile_name): """获取单个Auto配置""" profiles = get_auto_profiles() return profiles.get(profile_name) + def add_auto_profile(profile_name, profile_data): """添加Auto配置""" config = load_config() @@ -170,6 +309,7 @@ def add_auto_profile(profile_name, profile_data): save_config(config) return profiles + def update_auto_profile(profile_name, profile_data): """更新Auto配置""" config = load_config() @@ -181,6 +321,7 @@ def update_auto_profile(profile_name, profile_data): return profiles[profile_name] return None + def delete_auto_profile(profile_name): """删除Auto配置""" if profile_name == "auto": @@ -194,6 +335,7 @@ def delete_auto_profile(profile_name): return True return False + # 初始化配置 config = load_config() UPSTREAM_PROVIDERS = config.get("providers", DEFAULT_PROVIDERS) @@ -202,8 +344,8 @@ MODEL_ALIASES = config.get("model_aliases", DEFAULT_MODEL_ALIASES) # 服务配置 SERVER_CONFIG = { "host": "0.0.0.0", - "port": 19007, - "debug": True, + "port": 16003, + "debug": False, } # 日志配置 @@ -213,14 +355,15 @@ LOG_CONFIG = { "log_errors": True, } -# 重试配置 +# 重试/熔断配置 RETRY_CONFIG = { "max_retries": 3, "retry_delay": 1, + "cooldown_seconds": 60, # 熔断冷却期，到期自动恢复(半开) "retry_on_errors": [ "connection_error", "timeout", "rate_limit", "server_error", ], -} \ No newline at end of file +} diff --git a/data/config.json b/data/config.json index cebf9c0..44dfded 100644 --- a/data/config.json +++ b/data/config.json @@ -4,59 +4,191 @@ "id": "local-qwen", "name": "Local Qwen", "priority": 1, - "base_url": "http://192.168.2.5:1234/v1", - "api_key": "sk-lm-fuP5tGU8:Hi7YU87jHyDP6Ay8Tl2j", + "base_url": "http://121.40.164.32:18003/v1", + "api_key": "sk-xxxx", "models": [ - "qwen/qwen3.5-35b-a3b", - "qwen3.5-4b", - "qwen3.5", - "qwen" + { + "name": "unsloth/Qwen3.8-27B-Q6_K", + "capabilities": [ + "text", + "vision" + ] + }, + { + "name": "unsloth/Qwen3.8-27B-Q4_K_M", + "capabilities": [ + "text", + "vision" + ] + } ], - "default_model": "qwen/qwen3.5-35b-a3b", - "timeout": 120, + "default_model": "unsloth/Qwen3.8-27B-Q6_K", + "capabilities": [ + "text", + "vision" + ], + "timeout": 180, "enabled": true }, { - "id": "siliconflow-deepseek", - "name": "SiliconFlow DeepSeek", + "id": "siliconflow-llm", + "name": "SiliconFlow LLM", "priority": 2, "base_url": "https://api.siliconflow.cn/v1", "api_key": "sk-fhpoexpptvjghpnphtaxbkhjwulzovoqfffbckcfscjmwhcg", "models": [ - "Pro/deepseek-ai/DeepSeek-V3.2", - "deepseek-v3", - "deepseek" + { + "name": "deepseek-ai/DeepSeek-V4-Flash", + "capabilities": [ + "text" + ] + }, + { + "name": "meituan-longcat/LongCat-2.0", + "capabilities": [ + "text" + ] + } ], - "default_model": "Pro/deepseek-ai/DeepSeek-V3.2", - "timeout": 120, + "default_model": "deepseek-ai/DeepSeek-V4-Flash", + "capabilities": [ + "text" + ], + "timeout": 180, + "enabled": true + }, + { + "id": "autodl", + "name": "Autodl", + "priority": 3, + "base_url": "https://www.autodl.art/api/v1", + "api_key": "F9MBfolzuapqTsD4KmUf9qen720rXvUZ3Sp3IrWiCTukqonx", + "models": [ + { + "name": "qwen3.6-plus", + "capabilities": [ + "text", + "vision" + ] + }, + { + "name": "GLM-5.3-flash", + "capabilities": [ + "text", + "vision" + ] + } + ], + "default_model": "GLM-5.3-flash", + "capabilities": [ + "text", + "vision" + ], + "timeout": 180, + "enabled": true + }, + { + "id": "autodl-image", + "name": "Autodl Image", + "priority": 4, + "base_url": "https://www.autodl.art/api/v1", + "api_key": "F9MBfolzuapqTsD4KmUf9qen720rXvUZ3Sp3IrWiCTukqonx", + "models": [ + { + "name": "Qwen-Image", + "capabilities": [ + "image_gen" + ] + } + ], + "default_model": "Qwen-Image", + "capabilities": [ + "image_gen" + ], + "timeout": 180, "enabled": true } ], "model_aliases": { "auto": "auto", - "qwen": "qwen3.5-4b", - "qwen3.5": "qwen3.5-4b", - "qwen3.5-4b": "qwen3.5-4b", - "deepseek": "Pro/deepseek-ai/DeepSeek-V3.2", - "deepseek-v3": "Pro/deepseek-ai/DeepSeek-V3.2", - "deepseek-v3.2": "Pro/deepseek-ai/DeepSeek-V3.2" + "auto-text": "auto-text", + "auto-vision": "auto-vision", + "auto-image": "auto-image", + "auto-voice-out": "auto-voice-out", + "auto-voice-in": "auto-voice-in", + "auto-video": "auto-video", + "auto-vlm": "auto-vision", + "qwen": "unsloth/Qwen3.8-27B-Q6_K", + "qwen3.8": "unsloth/Qwen3.8-27B-Q6_K", + "local": "unsloth/Qwen3.8-27B-Q6_K", + "deepseek": "deepseek-ai/DeepSeek-V4-Flash", + "deepseek-v4-flash": "deepseek-ai/DeepSeek-V4-Flash", + "longcat": "meituan-longcat/LongCat-2.0", + "glm": "GLM-5.3-flash", + "qwen3.6-plus": "qwen3.6-plus", + "gpt-4": "GLM-5.3-flash" }, "auto_profiles": { - "auto-vlm": { - "name": "auto-vlm", - "description": "视觉多模态大模型自动版", + "auto": { + "name": "默认Auto", + "description": "文本推理 - 自动选择可用提供商", + "capability": "text", "providers": [ - "local-qwen", - "siliconflow-deepseek" + "*" ], "strategy": "priority" }, - "auto": { - "name": "auto", - "description": "auto", + "auto-text": { + "name": "文本推理", + "description": "纯文本推理，自动选择文本模型", + "capability": "text", "providers": [ - "local-qwen", - "siliconflow-deepseek" + "*" + ], + "strategy": "priority" + }, + "auto-vision": { + "name": "视觉能力", + "description": "多模态视觉理解，自动选择视觉模型", + "capability": "vision", + "providers": [ + "*" + ], + "strategy": "priority" + }, + "auto-image": { + "name": "图片生成", + "description": "文生图，自动选择生图模型", + "capability": "image_gen", + "providers": [ + "*" + ], + "strategy": "priority" + }, + "auto-voice-out": { + "name": "语音输出", + "description": "语音合成(TTS)，自动选择语音输出模型", + "capability": "audio_out", + "providers": [ + "*" + ], + "strategy": "priority" + }, + "auto-voice-in": { + "name": "语音输入", + "description": "语音识别(ASR)，自动选择语音输入模型", + "capability": "audio_in", + "providers": [ + "*" + ], + "strategy": "priority" + }, + "auto-video": { + "name": "视频生成", + "description": "文生视频，自动选择视频生成模型", + "capability": "video_gen", + "providers": [ + "*" ], "strategy": "priority" } diff --git a/run.sh b/run.sh index 3ac301c..1934109 100755 --- a/run.sh +++ b/run.sh @@ -1,3 +1,4 @@ #!/bin/bash -cd /home/xian/.openclaw/common/projects/llm-proxy -exec python3 app.py +# 前台启动（推荐用 ./start.sh 后台管理） +cd "$(dirname "$0")" +exec /home/hz1/miniconda3/envs/openclaw/bin/python3 app.py diff --git a/start.sh b/start.sh new file mode 100755 index 0000000..9529724 --- /dev/null +++ b/start.sh @@ -0,0 +1,59 @@ +#!/bin/bash +# 大模型API中转系统 启动/停止/状态脚本 +# 用法: ./start.sh [start|stop|restart|status] 默认 start +cd "$(dirname "$0")" + +PORT=16003 +APP=app.py +PYTHON=/home/hz1/miniconda3/envs/openclaw/bin/python3 +PID_FILE=data/proxy.pid +mkdir -p data logs + +if [ ! -x "$PYTHON" ]; then + PYTHON=python3 +fi + +start() { + if [ -f "$PID_FILE" ] && kill -0 "$(cat "$PID_FILE")" 2>/dev/null; then + echo "LLM Proxy 已在运行 (PID $(cat "$PID_FILE"), 端口 $PORT)" + return 0 + fi + nohup "$PYTHON" "$APP" > logs/start.log 2>&1 & + echo $! > "$PID_FILE" + sleep 1.5 + if kill -0 "$(cat "$PID_FILE")" 2>/dev/null; then + echo "✅ LLM Proxy 已启动 (PID $(cat "$PID_FILE"))" + echo " 前台API: http://:$PORT/v1/chat/completions" + echo " 后台管理: http://:$PORT/admin" + else + echo "❌ 启动失败，查看日志: logs/start.log" + tail -20 logs/start.log + return 1 + fi +} + +stop() { + if [ -f "$PID_FILE" ] && kill -0 "$(cat "$PID_FILE")" 2>/dev/null; then + kill "$(cat "$PID_FILE")" + rm -f "$PID_FILE" + echo "🛑 LLM Proxy 已停止" + else + echo "LLM Proxy 未在运行" + fi +} + +status() { + if [ -f "$PID_FILE" ] && kill -0 "$(cat "$PID_FILE")" 2>/dev/null; then + echo "✅ LLM Proxy 运行中 (PID $(cat "$PID_FILE"), 端口 $PORT)" + else + echo "❌ LLM Proxy 未运行" + fi +} + +case "${1:-start}" in + start) start ;; + stop) stop ;; + restart) stop; sleep 1; start ;; + status) status ;; + *) echo "用法: $0 [start|stop|restart|status]"; exit 1 ;; +esac diff --git a/templates/auto-profiles.html b/templates/auto-profiles.html index b1d1810..de8ae77 100644 --- a/templates/auto-profiles.html +++ b/templates/auto-profiles.html @@ -51,7 +51,7 @@

Auto配置管理

-

创建自定义的自动选择模式，指定候选提供商和优先级

+

每个Auto配置固定一个功能类型，并从模型管理中挑选具备该能力的模型

- +
- +
+ placeholder="例如: auto-text, auto-fast">

调用时使用 model="此名称"

+ placeholder="例如: 快速文本">
+
+ + +

文本推理 / 视觉能力 / 语音输出 / 语音输入 / 图片生成 / 视频生成

+
+