Compare commits

...
6 Commits
Author SHA1 Message Date
hz4th_coder eea7269eda feat: 步骤5填充字段改用智能体执行
- 新增步骤5任务模板(config/agent_fill_fields_template.txt)
- 步骤5调用智能体hz4th_editor执行:
  1. 获取API文档了解对应类别字段定义
  2. 从内容库获取相关内容数据
  3. 整理产品参数
  4. 通过API提交到ParamHub审核系统
- /process页面新增步骤5模板编辑面板
- 步骤6改为确认提交结果(备用本地提交)
- 新增API: GET/POST /api/process/fill-fields-template
2026-07-15 16:31:00 +08:00
hz4th_coder 5fdcf7a5b6 refactor: 步骤4智能体任务改为传递内容库ID
- 模板改为传递内容库数据ID列表而非内容摘要
- 智能体任务改为分析ID对应数据与产品的相关性和参数提取价值
- 智能体输出改为relevant_ids列表+分析说明
- 步骤3保存内容库时记录article_id
- 步骤4根据智能体返回的ID从内容库获取实际内容
- 更新_fill_fields适配新的extracted_data格式
2026-07-15 13:00:03 +08:00
hz4th_coder 96c479916b feat: 步骤4提取产品数据改用智能体hz4th_editor执行
- 步骤4调用 openclaw agent --agent hz4th_editor --message 执行提取任务
- 新增任务文本模板(config/agent_task_template.txt),支持变量填充
- /process页面新增模板编辑面板,可查看/编辑/保存/预览模板
- 模板变量: {{product_name}} {{category}} {{subcategory}} {{library_results}} {{internet_results}}
- 新增API: GET/POST /api/process/agent-template, POST /api/process/agent-template/preview
2026-07-15 11:18:47 +08:00
hz4th_coder c833394f06 fix: 统一处理流程和后台任务显示
- 处理流程抓取失败时记录到失败URL库
- /search页面新增产品处理会话展示区
- 同时显示后台抓取任务和产品处理会话
- 自动刷新同时更新三种数据:任务、会话、失败URL
2026-07-14 15:36:18 +08:00
hz4th_coder 5e3e9e7335 fix: 修复处理流程和后台任务问题
- 步骤简要数值显示在每个步骤圆圈下方
- 处理流程抓取网页后保存到内容库
- 修复后台任务结果解析错误
2026-07-14 15:13:31 +08:00
hz4th_coder 1712fb0756 feat: 处理监控页面显示步骤简要数值
- 显示内容库搜索结果数
- 显示互联网搜索结果数
- 显示抓取成功数
- 显示数据提取/填充状态
- 显示审核ID
2026-07-14 12:42:59 +08:00
10 changed files with 1143 additions and 36 deletions
+71
View File
@@ -0,0 +1,71 @@
## 任务背景
### 产品基本信息
- **产品名称:** {{product_name}}
- **产品类别:** {{category}}
- **子类别:** {{subcategory}}
### 相关内容数据ID
上一步已筛选出以下与产品直接相关且对参数提取有用的内容库数据ID:
{{relevant_content_ids}}
## 任务要求
请完成以下工作:
### 1. 获取对应类别的字段配置
首先,请访问 ParamHub API 文档获取对应类别的字段定义:
- API文档地址:http://192.168.2.8:12007/hz4th_coder/param-hub-python/src/branch/master/API.md
- 根据产品类别({{category}})确定应该使用哪个API
- AI模型 → `/api/models`
- GPU → `/api/gpus`
- CPU → `/api/cpus`
- 其他动态分类 → `/api/items/{category_id}`
### 2. 从内容库获取数据内容
根据上述数据ID,从内容库中获取每条数据的完整内容。
### 3. 整理产品参数
根据获取到的内容,提取并整理产品的各项参数,严格按照API文档中定义的字段格式填充。
### 4. 提交到审核系统
使用以下命令提交到ParamHub审核系统:
```bash
# 先登录获取cookie
curl -c /tmp/paramhub_cookie.txt -X POST "http://localhost:16041/login" \
-H "Content-Type: application/json" \
-d '{"password": "admin123"}'
# 然后提交产品数据(根据类别选择对应的API)
curl -b /tmp/paramhub_cookie.txt -X POST "http://localhost:16041/api/{对应类别API}" \
-H "Content-Type: application/json" \
-d '{
"name": "{{product_name}}",
"visible": true,
"is_pinned": false,
// ... 其他字段根据API文档填充
}'
```
### 5. 输出要求
请以JSON格式输出执行结果:
```json
{
"success": true,
"review_id": "审核ID(如果提交成功)",
"submitted_data": {
"name": "产品名称",
"field1": "值1",
"field2": "值2"
},
"data_sources": [数据ID列表],
"message": "执行说明"
}
```
**注意:**
- 严格按照API文档的字段定义填充数据
- 不要编造或推测任何参数,只使用内容库中实际存在的信息
- 如果某些字段无法从内容中提取,可以留空或填写默认值
- 提交成功后记录返回的review_id
+53
View File
@@ -0,0 +1,53 @@
## 任务背景
### 内容库相关数据ID列表
以下是从内容库中搜索到的相关文章数据ID:
{{library_results}}
### 互联网搜索已入库数据ID列表
以下是从互联网搜索并已成功抓取入库的数据ID:
{{internet_results}}
## 产品信息
- **产品名称:** {{product_name}}
- **产品类别:** {{category}}
- **子类别:** {{subcategory}}
## 任务要求
请分析上述内容库ID对应的数据,判断哪些与产品「{{product_name}}」**直接相关**且**对提取产品参数有用**。
### 判断标准:
1. **直接相关性**:内容必须明确提及该产品名称或其主要型号,排除仅提及相似产品或竞品的内容
2. **参数提取价值**:内容应包含可用于填充产品字段的信息,如:
- 产品规格参数(尺寸、重量、容量等)
- 技术规格(性能指标、接口、兼容性等)
- 功能特性
- 发布信息(发布日期、价格等)
- 其他结构化产品数据
### 输出要求:
请以JSON格式输出分析结果,包含以下字段:
```json
{
"relevant_ids": [1, 2, 3],
"analysis": {
"1": "简要说明为什么这条数据相关且有用",
"2": "...",
"3": "..."
},
"excluded_ids": [4, 5],
"exclusion_reasons": {
"4": "简要说明排除原因",
"5": "..."
},
"confidence": "high/medium/low"
}
```
**注意:**
- `relevant_ids`:与产品直接相关且对参数提取有用的数据ID列表
- `analysis`:每个相关ID的简要分析说明
- `excluded_ids`:被排除的ID列表(可选)
- `exclusion_reasons`:排除原因说明(可选)
- `confidence`:整体判断的置信度
+161 -1
View File
@@ -4,10 +4,15 @@
from flask import Blueprint, jsonify, request
from services.process_monitor import process_monitor, PROCESS_STEPS
from models.database import db
import os
import logging
logger = logging.getLogger('process_monitor_api')
# 模板文件路径
TEMPLATE_DIR = os.path.join(os.path.dirname(os.path.dirname(__file__)), 'config')
AGENT_TEMPLATE_FILE = os.path.join(TEMPLATE_DIR, 'agent_task_template.txt')
bp = Blueprint('process_monitor', __name__, url_prefix='/api/process')
@@ -177,4 +182,159 @@ def get_step_detail(session_id, step_num):
return jsonify({
'success': True,
'step': step
})
})
@bp.route('/agent-template', methods=['GET'])
def get_agent_template():
"""获取智能体任务文本模板"""
try:
if os.path.exists(AGENT_TEMPLATE_FILE):
with open(AGENT_TEMPLATE_FILE, 'r', encoding='utf-8') as f:
template = f.read()
return jsonify({
'success': True,
'template': template
})
else:
return jsonify({
'success': False,
'error': '模板文件不存在'
}), 404
except Exception as e:
logger.error(f"获取模板失败: {e}")
return jsonify({'success': False, 'error': str(e)}), 500
@bp.route('/agent-template', methods=['POST'])
def save_agent_template():
"""保存智能体任务文本模板"""
try:
data = request.get_json()
template = data.get('template', '')
if not template:
return jsonify({'success': False, 'error': '模板内容不能为空'}), 400
os.makedirs(TEMPLATE_DIR, exist_ok=True)
with open(AGENT_TEMPLATE_FILE, 'w', encoding='utf-8') as f:
f.write(template)
return jsonify({
'success': True,
'message': '模板已保存'
})
except Exception as e:
logger.error(f"保存模板失败: {e}")
return jsonify({'success': False, 'error': str(e)}), 500
@bp.route('/agent-template/preview', methods=['POST'])
def preview_agent_template():
"""预览填充后的任务文本"""
try:
data = request.get_json()
product_name = data.get('product_name', '示例产品')
category = data.get('category', '示例类别')
subcategory = data.get('subcategory', '示例子类别')
# 读取模板
if os.path.exists(AGENT_TEMPLATE_FILE):
with open(AGENT_TEMPLATE_FILE, 'r', encoding='utf-8') as f:
template = f.read()
else:
return jsonify({'success': False, 'error': '模板文件不存在'}), 404
# 填充示例数据
filled = template.replace('{{product_name}}', product_name)
filled = filled.replace('{{category}}', category)
filled = filled.replace('{{subcategory}}', subcategory or '')
filled = filled.replace('{{library_results}}', 'ID 101: 示例文章标题A\nID 102: 示例文章标题B\nID 103: 示例文章标题C')
filled = filled.replace('{{internet_results}}', 'ID 104: 示例互联网抓取标题X\nID 105: 示例互联网抓取标题Y')
return jsonify({
'success': True,
'preview': filled
})
except Exception as e:
logger.error(f"预览模板失败: {e}")
return jsonify({'success': False, 'error': str(e)}), 500
# ===== 步骤5填充字段模板 API =====
FILL_FIELDS_TEMPLATE_FILE = os.path.join(TEMPLATE_DIR, 'agent_fill_fields_template.txt')
@bp.route('/fill-fields-template', methods=['GET'])
def get_fill_fields_template():
"""获取步骤5填充字段任务文本模板"""
try:
if os.path.exists(FILL_FIELDS_TEMPLATE_FILE):
with open(FILL_FIELDS_TEMPLATE_FILE, 'r', encoding='utf-8') as f:
template = f.read()
return jsonify({
'success': True,
'template': template
})
else:
return jsonify({
'success': False,
'error': '模板文件不存在'
}), 404
except Exception as e:
logger.error(f"获取填充字段模板失败: {e}")
return jsonify({'success': False, 'error': str(e)}), 500
@bp.route('/fill-fields-template', methods=['POST'])
def save_fill_fields_template():
"""保存步骤5填充字段任务文本模板"""
try:
data = request.get_json()
template = data.get('template', '')
if not template:
return jsonify({'success': False, 'error': '模板内容不能为空'}), 400
os.makedirs(TEMPLATE_DIR, exist_ok=True)
with open(FILL_FIELDS_TEMPLATE_FILE, 'w', encoding='utf-8') as f:
f.write(template)
return jsonify({
'success': True,
'message': '模板已保存'
})
except Exception as e:
logger.error(f"保存填充字段模板失败: {e}")
return jsonify({'success': False, 'error': str(e)}), 500
@bp.route('/fill-fields-template/preview', methods=['POST'])
def preview_fill_fields_template():
"""预览填充后的步骤5任务文本"""
try:
data = request.get_json()
product_name = data.get('product_name', '示例产品')
category = data.get('category', '示例类别')
subcategory = data.get('subcategory', '示例子类别')
# 读取模板
if os.path.exists(FILL_FIELDS_TEMPLATE_FILE):
with open(FILL_FIELDS_TEMPLATE_FILE, 'r', encoding='utf-8') as f:
template = f.read()
else:
return jsonify({'success': False, 'error': '模板文件不存在'}), 404
# 填充示例数据
filled = template.replace('{{product_name}}', product_name)
filled = filled.replace('{{category}}', category)
filled = filled.replace('{{subcategory}}', subcategory or '')
filled = filled.replace('{{relevant_content_ids}}', 'ID 101: 示例相关文章A\nID 102: 示例相关文章B\nID 103: 示例相关文章C')
return jsonify({
'success': True,
'preview': filled
})
except Exception as e:
logger.error(f"预览填充字段模板失败: {e}")
return jsonify({'success': False, 'error': str(e)}), 500
+406 -30
View File
@@ -1,9 +1,11 @@
"""
处理步骤监控服务 - 记录和监控产品处理流程
"""
import os
import time
import uuid
import json
import subprocess
import threading
import logging
from datetime import datetime
@@ -18,8 +20,8 @@ PROCESS_STEPS = [
{'num': 1, 'name': '搜索内容库', 'description': '从内容库搜索相关文章'},
{'num': 2, 'name': '搜索互联网', 'description': '从互联网搜索最新数据'},
{'num': 3, 'name': '抓取网页内容', 'description': '抓取搜索结果网页的详细内容'},
{'num': 4, 'name': '提取产品数据', 'description': '从抓取内容中提取产品相关数据'},
{'num': 5, 'name': '填充字段', 'description': '根据分类字段配置填充数据'},
{'num': 4, 'name': '提取产品数据(智能体)', 'description': '调用hz4th_editor智能体提取产品相关内容'},
{'num': 5, 'name': '填充字段(智能体)', 'description': '调用hz4th_editor智能体整理产品数据并提交审核'},
{'num': 6, 'name': '提交审核', 'description': '提交到ParamHub待审核区'},
]
@@ -102,6 +104,7 @@ class ProcessMonitor:
self._start_step(session_id, product_name, 3, '抓取网页内容')
try:
fetched = []
failed_count = 0
urls_to_fetch = [r['url'] for r in all_data['internet_results'][:5]]
for i, url in enumerate(urls_to_fetch):
@@ -110,51 +113,190 @@ class ProcessMonitor:
fetch_result = search_service.fetch_url_content(url)
if fetch_result.get('success'):
title = fetch_result.get('title', '')
content = fetch_result.get('content', '')
article_id = None
# 保存到内容库
try:
existing = db.search_articles(url)
if existing and len(existing) > 0:
# 已存在,使用现有ID
article_id = existing[0].get('id')
logger.info(f"[{session_id}] 内容库已存在: {title[:30]}, ID={article_id}")
else:
# 新增,获取返回的ID
article_id = db.add_article(
product_names=[],
category=category or '',
keywords=[],
summary=content[:200] if content else '',
content=content,
source=url,
url=url,
search_title=title
)
logger.info(f"[{session_id}] 已保存到内容库: {title[:30]}, ID={article_id}")
except Exception as save_error:
logger.warning(f"[{session_id}] 保存内容库失败: {save_error}")
fetched.append({
'id': article_id,
'url': url,
'title': fetch_result.get('title', ''),
'content': fetch_result.get('content', '')[:500]
'title': title,
'content': content[:500]
})
else:
# 记录失败URL
failed_count += 1
error_msg = fetch_result.get('error', '抓取失败')
try:
db.add_failed_url(url, product_name, error_msg, source='process_monitor')
logger.warning(f"[{session_id}] 抓取失败,已记录: {url}")
except Exception as e:
logger.error(f"[{session_id}] 记录失败URL出错: {e}")
time.sleep(0.3)
all_data['fetched_contents'] = fetched
self._complete_step(session_id, 3, {'count': len(fetched)})
logger.info(f"[{session_id}] 步骤3完成: 抓取 {len(fetched)} 个网页")
self._complete_step(session_id, 3, {'count': len(fetched), 'failed': failed_count})
logger.info(f"[{session_id}] 步骤3完成: 抓取 {len(fetched)} 个网页, 失败 {failed_count}")
except Exception as e:
self._fail_step(session_id, 3, str(e))
# 步骤4: 提取产品数据
# 步骤4: 提取产品数据(调用智能体执行)
if not self._check_pause(session_id):
self._start_step(session_id, product_name, 4, '提取产品数据')
self._start_step(session_id, product_name, 4, '提取产品数据(智能体)')
try:
extracted = self._extract_data(product_name, all_data)
all_data['extracted_data'] = extracted
# 构建任务文本
task_text = self._build_agent_task(
product_name, category, subcategory, all_data
)
if extracted:
self._complete_step(session_id, 4, {'has_data': True})
# 调用智能体
agent_result = self._call_agent(task_text)
if agent_result.get('success'):
parsed = self._parse_agent_response(agent_result.get('output', ''))
if parsed and parsed.get('relevant_ids'):
# 根据ID从内容库获取实际内容
relevant_contents = []
for aid in parsed['relevant_ids']:
article = db.get_article_by_id(aid)
if article:
relevant_contents.append({
'id': aid,
'title': article.get('search_title', ''),
'url': article.get('url', ''),
'content': article.get('content', ''),
'summary': article.get('summary', ''),
'analysis': parsed.get('analysis', {}).get(str(aid), '')
})
all_data['extracted_data'] = {
'name': product_name,
'relevant_ids': parsed['relevant_ids'],
'relevant_contents': relevant_contents,
'confidence': parsed.get('confidence', 'unknown'),
'raw_output': agent_result.get('output', '')
}
self._complete_step(session_id, 4, {
'has_data': True,
'agent': 'hz4th_editor',
'task_text': task_text,
'relevant_ids': parsed['relevant_ids'],
'relevant_count': len(relevant_contents),
'confidence': parsed.get('confidence', 'unknown'),
'agent_output': agent_result.get('output', '')[:2000]
})
logger.info(f"[{session_id}] 步骤4完成: 智能体返回 {len(parsed['relevant_ids'])} 个相关ID")
else:
all_data['extracted_data'] = None
self._complete_step(session_id, 4, {
'has_data': False,
'agent': 'hz4th_editor',
'task_text': task_text,
'agent_output': agent_result.get('output', '')[:2000]
}, status='skipped')
result['message'] = '智能体未找到相关数据ID'
else:
self._complete_step(session_id, 4, {'has_data': False}, status='skipped')
result['message'] = '无法提取有效数据'
self._fail_step(session_id, 4, f"智能体调用失败: {agent_result.get('error', '未知错误')}")
result['message'] = f'智能体调用失败: {agent_result.get("error")}'
except Exception as e:
self._fail_step(session_id, 4, str(e))
# 步骤5: 填充字段
# 步骤5: 填充字段并提交审核(调用智能体执行)
if not self._check_pause(session_id) and all_data['extracted_data']:
self._start_step(session_id, product_name, 5, '填充字段')
self._start_step(session_id, product_name, 5, '填充字段并提交(智能体)')
try:
filled = self._fill_fields(all_data['extracted_data'], category, subcategory)
all_data['filled_data'] = filled
# 构建任务文本
fill_task_text = self._build_fill_fields_task(
product_name, category, subcategory, all_data['extracted_data']
)
if filled:
self._complete_step(session_id, 5, {'filled': True})
# 调用智能体
fill_agent_result = self._call_agent(fill_task_text)
if fill_agent_result.get('success'):
fill_parsed = self._parse_fill_agent_response(fill_agent_result.get('output', ''))
if fill_parsed and fill_parsed.get('success'):
review_id = fill_parsed.get('review_id')
submitted_data = fill_parsed.get('submitted_data', {})
all_data['filled_data'] = submitted_data
self._complete_step(session_id, 5, {
'filled': True,
'agent': 'hz4th_editor',
'task_text': fill_task_text,
'review_id': review_id,
'submitted_data': submitted_data,
'agent_output': fill_agent_result.get('output', '')[:2000]
})
# 如果智能体返回了review_id,直接标记步骤6完成
if review_id:
result['success'] = True
result['review_id'] = review_id
db.update_session_status(session_id, 'completed',
review_id=review_id,
result=json.dumps(result, ensure_ascii=False))
db.add_process_history(
product_name=product_name,
category=category,
subcategory=subcategory,
status='submitted',
review_id=review_id,
details=all_data
)
logger.info(f"[{session_id}] 步骤5完成: 智能体提交成功, review_id={review_id}")
else:
self._complete_step(session_id, 5, {
'filled': True,
'agent': 'hz4th_editor',
'task_text': fill_task_text,
'submitted_data': submitted_data,
'message': '数据已整理但未获取到review_id',
'agent_output': fill_agent_result.get('output', '')[:2000]
})
result['message'] = '数据已整理但提交状态未知'
else:
self._fail_step(session_id, 5, f"智能体执行失败: {fill_parsed.get('message', '未知错误') if fill_parsed else '解析失败'}")
result['message'] = f'智能体执行失败'
else:
self._fail_step(session_id, 5, '填充数据失败')
self._fail_step(session_id, 5, f"智能体调用失败: {fill_agent_result.get('error', '未知错误')}")
result['message'] = f'智能体调用失败: {fill_agent_result.get("error")}'
except Exception as e:
self._fail_step(session_id, 5, str(e))
# 步骤6: 提交审核
if not self._check_pause(session_id) and all_data['filled_data']:
self._start_step(session_id, product_name, 6, '提交审核')
# 步骤6: 确认提交结果(如果步骤5未获取到review_id,尝试本地提交)
if not self._check_pause(session_id) and all_data['filled_data'] and not result.get('review_id'):
self._start_step(session_id, product_name, 6, '确认提交结果')
try:
category_type = self._get_category_type(category)
success, review_id_or_error = paramhub_client.submit_for_review(
@@ -180,7 +322,7 @@ class ProcessMonitor:
review_id=review_id_or_error,
details=all_data
)
logger.info(f"[{session_id}] 步骤6完成: 提交成功")
logger.info(f"[{session_id}] 步骤6完成: 本地提交成功")
else:
self._fail_step(session_id, 6, review_id_or_error)
db.update_session_status(session_id, 'failed')
@@ -271,8 +413,231 @@ class ProcessMonitor:
return {'session': session, 'steps': steps}
return None
def _build_agent_task(self, product_name, category, subcategory, all_data):
"""构建智能体任务文本"""
# 读取模板
template_file = os.path.join(
os.path.dirname(os.path.dirname(__file__)),
'config', 'agent_task_template.txt'
)
if os.path.exists(template_file):
with open(template_file, 'r', encoding='utf-8') as f:
template = f.read()
else:
# 默认模板
template = (
"请分析以下数据ID是否与产品「{{product_name}}」相关且对提取参数有用。\n"
"类别: {{category}} / {{subcategory}}\n\n"
"内容库结果ID: {{library_results}}\n\n"
"互联网已入库ID: {{internet_results}}\n\n"
"要求:输出相关且有用的ID列表,以JSON格式输出。"
)
# 构建内容库搜索结果ID列表
library_ids = []
for article in all_data.get('library_results', []):
aid = article.get('id')
if aid:
title = article.get('search_title', article.get('title', ''))
library_ids.append(f"ID {aid}: {title}")
library_text = '\n'.join(library_ids) if library_ids else '(无内容库搜索结果)'
# 构建互联网已入库数据ID列表
internet_ids = []
for item in all_data.get('fetched_contents', []):
aid = item.get('id')
if aid:
title = item.get('title', '')
internet_ids.append(f"ID {aid}: {title}")
internet_text = '\n'.join(internet_ids) if internet_ids else '(无互联网已入库数据)'
# 填充模板
task = template.replace('{{product_name}}', product_name or '未知')
task = task.replace('{{category}}', category or '未分类')
task = task.replace('{{subcategory}}', subcategory or '')
task = task.replace('{{library_results}}', library_text)
task = task.replace('{{internet_results}}', internet_text)
return task
def _call_agent(self, task_text):
"""调用智能体执行任务"""
try:
cmd = [
'openclaw', 'agent',
'--agent', 'hz4th_editor',
'--message', task_text
]
logger.info(f"调用智能体命令: openclaw agent --agent hz4th_editor --message '[任务文本 {len(task_text)} 字符]'")
result = subprocess.run(
cmd,
capture_output=True,
text=True,
timeout=300 # 5分钟超时
)
if result.returncode == 0:
output = result.stdout.strip()
logger.info(f"智能体返回: {output[:500]}...")
return {'success': True, 'output': output}
else:
error = result.stderr.strip() or result.stdout.strip()
logger.error(f"智能体调用失败: {error}")
return {'success': False, 'error': error}
except subprocess.TimeoutExpired:
return {'success': False, 'error': '智能体执行超时(>5分钟)'}
except FileNotFoundError:
return {'success': False, 'error': 'openclaw命令未找到'}
except Exception as e:
return {'success': False, 'error': str(e)}
def _parse_agent_response(self, output):
"""解析智能体返回的结果,提取relevant_ids"""
if not output:
return None
# 尝试从输出中提取JSON
import re
parsed_data = None
# 查找JSON块
json_match = re.search(r'```(?:json)?\s*(\{.*?\})\s*```', output, re.DOTALL)
if json_match:
try:
parsed_data = json.loads(json_match.group(1))
except json.JSONDecodeError:
pass
# 尝试直接解析整个输出为JSON
if not parsed_data:
try:
parsed_data = json.loads(output)
except json.JSONDecodeError:
pass
if parsed_data:
relevant_ids = parsed_data.get('relevant_ids', [])
# 确保都是整数
relevant_ids = [int(x) for x in relevant_ids if str(x).isdigit()]
return {
'relevant_ids': relevant_ids,
'analysis': parsed_data.get('analysis', {}),
'excluded_ids': parsed_data.get('excluded_ids', []),
'exclusion_reasons': parsed_data.get('exclusion_reasons', {}),
'confidence': parsed_data.get('confidence', 'unknown'),
'raw_output': output
}
# 无法解析为JSON,尝试从文本中提取ID
id_matches = re.findall(r'(?:ID|id)[\s:]*(\d+)', output)
if id_matches:
return {
'relevant_ids': [int(x) for x in id_matches],
'analysis': {},
'confidence': 'low',
'raw_output': output
}
return None
def _build_fill_fields_task(self, product_name, category, subcategory, extracted_data):
"""构建步骤5填充字段的智能体任务文本"""
# 读取模板
template_file = os.path.join(
os.path.dirname(os.path.dirname(__file__)),
'config', 'agent_fill_fields_template.txt'
)
if os.path.exists(template_file):
with open(template_file, 'r', encoding='utf-8') as f:
template = f.read()
else:
# 默认模板
template = (
"请根据内容库数据ID {{relevant_content_ids}} 整理产品「{{product_name}}」的参数并提交审核。\n"
"类别: {{category}} / {{subcategory}}\n"
"参考API文档: http://192.168.2.8:12007/hz4th_coder/param-hub-python/src/branch/master/API.md"
)
# 构建相关内容ID列表
relevant_ids = extracted_data.get('relevant_ids', [])
relevant_contents = extracted_data.get('relevant_contents', [])
if relevant_contents:
content_lines = []
for item in relevant_contents:
aid = item.get('id', '')
title = item.get('title', '')
content_lines.append(f"ID {aid}: {title}")
relevant_text = '\n'.join(content_lines)
elif relevant_ids:
relevant_text = '\n'.join([f"ID {aid}" for aid in relevant_ids])
else:
relevant_text = '(无相关内容ID'
# 填充模板
task = template.replace('{{product_name}}', product_name or '未知')
task = task.replace('{{category}}', category or '未分类')
task = task.replace('{{subcategory}}', subcategory or '')
task = task.replace('{{relevant_content_ids}}', relevant_text)
return task
def _parse_fill_agent_response(self, output):
"""解析步骤5智能体返回的结果"""
if not output:
return None
import re
parsed_data = None
# 查找JSON块
json_match = re.search(r'```(?:json)?\s*(\{.*?\})\s*```', output, re.DOTALL)
if json_match:
try:
parsed_data = json.loads(json_match.group(1))
except json.JSONDecodeError:
pass
# 尝试直接解析整个输出为JSON
if not parsed_data:
try:
parsed_data = json.loads(output)
except json.JSONDecodeError:
pass
if parsed_data:
return {
'success': parsed_data.get('success', False),
'review_id': parsed_data.get('review_id'),
'submitted_data': parsed_data.get('submitted_data', {}),
'data_sources': parsed_data.get('data_sources', []),
'message': parsed_data.get('message', ''),
'raw_output': output
}
# 尝试从文本中提取review_id
review_match = re.search(r'review[_-]?id[\s:]*([\w-]+)', output, re.I)
if review_match:
return {
'success': True,
'review_id': review_match.group(1),
'submitted_data': {},
'message': '从输出中提取到review_id',
'raw_output': output
}
return None
def _extract_data(self, product_name, all_data):
"""提取产品数据"""
"""提取产品数据(备用,已被智能体替代)"""
all_content = []
for article in all_data.get('library_results', []):
@@ -301,23 +666,34 @@ class ProcessMonitor:
import re
filled = {
'name': extracted_data['name'],
'name': extracted_data.get('name', ''),
'visible': True,
'is_pinned': False
}
content = extracted_data.get('raw_content', '')
# 从relevant_contents中拼接所有内容
relevant_contents = extracted_data.get('relevant_contents', [])
all_content = '\n---\n'.join([
c.get('content', '') or c.get('summary', '')
for c in relevant_contents
if c.get('content') or c.get('summary')
])
params_match = re.search(r'(\d+(?:\.\d+)?)\s*[Bb]', content)
# 兼容旧格式
if not all_content:
all_content = extracted_data.get('raw_content', '')
params_match = re.search(r'(\d+(?:\.\d+)?)\s*[Bb]', all_content)
if params_match:
filled['parameters'] = f"{params_match.group(1)}B"
date_match = re.search(r'(\d{4}[-/]\d{1,2}[-/]\d{1,2})', content)
date_match = re.search(r'(\d{4}[-/]\d{1,2}[-/]\d{1,2})', all_content)
if date_match:
filled['publish_date'] = date_match.group(1).replace('/', '-')
filled['_source'] = 'auto_manager'
filled['_extracted_at'] = datetime.now().isoformat()
filled['_relevant_ids'] = extracted_data.get('relevant_ids', [])
return filled
+77 -1
View File
@@ -175,6 +175,31 @@
max-width: 80px;
}
.step-value {
font-size: 12px;
font-weight: 600;
color: #10b981;
margin-top: 3px;
padding: 2px 6px;
background: #d1fae5;
border-radius: 3px;
}
.step-node.running .step-value {
color: #667eea;
background: #e0e7ff;
}
.step-node.failed .step-value {
color: #ef4444;
background: #fee2e2;
}
.step-node.skipped .step-value {
color: #6b7280;
background: #f3f4f6;
}
@keyframes pulse {
0%, 100% { transform: scale(1); }
50% { transform: scale(1.1); }
@@ -325,4 +350,55 @@
.steps-progress {
flex-wrap: wrap;
}
}
}
/* 智能体任务模板区域 */
.template-section .panel-header {
display: flex;
justify-content: space-between;
align-items: center;
}
.template-actions {
display: flex;
gap: 8px;
}
.template-info {
background: #f0f4ff;
border: 1px solid #c7d2fe;
border-radius: 8px;
padding: 12px 16px;
margin-bottom: 15px;
font-size: 13px;
color: #4338ca;
}
.template-info p {
margin: 4px 0;
}
.template-info code {
background: #e0e7ff;
padding: 1px 5px;
border-radius: 3px;
font-size: 12px;
color: #3730a3;
}
.template-editor {
width: 100%;
font-family: 'Courier New', monospace;
font-size: 13px;
line-height: 1.5;
padding: 12px;
border: 1px solid #d1d5db;
border-radius: 8px;
resize: vertical;
background: #fafafa;
}
.template-editor:focus {
outline: none;
border-color: #667eea;
box-shadow: 0 0 0 3px rgba(102, 126, 234, 0.1);
}
+16
View File
@@ -472,6 +472,22 @@
color: #3730a3;
}
/* 产品处理会话区域 */
.process-sessions-section {
background: white;
border-radius: 12px;
box-shadow: 0 2px 4px rgba(0,0,0,0.05);
margin-top: 20px;
}
.process-sessions-section .panel-header {
background: #fef3c7;
}
.process-sessions-section .panel-header h2 {
color: #92400e;
}
.background-tasks-list {
display: flex;
flex-direction: column;
+189 -3
View File
@@ -9,6 +9,8 @@ document.addEventListener('DOMContentLoaded', () => {
loadStepDefinitions();
loadActiveProcesses();
loadHistory();
loadAgentTemplate();
loadFillFieldsTemplate();
// 启动自动刷新(每2秒)
startAutoRefresh();
@@ -133,17 +135,42 @@ function displayActiveProcesses(sessions) {
function renderStepsProgress(steps, currentStep) {
const totalSteps = 6;
const stepStatuses = {};
const stepDataMap = {};
// 构建步骤状态映射
// 构建步骤状态和数据映射
steps.forEach(s => {
stepStatuses[s.step_number] = s.step_status;
if (s.step_data) {
try {
stepDataMap[s.step_number] = typeof s.step_data === 'string' ? JSON.parse(s.step_data) : s.step_data;
} catch (e) {
stepDataMap[s.step_number] = s.step_data;
}
}
});
const stepNames = ['搜索内容库', '搜索互联网', '抓取网页', '提取数据', '填充字段', '提交审核'];
let html = '';
// 获取每个步骤的简要数值
function getStepValue(stepNum) {
const data = stepDataMap[stepNum];
if (!data) return '';
switch (stepNum) {
case 1: return data.count !== undefined ? `${data.count}` : '';
case 2: return data.count !== undefined ? `${data.count}` : '';
case 3: return data.count !== undefined ? `${data.count}` : '';
case 4: return data.has_data !== undefined ? (data.has_data ? '✓' : '✗') : (data.extracted ? '✓' : '');
case 5: return data.filled !== undefined ? (data.filled ? '✓' : '✗') : '';
case 6: return data.review_id ? '✓' : '';
default: return '';
}
}
let html = '<div class="steps-progress">';
for (let i = 1; i <= totalSteps; i++) {
const status = stepStatuses[i] || (i > currentStep ? 'pending' : '');
const stepValue = getStepValue(i);
let className = '';
if (status === 'completed') className = 'completed';
@@ -155,9 +182,11 @@ function renderStepsProgress(steps, currentStep) {
<div class="step-node ${className}">
<div class="step-circle">${i}</div>
<div class="step-label">${stepNames[i-1]}</div>
${stepValue ? `<div class="step-value">${stepValue}</div>` : ''}
</div>
`;
}
html += '</div>';
return html;
}
@@ -401,4 +430,161 @@ function showToast(message, type = '') {
setTimeout(() => {
toast.classList.remove('active');
}, 3000);
}
}
// ===== 智能体任务模板 =====
// 加载模板
async function loadAgentTemplate() {
try {
const response = await fetch(`${API_BASE}/api/process/agent-template`);
const data = await response.json();
if (data.success) {
document.getElementById('agent-template-editor').value = data.template;
} else {
document.getElementById('agent-template-editor').value = '// 模板加载失败: ' + (data.error || '未知错误');
}
} catch (error) {
console.error('加载模板失败:', error);
document.getElementById('agent-template-editor').value = '// 加载模板失败: ' + error.message;
}
}
// 保存模板
async function saveTemplate() {
const template = document.getElementById('agent-template-editor').value;
if (!template.trim()) {
showToast('模板内容不能为空', 'error');
return;
}
try {
const response = await fetch(`${API_BASE}/api/process/agent-template`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ template })
});
const data = await response.json();
if (data.success) {
showToast('模板已保存 ✓', 'success');
} else {
showToast('保存失败: ' + data.error, 'error');
}
} catch (error) {
showToast('保存失败: ' + error.message, 'error');
}
}
// 预览模板
function previewTemplate() {
document.getElementById('template-preview-modal').classList.add('active');
doPreview();
}
// 执行预览
async function doPreview() {
const product = document.getElementById('preview-product').value || '示例产品';
const category = document.getElementById('preview-category').value || 'AI模型';
try {
const response = await fetch(`${API_BASE}/api/process/agent-template/preview`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
product_name: product,
category: category,
subcategory: ''
})
});
const data = await response.json();
if (data.success) {
document.getElementById('template-preview-content').textContent = data.preview;
} else {
document.getElementById('template-preview-content').textContent = '预览失败: ' + data.error;
}
} catch (error) {
document.getElementById('template-preview-content').textContent = '预览失败: ' + error.message;
}
}
// ===== 步骤5填充字段模板 =====
// 加载步骤5模板
async function loadFillFieldsTemplate() {
try {
const response = await fetch(`${API_BASE}/api/process/fill-fields-template`);
const data = await response.json();
if (data.success) {
document.getElementById('fill-fields-template-editor').value = data.template;
} else {
document.getElementById('fill-fields-template-editor').value = '// 模板加载失败: ' + (data.error || '未知错误');
}
} catch (error) {
console.error('加载填充字段模板失败:', error);
document.getElementById('fill-fields-template-editor').value = '// 加载模板失败: ' + error.message;
}
}
// 保存步骤5模板
async function saveFillFieldsTemplate() {
const template = document.getElementById('fill-fields-template-editor').value;
if (!template.trim()) {
showToast('模板内容不能为空', 'error');
return;
}
try {
const response = await fetch(`${API_BASE}/api/process/fill-fields-template`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ template })
});
const data = await response.json();
if (data.success) {
showToast('步骤5模板已保存 ✓', 'success');
} else {
showToast('保存失败: ' + data.error, 'error');
}
} catch (error) {
showToast('保存失败: ' + error.message, 'error');
}
}
// 预览步骤5模板
function previewFillFieldsTemplate() {
document.getElementById('template-preview-modal').classList.add('active');
doFillFieldsPreview();
}
// 执行步骤5预览
async function doFillFieldsPreview() {
const product = document.getElementById('preview-product').value || '示例产品';
const category = document.getElementById('preview-category').value || 'AI模型';
try {
const response = await fetch(`${API_BASE}/api/process/fill-fields-template/preview`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
product_name: product,
category: category,
subcategory: ''
})
});
const data = await response.json();
if (data.success) {
document.getElementById('template-preview-content').textContent = data.preview;
} else {
document.getElementById('template-preview-content').textContent = '预览失败: ' + data.error;
}
} catch (error) {
document.getElementById('template-preview-content').textContent = '预览失败: ' + error.message;
}
}
+70 -1
View File
@@ -22,6 +22,7 @@ document.addEventListener('DOMContentLoaded', () => {
// 加载失败URL和后台任务
loadFailedUrls();
loadBackgroundTasks();
loadProcessSessions();
// 启动自动刷新后台任务列表(每3秒)
startAutoRefreshTasks();
@@ -773,7 +774,12 @@ async function loadBackgroundTasks() {
const progressPercent = task.total > 0 ?
Math.round((task.progress / task.total) * 100) : 0;
const result = task.result ? JSON.parse(task.result) : {};
let result = {};
try {
result = task.result ? JSON.parse(task.result) : {};
} catch (e) {
result = {};
}
return `
<div class="background-task-item ${statusClass}">
@@ -875,6 +881,8 @@ function startAutoRefreshTasks() {
autoRefreshTasksInterval = setInterval(() => {
loadBackgroundTasks();
loadProcessSessions(); // 同时刷新处理会话
loadFailedUrls(); // 同时刷新失败URL
}, 3000); // 每3秒刷新一次
}
@@ -884,4 +892,65 @@ function stopAutoRefreshTasks() {
clearInterval(autoRefreshTasksInterval);
autoRefreshTasksInterval = null;
}
}
// 加载产品处理会话
async function loadProcessSessions() {
try {
const response = await fetch(`${API_BASE}/api/process/recent?limit=10`);
const data = await response.json();
if (data.success) {
const container = document.getElementById('process-sessions-list');
if (data.sessions.length === 0) {
container.innerHTML = '<div class="empty-text">暂无处理会话</div>';
} else {
container.innerHTML = data.sessions.map(session => {
const statusClass = {
'pending': 'status-pending',
'running': 'status-running',
'paused': 'status-warning',
'completed': 'status-success',
'failed': 'status-error',
'stopped': 'status-warning'
}[session.status] || '';
const statusText = {
'pending': '等待中',
'running': '处理中',
'paused': '已暂停',
'completed': '已完成',
'failed': '失败',
'stopped': '已停止'
}[session.status] || session.status;
return `
<div class="background-task-item ${statusClass}">
<div class="task-info">
<div class="task-id">${escapeHtml(session.product_name)}</div>
<div class="task-status">
<span class="status-badge ${statusClass}">${statusText}</span>
${session.status === 'running' || session.status === 'paused' ?
`<span class="task-progress">步骤 ${session.current_step || 0}/6</span>` : ''}
${session.category ? `<span>分类: ${escapeHtml(session.category)}</span>` : ''}
${session.review_id ? `<span>审核ID: ${escapeHtml(session.review_id)}</span>` : ''}
</div>
<div class="task-time">${session.created_at || ''}
${session.finished_at ? ' → ' + session.finished_at : ''}
</div>
</div>
<div class="task-actions">
<a href="/process" class="btn btn-sm btn-secondary">
<i class="ri-eye-line"></i> 监控
</a>
</div>
</div>
`;
}).join('');
}
}
} catch (error) {
console.error('加载处理会话出错:', error);
}
}
+80
View File
@@ -49,6 +49,86 @@
</div>
</div>
<!-- 智能体任务模板区域 -->
<div class="panel template-section">
<div class="panel-header">
<h2><i class="ri-robot-line"></i> 步骤4:提取产品数据 - 智能体任务模板</h2>
<div class="template-actions">
<button onclick="previewTemplate()" class="btn btn-secondary btn-sm">
<i class="ri-eye-line"></i> 预览
</button>
<button onclick="saveTemplate()" class="btn btn-primary btn-sm">
<i class="ri-save-line"></i> 保存模板
</button>
</div>
</div>
<div class="panel-body">
<div class="template-info">
<p><strong>说明:</strong>此模板用于步骤4「提取产品数据」中调用智能体 <code>hz4th_editor</code> 的任务文本。</p>
<p><strong>可用变量:</strong>
<code>{{product_name}}</code> 产品名称、
<code>{{category}}</code> 类别、
<code>{{subcategory}}</code> 子类别、
<code>{{library_results}}</code> 内容库搜索结果、
<code>{{internet_results}}</code> 互联网抓取内容
</p>
<p><strong>调用命令:</strong><code>openclaw agent --agent hz4th_editor --message "[填充后的任务文本]"</code></p>
</div>
<textarea id="agent-template-editor" class="template-editor" rows="15" placeholder="加载模板中..."></textarea>
</div>
</div>
<!-- 步骤5填充字段模板区域 -->
<div class="panel template-section">
<div class="panel-header">
<h2><i class="ri-edit-box-line"></i> 步骤5:填充字段并提交 - 智能体任务模板</h2>
<div class="template-actions">
<button onclick="previewFillFieldsTemplate()" class="btn btn-secondary btn-sm">
<i class="ri-eye-line"></i> 预览
</button>
<button onclick="saveFillFieldsTemplate()" class="btn btn-primary btn-sm">
<i class="ri-save-line"></i> 保存模板
</button>
</div>
</div>
<div class="panel-body">
<div class="template-info">
<p><strong>说明:</strong>此模板用于步骤5「填充字段并提交审核」中调用智能体 <code>hz4th_editor</code> 的任务文本。</p>
<p><strong>可用变量:</strong>
<code>{{product_name}}</code> 产品名称、
<code>{{category}}</code> 类别、
<code>{{subcategory}}</code> 子类别、
<code>{{relevant_content_ids}}</code> 上一步筛选的相关内容数据ID
</p>
<p><strong>任务目标:</strong>智能体根据API文档获取字段定义,整理产品参数,并通过API提交到ParamHub审核系统。</p>
</div>
<textarea id="fill-fields-template-editor" class="template-editor" rows="15" placeholder="加载模板中..."></textarea>
</div>
</div>
<!-- 预览模态框 -->
<div id="template-preview-modal" class="modal">
<div class="modal-content large">
<div class="modal-header">
<h3><i class="ri-eye-line"></i> 任务文本预览</h3>
<button onclick="closeModal('template-preview-modal')" class="close-btn">
<i class="ri-close-line"></i>
</button>
</div>
<div class="modal-body">
<div class="preview-meta" style="margin-bottom: 15px;">
<label>产品名称: <input type="text" id="preview-product" value="示例产品" style="padding: 4px 8px; border: 1px solid #ddd; border-radius: 4px;"></label>
<label style="margin-left: 10px;">类别: <input type="text" id="preview-category" value="AI模型" style="padding: 4px 8px; border: 1px solid #ddd; border-radius: 4px;"></label>
<button onclick="doPreview()" class="btn btn-sm btn-secondary" style="margin-left: 10px;"><i class="ri-refresh-line"></i> 刷新预览</button>
</div>
<pre id="template-preview-content" style="background: #f8f9fa; padding: 15px; border-radius: 8px; white-space: pre-wrap; font-size: 13px; max-height: 500px; overflow-y: auto;">加载中...</pre>
</div>
<div class="modal-footer">
<button onclick="closeModal('template-preview-modal')" class="btn btn-secondary">关闭</button>
</div>
</div>
</div>
<!-- 处理历史区域 -->
<div class="panel history-section">
<div class="panel-header">
+20
View File
@@ -111,6 +111,26 @@
</div>
</div>
<!-- 产品处理会话区域 -->
<div class="panel process-sessions-section">
<div class="panel-header">
<h2><i class="ri-play-circle-line"></i> 产品处理会话</h2>
<div class="panel-actions">
<a href="/process" class="btn btn-primary btn-sm">
<i class="ri-external-link-line"></i> 监控页面
</a>
<button onclick="loadProcessSessions()" class="btn btn-secondary btn-sm">
<i class="ri-refresh-line"></i> 刷新
</button>
</div>
</div>
<div class="panel-body">
<div id="process-sessions-list" class="background-tasks-list">
<div class="empty-text">暂无处理会话</div>
</div>
</div>
</div>
<!-- 失败URL区域 -->
<div class="panel failed-urls-section">
<div class="panel-header">