6 Commits
Author SHA1 Message Date
hz4th_coder 82ff8a2610 v1.7.3 前端新增后端选择器 + 暴露 /api/backends + 接通 chrome-cdp 后端
- 新增 GET /api/backends 返回可用后端/默认顺序/说明,供前端动态渲染
- 前端表单新增「捕获后端」下拉框: 自动(默认,优先agent-browser超时自动回退playwright)/agent-browser/playwright/chrome-cdp,不可用的置灰
- 接通 chrome-cdp 后端: 之前函数已实现但 capture_webpage 未接线(传 backend=chrome-cdp 会报 Unknown backend),现支持 screenshot/html/text,连接已开的 Chrome(--remote-debugging-port)
- 前端 currentData 携带 backend 字段
2026-08-31 16:56:22 +08:00
hz4th_coder 1e69e7ca57 v1.7.2 修复慢加载页面报 'Operation timed out':agent-browser 打开超时自动切 Playwright 重试
- 根因: agent-browser open 导航内部超时约30s(CDP command timed out: Page.navigate),慢加载/持续加载页面(如资源挂起导致 load 事件永不触发)会报 Failed to open URL: Operation timed out
- agent-browser open 子进程超时 30s→45s,让其返回自己的超时报错而非被硬杀
- 超时判定放宽到 timed out/timeout 全变体,触发自动回退: capture_webpage 与 smart_capture 在 agent-browser 打开超时时自动切 Playwright 重试(domcontentloaded 更宽容,不等全部资源)
- Playwright goto domcontentloaded 超时 30s→60s
- 错误提示更友好(说明慢加载+已自动回退+建议)
- 实测: 挂起资源测试页 agent-browser 超时→playwright 成功, 耗时~39s; 正常页面无回归
2026-08-31 16:51:17 +08:00
hz4th_coder 951f941eee v1.7.1 修复前端捕获报错: fetch headers 不允许非 ISO-8859-1 字符(中文'游客'导致 Failed to read headers)
- 前端 /api/capture 请求头去掉 X-Caller: '游客'(中文在 HTTP 头非法,浏览器 fetch 直接抛错)
- 保留 X-From: web(ASCII 安全),调用者缺省由服务端默认 游客,行为不变
2026-08-31 16:27:04 +08:00
hz4th_coder 9c63887e10 v1.7.0 历史记录标注调用者/调用方式 + 每次提取自动按月归档原始HTML + 修复agent-browser HTML JSON转义bug
- 历史记录新增 caller(调用者: 游客/项目名) 与 call_method(调用方式: web/api)
  * 请求体 caller 或请求头 X-Caller/X-Project/X-App 标识调用者, 默认游客
  * 请求体 call_method 或 X-From 标识方式, 缺省按 Referer 自动判定(本前端=web, 否则api)
  * 前端提取请求带 X-From:web / X-Caller:游客; news-tracker 调用带 caller=news-tracker
- 每次提取后自动保存最原始 HTML 到 data/html/<YYYY-MM>/ 按月目录归档
  * screenshot/html/text/smart 四种动作均顺带抓取原始HTML并落盘
  * 历史详情新增 📄原始HTML文件 链接(GET /api/history/<id>/html), 删除记录连带删文件
- 修复: agent-browser 后端 html 提取返回 JSON 转义文本(非最原始HTML)的bug, 现解码为干净HTML
2026-08-31 13:12:22 +08:00
hz4th_coder e5ff90c46a 新增按需截图功能:滚动截图 + 视觉大模型实时判断
- 新 action=smart:每次截图后用视觉大模型判断主题内容是否已截全、是否还需向下滚动,自动滚动到判定完成,最后纵向拼接成长图
- 默认视觉模型 qwen3.6-plus(autodl),支持 agent-browser / playwright 双后端
- LLM 接口可网页配置:⚙️智能配置弹窗(base_url/api_key/model/提示词/最大滚动次数/滚动比例/超时) + 测试连接 + 持久化到 data/config.json
- 新增 API:GET/POST /api/smart/config、POST /api/smart/test
- 安全兜底:大模型判断失败保守停止、滚动位置不再变化判底部、最大滚动次数上限
- 历史新增 smart 类型:入库拼接长图+AI判断过程日志,详情弹窗可查看每一步截图判断
- 前端:智能模式提示条 + 结果展示拼接长图 + AI判断过程折叠查看 + 历史筛选/徽章/详情
2026-08-29 18:08:22 +08:00
hz4th_coder 2a9f187ce8 新增可读文本提取 + 历史记录分页管理
- 新增 text 动作:提取页面可读文本(innerText + 空行/行尾清理,剔除脚本样式标签),agent-browser 与 playwright 双后端支持,自动提取页面标题
- 提取历史持久化(SQLite WAL):每次捕获自动入库,截图文件存持久化目录
- 新增 API:GET /api/history(分页/类型筛选/搜索)、GET /api/history/<id>(详情)、GET /api/history/<id>/file(截图)、DELETE /api/history/<id>
- 前端:新增提取文本选项+干净文本展示;新增提取历史卡片(分页/筛选/搜索/查看弹窗/删除/下载/复制)
2026-08-29 17:37:43 +08:00
3 changed files with 1729 additions and 38 deletions
+3
View File
@@ -31,6 +31,9 @@ captures/
*.png
*.html
# 运行时数据(历史记录数据库 + 截图文件)
data/
# 环境变量
.env
.env.local
+871 -22
View File
File diff suppressed because it is too large Load Diff
+855 -16
View File
@@ -3,7 +3,7 @@
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Web Capture - 网页截图与代码提取</title>
<title>Web Capture - 网页截图/HTML/文本提取</title>
<style>
* {
margin: 0;
@@ -77,6 +77,7 @@
.radio-group {
display: flex;
gap: 20px;
flex-wrap: wrap;
}
.radio-label {
@@ -201,16 +202,62 @@
word-wrap: break-word;
}
.result-text {
background: #f8f9fa;
border: 1px solid #e0e0e0;
color: #222;
padding: 20px;
border-radius: 8px;
overflow: auto;
max-height: 600px;
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;
font-size: 14px;
line-height: 1.8;
white-space: pre-wrap;
word-wrap: break-word;
}
.actions {
margin-top: 20px;
display: flex;
gap: 10px;
flex-wrap: wrap;
}
.btn-secondary {
background: #6c757d;
padding: 10px 20px;
font-size: 14px;
width: auto;
margin-top: 0;
}
.btn-small {
background: #667eea;
color: white;
border: none;
padding: 6px 14px;
font-size: 13px;
border-radius: 6px;
cursor: pointer;
transition: background 0.2s;
}
.btn-small:hover {
background: #5568d8;
}
.btn-small.danger {
background: #dc3545;
}
.btn-small.danger:hover {
background: #c82333;
}
.btn-small:disabled {
opacity: 0.5;
cursor: not-allowed;
}
.error {
@@ -227,6 +274,266 @@
display: block;
}
/* ===== 历史记录区域 ===== */
.history-header {
display: flex;
align-items: center;
justify-content: space-between;
flex-wrap: wrap;
gap: 15px;
margin-bottom: 20px;
}
.history-header h3 {
font-size: 1.4em;
color: #333;
}
.history-toolbar {
display: flex;
gap: 10px;
flex-wrap: wrap;
align-items: center;
}
.history-toolbar select,
.history-toolbar input[type="text"] {
width: auto;
min-width: 120px;
padding: 8px 12px;
font-size: 14px;
}
.history-toolbar .search-input {
min-width: 220px;
}
.history-count {
color: #666;
font-size: 14px;
}
.history-table {
width: 100%;
border-collapse: collapse;
}
.history-table th,
.history-table td {
padding: 12px 10px;
text-align: left;
border-bottom: 1px solid #eee;
vertical-align: middle;
}
.history-table th {
font-size: 13px;
color: #666;
font-weight: 600;
white-space: nowrap;
}
.history-table td {
font-size: 14px;
}
.history-table tr:hover td {
background: #f8f9ff;
}
.badge {
display: inline-block;
padding: 3px 10px;
border-radius: 20px;
font-size: 12px;
font-weight: 600;
white-space: nowrap;
}
.badge-screenshot {
background: #e3f2fd;
color: #1565c0;
}
.badge-html {
background: #fff3e0;
color: #e65100;
}
.badge-text {
background: #e8f5e9;
color: #2e7d32;
}
.badge-failed {
background: #fdecea;
color: #c62828;
}
.badge-api {
background: #ede7f6;
color: #5e35b1;
}
.badge-web {
background: #e0f7fa;
color: #00695c;
}
.history-url {
max-width: 280px;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
color: #1565c0;
text-decoration: none;
display: inline-block;
vertical-align: bottom;
}
.history-title {
max-width: 220px;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
color: #333;
display: inline-block;
vertical-align: bottom;
}
.history-empty {
text-align: center;
padding: 40px;
color: #999;
font-size: 15px;
}
.pagination {
display: flex;
align-items: center;
justify-content: center;
gap: 8px;
margin-top: 20px;
flex-wrap: wrap;
}
.page-btn {
min-width: 36px;
padding: 7px 12px;
border: 1px solid #ddd;
background: white;
border-radius: 6px;
cursor: pointer;
font-size: 14px;
color: #333;
transition: all 0.2s;
}
.page-btn:hover:not(:disabled) {
border-color: #667eea;
color: #667eea;
}
.page-btn.active {
background: #667eea;
color: white;
border-color: #667eea;
}
.page-btn:disabled {
opacity: 0.4;
cursor: not-allowed;
}
.page-info {
color: #666;
font-size: 14px;
white-space: nowrap;
}
/* ===== 查看详情弹窗 ===== */
.modal-overlay {
display: none;
position: fixed;
top: 0;
left: 0;
right: 0;
bottom: 0;
background: rgba(0, 0, 0, 0.6);
z-index: 1000;
padding: 30px;
overflow-y: auto;
}
.modal-overlay.active {
display: flex;
}
.modal {
background: white;
border-radius: 14px;
width: 100%;
max-width: 900px;
margin: auto;
box-shadow: 0 20px 60px rgba(0, 0, 0, 0.4);
overflow: hidden;
}
.modal-header {
display: flex;
align-items: center;
justify-content: space-between;
padding: 18px 24px;
background: #f8f9fa;
border-bottom: 1px solid #eee;
gap: 15px;
flex-wrap: wrap;
}
.modal-header h3 {
font-size: 16px;
color: #333;
word-break: break-all;
}
.modal-meta {
padding: 10px 24px;
background: #f8f9fa;
border-bottom: 1px solid #eee;
font-size: 13px;
color: #666;
display: flex;
gap: 20px;
flex-wrap: wrap;
}
.modal-body {
padding: 20px 24px;
max-height: 70vh;
overflow: auto;
}
.modal-body .result-image {
width: 100%;
}
.modal-close {
background: #6c757d;
color: white;
border: none;
width: 32px;
height: 32px;
border-radius: 50%;
font-size: 18px;
cursor: pointer;
line-height: 1;
flex-shrink: 0;
}
.modal-close:hover {
background: #545b62;
}
.api-docs {
margin-top: 30px;
padding: 20px;
@@ -254,13 +561,24 @@
overflow-x: auto;
margin-top: 10px;
}
@media (max-width: 700px) {
.history-table th:nth-child(2),
.history-table td:nth-child(2),
.history-table th:nth-child(3),
.history-table td:nth-child(3),
.history-table th:nth-child(4),
.history-table td:nth-child(4) {
display: none;
}
}
</style>
</head>
<body>
<div class="container">
<div class="header">
<h1>🌐 Web Capture</h1>
<p>网页截图与代码提取服务</p>
<p>网页截图 / HTML / 可读文本 提取服务</p>
</div>
<div class="card">
@@ -281,8 +599,20 @@
<input type="radio" name="action" value="html">
📄 提取HTML
</label>
<label class="radio-label">
<input type="radio" name="action" value="text">
📝 提取文本
</label>
<label class="radio-label">
<input type="radio" name="action" value="smart">
🧠 按需截图(AI判断滚动)
</label>
<button type="button" class="btn-small" onclick="openSmartConfig()" style="margin-top:0;" title="配置视觉大模型接口">⚙️ 智能配置</button>
</div>
</div>
<div class="form-group" id="smartHint" style="display:none; padding:12px 16px; background:#eef2ff; border-radius:8px; color:#333; font-size:14px; line-height:1.8;">
🧠 <strong>按需截图模式</strong>:每次截图后由视觉大模型实时判断——是否已截全主题内容、还需不需要继续往下滚动,自动滚动直到大模型判定完成,最后拼成一张长图。大模型接口可在「⚙️ 智能配置」中随时修改(默认已配置 qwen3.6-plus)。
</div>
<div class="form-group">
<label>高级选项</label>
@@ -294,6 +624,16 @@
</div>
</div>
<div class="form-group">
<label>捕获后端</label>
<select id="backend" style="max-width:100%;">
<option value="auto">加载中…</option>
</select>
<small id="backendHint" style="color: #666; display: block; margin-top: 4px;">
自动 = 优先 agent-browser,打开页面超时自动回退 Playwright(推荐)
</small>
</div>
<div class="form-group">
<label>等待设置(重要!)</label>
<div class="row">
@@ -350,25 +690,153 @@
</div>
</div>
<div class="card">
<div class="history-header">
<h3>📜 提取历史</h3>
<div class="history-toolbar">
<select id="historyActionFilter" onchange="loadHistory(1)">
<option value="all">全部类型</option>
<option value="screenshot">📸 截图</option>
<option value="html">📄 HTML</option>
<option value="text">📝 文本</option>
<option value="smart">🧠 按需截图</option>
</select>
<input type="text" class="search-input" id="historySearch" placeholder="🔍 搜索网址/标题..." onkeydown="if(event.key==='Enter')loadHistory(1)">
<button class="btn-small" onclick="loadHistory(1)">搜索</button>
<button class="btn-small" onclick="refreshHistory()">🔄 刷新</button>
</div>
</div>
<div class="history-count" id="historyCount"></div>
<div style="overflow-x: auto; margin-top: 10px;">
<table class="history-table">
<thead>
<tr>
<th>时间</th>
<th>类型</th>
<th>调用者</th>
<th>方式</th>
<th>网址</th>
<th>标题</th>
<th>状态</th>
<th style="text-align: right;">操作</th>
</tr>
</thead>
<tbody id="historyBody"></tbody>
</table>
</div>
<div class="history-empty" id="historyEmpty" style="display: none;">暂无提取记录,先去上方提取一个网页吧 🚀</div>
<div class="pagination" id="pagination"></div>
</div>
<div class="card api-docs">
<h3>📚 使用说明</h3>
<p><strong>重要提示:</strong></p>
<ul style="margin: 10px 0; line-height: 1.8;">
<li>🧠 <strong>按需截图</strong>:每次截图后用视觉大模型实时判断是否已截全主题内容、是否还需继续滚动,自动滚动到内容结束并拼接成长图(大模型接口可在「⚙️ 智能配置」修改)</li>
<li>📝 <strong>提取文本</strong>:自动剔除脚本/样式/标签,得到干净的前端可读文本,适合直接用于分析/喂给大模型</li>
<li>⏱️ <strong>页面加载等待</strong>:某些网站需要验证过程(如 Cloudflare),请设置较长时间(10000-30000ms</li>
<li>🔄 <strong>滚动次数</strong>:用于加载动态内容(如微博、推特等),建议 3-5 次</li>
<li>🎯 <strong>全页截图</strong>:滚动加载后建议开启此选项</li>
<li>📜 <strong>提取历史</strong>:每次提取自动入库,支持分页浏览、按类型筛选、搜索与删除;每条记录会标注<strong>调用者</strong>(游客/项目名)与<strong>调用方式</strong>web/api),并自动把<strong>最原始 HTML</strong> 按月归档到 <code>data/html/&lt;月份&gt;/</code>(详情里可点「📄 原始HTML文件」查看)</li>
<li>👤 <strong>API 调用方标识</strong>:请求体带 <code>caller</code>(或请求头 <code>X-Caller</code>/<code>X-Project</code>)即可在历史里显示项目名;带 <code>call_method</code><code>X-From</code> 标注调用方式,缺省按 Referer 自动判定 web/api</li>
</ul>
<p style="margin-top: 15px;"><strong>API 调用:</strong></p>
<pre><code>POST /api/capture
{
"url": "https://example.com",
"action": "screenshot", // 或 "html"
"wait_time": 15000, // 重要!验证网站需设置较长等待
"scroll_times": 3, // 可选:加载动态内容
"scroll_delay": 1000, // 可选:滚动间隔
"full_page": true, // 可选:全页截图
"backend": "playwright" // 可选:playwright agent-browser
}</code></pre>
"action": "smart", // "screenshot" | "html" | "text" | "smart"
"wait_time": 15000, // 重要!验证网站需设置较长等待
"scroll_times": 3, // 可选:加载动态内容
"scroll_delay": 1000, // 可选:滚动间隔
"full_page": true, // 可选:全页截图
"backend": "playwright", // 可选:playwright / agent-browser
"caller": "news-tracker", // 可选:调用者标识(历史记录里显示,默认游客)
"call_method": "api", // 可选:调用方式 web/api(缺省自动判定)
"viewport": {"width":1280,"height":700},
"smart_config": { // 可选:临时覆盖按需截图 LLM 配置
"base_url": "https://www.autodl.art/api/v1",
"api_key": "sk-xxx",
"model": "qwen3.6-plus",
"max_scrolls": 20,
"scroll_ratio": 0.85
}
}
GET /api/smart/config // 获取按需截图 LLM 配置
POST /api/smart/config // 保存配置(body: {config:{...}}
POST /api/smart/test // 测试连接(body: {config:{...}} 可选)
GET /api/history?page=1&page_size=15&action=all&search=关键词 // 历史分页
GET /api/history/&lt;id&gt; // 历史详情
GET /api/history/&lt;id&gt;/file // 历史截图文件
DELETE /api/history/&lt;id&gt; // 删除历史</code></pre>
</div>
</div>
<!-- 智能截图配置弹窗 -->
<div class="modal-overlay" id="smartCfgOverlay" onclick="if(event.target===this)closeSmartConfig()">
<div class="modal">
<div class="modal-header">
<h3>⚙️ 智能截图 · 视觉大模型配置</h3>
<button class="modal-close" onclick="closeSmartConfig()"></button>
</div>
<div class="modal-body">
<div class="form-group">
<label for="scBaseUrl">接口地址 base_url</label>
<input type="text" id="scBaseUrl" placeholder="https://.../api/v1">
</div>
<div class="form-group">
<label for="scApiKey">API Key</label>
<input type="password" id="scApiKey" placeholder="sk-...">
</div>
<div class="form-group">
<label for="scModel">模型名称</label>
<input type="text" id="scModel" placeholder="qwen3.6-plus">
</div>
<div class="form-group">
<label for="scPrompt">判断提示词(告诉大模型如何判断是否截全)</label>
<textarea id="scPrompt" rows="7" style="width:100%; padding:12px; border:2px solid #e0e0e0; border-radius:8px; font-size:14px; font-family:inherit; line-height:1.6;"></textarea>
</div>
<div class="row">
<div>
<label for="scMaxScrolls">最大滚动次数</label>
<input type="number" id="scMaxScrolls" min="1" max="100" value="20">
</div>
<div>
<label for="scRatio">单次滚动比例(视口高度%)</label>
<input type="number" id="scRatio" min="10" max="100" value="85">
</div>
<div>
<label for="scTimeout">LLM 超时(秒)</label>
<input type="number" id="scTimeout" min="10" max="600" value="120">
</div>
</div>
<div id="scTestResult" style="margin-top:12px; font-size:14px; line-height:1.8;"></div>
</div>
<div class="actions" style="padding: 0 24px 20px;">
<button class="btn btn-secondary" onclick="testSmartConfig()">🧪 测试连接</button>
<button class="btn btn-secondary" onclick="saveSmartConfig()">💾 保存配置</button>
<button class="btn btn-secondary" onclick="closeSmartConfig()">关闭</button>
</div>
</div>
</div>
<!-- 历史详情弹窗 -->
<div class="modal-overlay" id="modalOverlay" onclick="if(event.target===this)closeModal()">
<div class="modal">
<div class="modal-header">
<h3 id="modalTitle">记录详情</h3>
<button class="modal-close" onclick="closeModal()"></button>
</div>
<div class="modal-meta" id="modalMeta"></div>
<div class="modal-body" id="modalBody"></div>
<div class="actions" style="padding: 0 24px 20px;">
<button class="btn btn-secondary" onclick="downloadHistoryContent()">💾 下载内容</button>
<button class="btn btn-secondary" onclick="copyHistoryContent()">📋 复制内容</button>
</div>
</div>
</div>
@@ -380,6 +848,128 @@
const submitBtn = document.getElementById('submitBtn');
let currentData = null;
let currentBlob = null;
let currentPage = 1;
let currentHistory = { page: 1, total: 0, total_pages: 1 };
let modalRecord = null;
let smartConfig = null;
let lastSmartHistoryId = null;
// 页面加载时读取历史 + 智能配置 + 可用后端
window.addEventListener('load', () => {
loadHistory(1);
loadSmartConfig();
loadBackends();
});
async function loadBackends() {
const sel = document.getElementById('backend');
const hint = document.getElementById('backendHint');
try {
const res = await fetch('/api/backends');
const data = await res.json();
if (!data.success) throw new Error(data.error || '加载失败');
const labels = data.labels || {};
const opts = ['auto'].concat(data.available || []);
sel.innerHTML = opts.map(v =>
`<option value="${v}">${escapeHtml(v)}${escapeHtml(labels[v] || '')}</option>`
).join('') + (data.unavailable || []).map(v =>
`<option value="${v}" disabled>${escapeHtml(v)}${escapeHtml(labels[v] || '')}(不可用)</option>`
).join('');
sel.value = 'auto';
hint.textContent = labels['auto'] || hint.textContent;
} catch (err) {
sel.innerHTML = '<option value="auto">自动</option>';
hint.textContent = '后端列表加载失败:' + err.message;
}
}
// 操作类型切换:智能模式显示提示条
document.querySelectorAll('input[name="action"]').forEach(r => {
r.addEventListener('change', () => {
const smart = document.querySelector('input[name="action"]:checked').value === 'smart';
document.getElementById('smartHint').style.display = smart ? 'block' : 'none';
});
});
/* ===== 智能截图配置 ===== */
async function loadSmartConfig() {
try {
const res = await fetch('/api/smart/config');
const data = await res.json();
if (data.success) smartConfig = data.config;
} catch (e) {}
}
function openSmartConfig() {
if (!smartConfig) return;
document.getElementById('scBaseUrl').value = smartConfig.base_url || '';
document.getElementById('scApiKey').value = smartConfig.api_key || '';
document.getElementById('scModel').value = smartConfig.model || '';
document.getElementById('scPrompt').value = smartConfig.prompt || '';
document.getElementById('scMaxScrolls').value = smartConfig.max_scrolls || 20;
document.getElementById('scRatio').value = Math.round((smartConfig.scroll_ratio || 0.85) * 100);
document.getElementById('scTimeout').value = smartConfig.timeout || 120;
document.getElementById('scTestResult').innerHTML = '';
document.getElementById('smartCfgOverlay').classList.add('active');
}
function closeSmartConfig() {
document.getElementById('smartCfgOverlay').classList.remove('active');
}
function collectSmartConfigForm() {
return {
base_url: document.getElementById('scBaseUrl').value.trim(),
api_key: document.getElementById('scApiKey').value.trim(),
model: document.getElementById('scModel').value.trim(),
prompt: document.getElementById('scPrompt').value.trim(),
max_scrolls: parseInt(document.getElementById('scMaxScrolls').value) || 20,
scroll_ratio: (parseInt(document.getElementById('scRatio').value) || 85) / 100,
timeout: parseInt(document.getElementById('scTimeout').value) || 120
};
}
async function testSmartConfig() {
const cfg = collectSmartConfigForm();
const el = document.getElementById('scTestResult');
el.innerHTML = '<span style="color:#667eea;">⏳ 正在测试连接...</span>';
try {
const res = await fetch('/api/smart/test', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ config: cfg })
});
const data = await res.json();
if (data.success) {
el.innerHTML = `<span style="color:#2e7d32;">✅ 连接成功!耗时 ${data.latency}s,模型 ${escapeHtml(data.model)} 回复:${escapeHtml(data.reply)}</span>`;
} else {
el.innerHTML = `<span style="color:#c62828;">❌ 测试失败:${escapeHtml(data.error || '未知错误')}</span>`;
}
} catch (err) {
el.innerHTML = `<span style="color:#c62828;">❌ 测试失败:${escapeHtml(err.message)}</span>`;
}
}
async function saveSmartConfig() {
const cfg = collectSmartConfigForm();
const el = document.getElementById('scTestResult');
try {
const res = await fetch('/api/smart/config', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ config: cfg })
});
const data = await res.json();
if (data.success) {
smartConfig = data.config;
el.innerHTML = '<span style="color:#2e7d32;">✅ 配置已保存并生效</span>';
} else {
el.innerHTML = `<span style="color:#c62828;">❌ 保存失败:${escapeHtml(data.error || '未知错误')}</span>`;
}
} catch (err) {
el.innerHTML = `<span style="color:#c62828;">❌ 保存失败:${escapeHtml(err.message)}</span>`;
}
}
form.addEventListener('submit', async (e) => {
e.preventDefault();
@@ -400,10 +990,12 @@
scroll_delay: scrollDelay,
full_page: fullPage,
wait_time: waitTime,
backend: document.getElementById('backend').value || 'auto',
viewport: {
width: viewportWidth,
height: viewportHeight
}
},
smart_config: action === 'smart' ? (smartConfig || {}) : undefined
};
// 显示加载
@@ -416,7 +1008,8 @@
const response = await fetch('/api/capture', {
method: 'POST',
headers: {
'Content-Type': 'application/json'
'Content-Type': 'application/json',
'X-From': 'web'
},
body: JSON.stringify(currentData)
});
@@ -429,16 +1022,37 @@
if (action === 'screenshot') {
currentBlob = await response.blob();
const imageUrl = URL.createObjectURL(currentBlob);
document.getElementById('resultContent').innerHTML =
document.getElementById('resultContent').innerHTML =
`<img src="${imageUrl}" class="result-image" alt="截图结果">`;
} else {
const data = await response.json();
currentBlob = new Blob([data.html], { type: 'text/html' });
document.getElementById('resultContent').innerHTML =
`<div class="result-code"><pre>${escapeHtml(data.html)}</pre></div>`;
if (action === 'html') {
currentBlob = new Blob([data.html], { type: 'text/html' });
document.getElementById('resultContent').innerHTML =
`<div class="result-code"><pre>${escapeHtml(data.html)}</pre></div>`;
} else if (action === 'smart') {
lastSmartHistoryId = data.history_id || null;
currentBlob = new Blob([JSON.stringify(data.steps || [], null, 2)], { type: 'application/json' });
const stepsHtml = (data.steps || []).map(s =>
`<li style="margin:4px 0;">第 <strong>${s.step}</strong> 次截图:${s.complete ? '✅ 已完整,停止滚动' : '⏳ 继续滚动'} —— ${escapeHtml(s.reason || '')}</li>`
).join('');
document.getElementById('resultContent').innerHTML =
`<div style="margin-bottom:10px;color:#666;font-size:13px;">页面标题:${escapeHtml(data.title || '无')} 🧠 AI 共滚动 <strong>${data.total_steps || 0}</strong> 次判定完成,已拼接为长图(下方展示)</div>` +
`<img src="/api/history/${data.history_id}/file" class="result-image" alt="智能截图长图" onerror="this.style.display='none'">` +
`<details style="margin-top:12px;"><summary style="cursor:pointer;color:#667eea;font-weight:600;">📋 查看 AI 判断过程(${(data.steps || []).length} 步)</summary>` +
`<ul style="margin:10px 0 0 20px;line-height:1.8;color:#333;font-size:14px;">${stepsHtml}</ul></details>`;
} else {
currentBlob = new Blob([data.text], { type: 'text/plain;charset=utf-8' });
const lines = (data.text || '').split('\n').length;
document.getElementById('resultContent').innerHTML =
`<div style="margin-bottom:10px;color:#666;font-size:13px;">页面标题:${escapeHtml(data.title || '无')} ${lines} 行,${(data.text || '').length} 字符(已剔除标签与无效字符)</div>` +
`<div class="result-text">${escapeHtml(data.text || '')}</div>`;
}
}
result.classList.add('active');
// 捕获成功后刷新历史(回到第一页看最新)
loadHistory(1);
} catch (err) {
error.textContent = '❌ ' + err.message;
error.classList.add('active');
@@ -455,12 +1069,24 @@
}
function downloadResult() {
if (!currentData) return;
if (currentData.action === 'smart') {
if (lastSmartHistoryId) {
const a = document.createElement('a');
a.href = `/api/history/${lastSmartHistoryId}/file`;
a.download = 'smart_capture.png';
a.click();
}
return;
}
if (!currentBlob) return;
const url = URL.createObjectURL(currentBlob);
const a = document.createElement('a');
a.href = url;
a.download = currentData.action === 'screenshot' ? 'screenshot.png' : 'page.html';
const name = currentData.action === 'screenshot' ? 'screenshot.png'
: currentData.action === 'html' ? 'page.html' : 'page.txt';
a.download = name;
a.click();
URL.revokeObjectURL(url);
}
@@ -482,6 +1108,219 @@
error.classList.remove('active');
currentData = null;
currentBlob = null;
lastSmartHistoryId = null;
}
/* ===== 历史记录 ===== */
async function loadHistory(page) {
const action = document.getElementById('historyActionFilter').value;
const search = document.getElementById('historySearch').value.trim();
const qs = new URLSearchParams({ page: page, page_size: 15, action, search });
const res = await fetch('/api/history?' + qs.toString());
const data = await res.json();
if (!data.success) {
document.getElementById('historyEmpty').style.display = 'block';
document.getElementById('historyEmpty').textContent = '❌ 加载历史失败:' + (data.error || '未知错误');
return;
}
currentPage = page;
currentHistory = data;
renderHistory(data);
}
function refreshHistory() {
loadHistory(currentPage);
}
function renderHistory(data) {
const body = document.getElementById('historyBody');
const empty = document.getElementById('historyEmpty');
const count = document.getElementById('historyCount');
count.textContent = `${data.total} 条记录`;
if (!data.records.length) {
body.innerHTML = '';
empty.style.display = 'block';
empty.textContent = '暂无提取记录,先去上方提取一个网页吧 🚀';
renderPagination(data);
return;
}
empty.style.display = 'none';
const badges = {
screenshot: '📸 截图',
html: '📄 HTML',
text: '📝 文本',
smart: '🧠 按需截图'
};
body.innerHTML = data.records.map(r => {
const badgeClass = r.status === 'failed' ? 'badge-failed' : ('badge-' + r.action);
const badgeText = r.status === 'failed' ? '❌ 失败' : (badges[r.action] || r.action);
const statusText = r.status === 'failed'
? `<span style="color:#c62828;" title="${escapeHtml(r.error || '')}">失败</span>`
: '<span style="color:#2e7d32;">成功</span>';
const callerText = r.caller || '游客';
const methodText = r.call_method || 'api';
const methodBadge = methodText === 'web' ? 'badge-web' : 'badge-api';
return `<tr>
<td style="white-space:nowrap;">${escapeHtml(r.created_at)}</td>
<td><span class="badge ${escapeHtml(badgeClass)}">${badgeText}</span></td>
<td><span class="history-title" title="${escapeHtml(callerText)}">${escapeHtml(callerText)}</span></td>
<td><span class="badge ${methodBadge}" style="font-size:11px;padding:2px 8px;">${escapeHtml(methodText)}</span></td>
<td><a class="history-url" href="${escapeHtml(r.url)}" target="_blank" rel="noopener">${escapeHtml(r.url)}</a></td>
<td><span class="history-title" title="${escapeHtml(r.title || '')}">${escapeHtml(r.title || '—')}</span></td>
<td>${statusText}</td>
<td style="text-align:right; white-space:nowrap;">
<button class="btn-small" onclick="viewRecord(${r.id})">👁 查看</button>
<button class="btn-small danger" onclick="deleteRecord(${r.id})">🗑</button>
</td>
</tr>`;
}).join('');
renderPagination(data);
}
function renderPagination(data) {
const el = document.getElementById('pagination');
if (!data.records.length) {
el.innerHTML = '';
return;
}
let html = '';
html += `<button class="page-btn" onclick="loadHistory(${data.page - 1})" ${data.page <= 1 ? 'disabled' : ''}> 上一页</button>`;
const total = data.total_pages;
const cur = data.page;
let start = Math.max(1, cur - 2);
let end = Math.min(total, cur + 2);
if (end - start < 4) {
start = Math.max(1, end - 4);
end = Math.min(total, start + 4);
}
if (start > 1) html += `<button class="page-btn" onclick="loadHistory(1)">1</button>`;
if (start > 2) html += `<span style="color:#999;">…</span>`;
for (let p = start; p <= end; p++) {
html += `<button class="page-btn ${p === cur ? 'active' : ''}" onclick="loadHistory(${p})">${p}</button>`;
}
if (end < total - 1) html += `<span style="color:#999;">…</span>`;
if (end < total) html += `<button class="page-btn" onclick="loadHistory(${total})">${total}</button>`;
html += `<button class="page-btn" onclick="loadHistory(${data.page + 1})" ${data.page >= total ? 'disabled' : ''}>下一页 </button>`;
html += `<span class="page-info">第 ${data.page} / ${total} 页</span>`;
el.innerHTML = html;
}
async function viewRecord(id) {
const res = await fetch('/api/history/' + id);
const data = await res.json();
if (!data.success) {
alert('加载失败:' + (data.error || '未知错误'));
return;
}
modalRecord = data.record;
const r = modalRecord;
document.getElementById('modalTitle').textContent = (r.title || r.url || '记录详情');
const badgeClass = r.status === 'failed' ? 'badge-failed' : ('badge-' + r.action);
const badgeText = r.status === 'failed' ? '❌ 失败' : ({screenshot: '📸 截图', html: '📄 HTML', text: '📝 文本'}[r.action] || r.action);
const htmlLink = r.html_path
? `<span><a href="/api/history/${r.id}/html" target="_blank" style="color:#667eea;font-weight:600;" title="${escapeHtml(r.html_path)}">📄 原始HTML文件</a></span>`
: '';
document.getElementById('modalMeta').innerHTML =
`<span>类型:<span class="badge ${badgeClass}">${badgeText}</span></span>` +
`<span>时间:${escapeHtml(r.created_at)}</span>` +
`<span>调用者:${escapeHtml(r.caller || '游客')}</span>` +
`<span>方式:${escapeHtml(r.call_method || 'api')}</span>` +
`<span>后端:${escapeHtml(r.backend || '—')}</span>` +
`<span>状态:${r.status === 'failed' ? '<span style="color:#c62828;">失败</span>' : '<span style="color:#2e7d32;">成功</span>'}</span>` +
htmlLink;
const body = document.getElementById('modalBody');
if (r.status === 'failed') {
body.innerHTML = `<div class="result-text" style="color:#c62828;">❌ 提取失败:${escapeHtml(r.error || '')}</div>`;
} else if (r.action === 'screenshot') {
body.innerHTML = `<img src="/api/history/${r.id}/file" class="result-image" alt="历史截图" onerror="this.parentElement.innerHTML='&lt;p style=&quot;color:#c62828;&quot;&gt;截图文件不存在或已删除&lt;/p&gt;'">`;
} else if (r.action === 'html') {
body.innerHTML = `<div class="result-code"><pre>${escapeHtml(r.content || '')}</pre></div>`;
} else if (r.action === 'smart') {
let stepsHtml = '';
let stepsCount = 0;
try {
const steps = JSON.parse(r.content || '[]');
stepsCount = steps.length;
stepsHtml = steps.map(s =>
`<li style="margin:4px 0;">第 <strong>${s.step}</strong> 次截图:${s.complete ? '✅ 已完整,停止滚动' : '⏳ 继续滚动'} —— ${escapeHtml(s.reason || '')}</li>`
).join('');
} catch (e) {}
body.innerHTML =
`<img src="/api/history/${r.id}/file" class="result-image" alt="智能截图长图" onerror="this.style.display='none'">` +
(stepsHtml ? `<details open style="margin-top:14px;"><summary style="cursor:pointer;color:#667eea;font-weight:600;">📋 AI 判断过程(${stepsCount} 步)</summary><ul style="margin:10px 0 0 20px;line-height:1.8;color:#333;font-size:14px;">${stepsHtml}</ul></details>` : '');
} else {
body.innerHTML = `<div class="result-text">${escapeHtml(r.content || '')}</div>`;
}
document.getElementById('modalOverlay').classList.add('active');
}
function closeModal() {
document.getElementById('modalOverlay').classList.remove('active');
modalRecord = null;
}
async function deleteRecord(id) {
if (!confirm('确定删除这条记录吗?截图文件将一并删除。')) return;
const res = await fetch('/api/history/' + id, { method: 'DELETE' });
const data = await res.json();
if (data.success) {
// 若当前页删空了则回退一页
if (currentHistory.records.length === 1 && currentPage > 1) {
loadHistory(currentPage - 1);
} else {
loadHistory(currentPage);
}
} else {
alert('删除失败:' + (data.error || '未知错误'));
}
}
function currentModalBlob() {
if (!modalRecord) return null;
if (modalRecord.action === 'screenshot') return null; // 图片用链接下载
const type = modalRecord.action === 'html' ? 'text/html' : 'text/plain;charset=utf-8';
return new Blob([modalRecord.content || ''], { type });
}
function downloadHistoryContent() {
if (!modalRecord) return;
if (modalRecord.action === 'screenshot' || modalRecord.action === 'smart') {
const a = document.createElement('a');
a.href = `/api/history/${modalRecord.id}/file`;
a.download = modalRecord.action === 'smart' ? `smart_${modalRecord.id}.png` : `capture_${modalRecord.id}.png`;
a.click();
return;
}
const blob = currentModalBlob();
if (!blob) return;
const url = URL.createObjectURL(blob);
const a = document.createElement('a');
a.href = url;
a.download = modalRecord.action === 'html' ? `page_${modalRecord.id}.html` : `page_${modalRecord.id}.txt`;
a.click();
URL.revokeObjectURL(url);
}
function copyHistoryContent() {
if (!modalRecord || !modalRecord.content) return;
navigator.clipboard.writeText(modalRecord.content).then(() => {
alert('已复制到剪贴板');
});
}
</script>
</body>