v1.9.0 网址缓存查询(开关+时效TTL+强制刷新) + 提取失败自动重试(次数可配默认2)
1) 网址缓存:
- /api/capture 请求前先查提取历史: 同 url+action 成功记录且在 TTL 内直接返回(截图返回文件/文本内容返回数据), 不重新提取
- 缓存开关 enabled + 有效期 ttl_seconds(默认3600s=1小时) 可配; 请求带 refresh=true 强制绕过缓存
- 命中响应带 from_cache/cached_at/history_id/cache_ttl_seconds 标记
- GET/POST /api/capture/settings 配置接口; 前端⚙️缓存/重试设置弹窗(开关/有效期/重试次数/间隔) + 高级选项区缓存状态徽章 + 强制刷新勾选框 + 结果区缓存命中提示条
2) 失败重试:
- 提取失败(网络/超时/浏览器错误)自动重试, max_retries 默认2(最多尝试3次), delay_seconds 间隔默认2s
- 响应带 attempts=总尝试次数; 仅最终失败才入库 failed 记录
3) 修复: playwright goto 失败异常被吞(连接拒绝的地址返回空内容成功) -> 两次 goto 失败直接返回失败 + 错误页(chrome-error://about:blank)兜底判断, 保证真实失败能触发重试
验证: text/screenshot 缓存命中(cmp 同图)、refresh 绕过、重试 attempts=2/3、设置保存恢复
This commit is contained in:
+137
-2
@@ -608,6 +608,7 @@
|
||||
🧠 按需截图(AI判断滚动)
|
||||
</label>
|
||||
<button type="button" class="btn-small" onclick="openSmartConfig()" style="margin-top:0;" title="配置视觉大模型接口">⚙️ 智能配置</button>
|
||||
<button type="button" class="btn-small" onclick="openCaptureCfg()" style="margin-top:0;" title="网址缓存与失败重试设置">⚙️ 缓存/重试</button>
|
||||
</div>
|
||||
</div>
|
||||
<div class="form-group" id="smartHint" style="display:none; padding:12px 16px; background:#eef2ff; border-radius:8px; color:#333; font-size:14px; line-height:1.8;">
|
||||
@@ -621,7 +622,14 @@
|
||||
<input type="checkbox" id="fullPage">
|
||||
全页截图
|
||||
</label>
|
||||
<label class="checkbox-label">
|
||||
<input type="checkbox" id="forceRefresh">
|
||||
🔄 强制刷新(绕过网址缓存,重新提取)
|
||||
</label>
|
||||
</div>
|
||||
<small style="color: #666; display: block; margin-top: 4px;">
|
||||
📦 网址缓存已<span id="cacheStateBadge">…</span>(有效期 <span id="cacheTtlBadge">…</span>),同网址命中缓存直接返回历史结果,不会重复提取。
|
||||
</small>
|
||||
</div>
|
||||
|
||||
<div class="form-group">
|
||||
@@ -738,6 +746,8 @@
|
||||
<p><strong>重要提示:</strong></p>
|
||||
<ul style="margin: 10px 0; line-height: 1.8;">
|
||||
<li>🧠 <strong>按需截图</strong>:每次截图后用视觉大模型实时判断是否已截全主题内容、是否还需继续滚动,自动滚动到内容结束并拼接成长图;支持<strong>多视觉大模型接口配置</strong>(「⚙️ 智能配置」中增删改,列表顺序=优先级,失败/报错自动降级到下一个接口)</li>
|
||||
<li>📦 <strong>网址缓存</strong>:默认开启——同一网址先查提取历史(同动作、成功、在有效期内)直接返回,不重复提取;未命中才实时提取。可「⚙️ 缓存/重试」关开关/改有效期,或请求带 <code>refresh:true</code> 强制绕过</li>
|
||||
<li>🔁 <strong>失败自动重试</strong>:提取失败(网络/超时/浏览器错误)自动重试,次数可设(默认 2 次),响应带 <code>attempts</code> 字段</li>
|
||||
<li>📊 <strong>提取历史统计</strong>:历史区点「📊 统计」,可按天/操作类型/后端/状态/调用者/调用方式等角度切换图表(柱状/饼图可换,支持 7/30/90 天范围,可下载 PNG)</li>
|
||||
<li>📝 <strong>提取文本</strong>:自动剔除脚本/样式/标签,得到干净的前端可读文本,适合直接用于分析/喂给大模型</li>
|
||||
<li>⏱️ <strong>页面加载等待</strong>:某些网站需要验证过程(如 Cloudflare),请设置较长时间(10000-30000ms)</li>
|
||||
@@ -751,6 +761,7 @@
|
||||
{
|
||||
"url": "https://example.com",
|
||||
"action": "smart", // "screenshot" | "html" | "text" | "smart"
|
||||
"refresh": false, // 可选:true=强制实时提取(绕过网址缓存)
|
||||
"wait_time": 15000, // 重要!验证网站需设置较长等待
|
||||
"scroll_times": 3, // 可选:加载动态内容
|
||||
"scroll_delay": 1000, // 可选:滚动间隔
|
||||
@@ -772,6 +783,7 @@
|
||||
GET /api/smart/config // 获取智能截图配置(含 endpoints 多接口列表)
|
||||
POST /api/smart/config // 保存配置(body: {config:{endpoints:[...], prompt, max_scrolls,...}})
|
||||
POST /api/smart/test // 测试指定接口(body: {endpoint_id} 或 {endpoint:{...}} 或旧版 {config:{...}})
|
||||
GET /api/capture/settings // 网址缓存+失败重试 配置({cache:{enabled,ttl_seconds}, retry:{max_retries,delay_seconds}})
|
||||
GET /api/history?page=1&page_size=15&action=all&search=关键词 // 历史分页
|
||||
GET /api/history/stats?dimension=day&days=30 // 历史统计(dimension: day/action/backend/status/caller/method)
|
||||
GET /api/history/<id> // 历史详情
|
||||
@@ -825,6 +837,57 @@ DELETE /api/history/<id> // 删除历史</code></pre>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- 网址缓存 + 失败重试 设置弹窗 -->
|
||||
<div class="modal-overlay" id="captureCfgOverlay" onclick="if(event.target===this)closeCaptureCfg()">
|
||||
<div class="modal">
|
||||
<div class="modal-header">
|
||||
<h3>⚙️ 网址缓存 & 失败重试</h3>
|
||||
<button class="modal-close" onclick="closeCaptureCfg()">✕</button>
|
||||
</div>
|
||||
<div class="modal-body">
|
||||
<div style="padding:12px 16px;background:#eef2ff;border-radius:8px;font-size:13px;line-height:1.8;color:#333;margin-bottom:16px;">
|
||||
📦 <strong>网址缓存</strong>:启用后,请求同一网址时会先从提取历史中查一次(同动作、成功、且在有效期内),命中直接返回历史结果,<strong>不重新提取</strong>;未命中才实时提取。勾选捕获页的「强制刷新」可绕过缓存。
|
||||
</div>
|
||||
<div style="padding:14px;border:1px solid #e0e0e0;border-radius:10px;margin-bottom:16px;background:#fafbff;">
|
||||
<div style="display:flex;align-items:center;gap:10px;margin-bottom:12px;">
|
||||
<strong style="color:#333;flex-shrink:0;">📦 启用网址缓存</strong>
|
||||
<label class="checkbox-label" style="font-size:14px;">
|
||||
<input type="checkbox" id="cacheEnabled"> 开/关
|
||||
</label>
|
||||
</div>
|
||||
<div style="display:flex;gap:14px;flex-wrap:wrap;">
|
||||
<div style="flex:1;min-width:180px;">
|
||||
<label for="cacheTtl" style="font-size:13px;">缓存有效期(分钟)</label>
|
||||
<input type="number" id="cacheTtl" min="1" max="10080" value="60" style="width:100%;padding:10px;border:1.5px solid #e0e0e0;border-radius:6px;">
|
||||
<small style="color:#999;display:block;margin-top:4px;">超期后再次提取自动刷新缓存。默认 60 分钟。</small>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div style="padding:14px;border:1px solid #e0e0e0;border-radius:10px;background:#fafbff;">
|
||||
<div style="font-size:13px;line-height:1.8;color:#333;margin-bottom:12px;">
|
||||
🔁 <strong>失败重试</strong>:提取失败(网络/超时/浏览器错误)时自动重试,重试次数可设置。
|
||||
</div>
|
||||
<div style="display:flex;gap:14px;flex-wrap:wrap;">
|
||||
<div style="flex:1;min-width:150px;">
|
||||
<label for="retryCount" style="font-size:13px;">重试次数(默认 2)</label>
|
||||
<input type="number" id="retryCount" min="0" max="10" value="2" style="width:100%;padding:10px;border:1.5px solid #e0e0e0;border-radius:6px;">
|
||||
<small style="color:#999;display:block;margin-top:4px;">0 = 不重试。2 即最多尝试 3 次。</small>
|
||||
</div>
|
||||
<div style="flex:1;min-width:150px;">
|
||||
<label for="retryDelay" style="font-size:13px;">重试间隔(秒)</label>
|
||||
<input type="number" id="retryDelay" min="0" max="60" value="2" style="width:100%;padding:10px;border:1.5px solid #e0e0e0;border-radius:6px;">
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div id="captureCfgResult" style="margin-top:12px;font-size:14px;line-height:1.8;"></div>
|
||||
</div>
|
||||
<div class="actions" style="padding: 0 24px 20px;">
|
||||
<button class="btn btn-secondary" onclick="saveCaptureCfg()">💾 保存设置</button>
|
||||
<button class="btn btn-secondary" onclick="closeCaptureCfg()">关闭</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- 历史图示统计弹窗 -->
|
||||
<div class="modal-overlay" id="statsOverlay" onclick="if(event.target===this)closeStats()">
|
||||
<div class="modal">
|
||||
@@ -884,14 +947,79 @@ DELETE /api/history/<id> // 删除历史</code></pre>
|
||||
let modalRecord = null;
|
||||
let smartConfig = null;
|
||||
let lastSmartHistoryId = null;
|
||||
let captureSettings = { cache: { enabled: true, ttl_seconds: 3600 }, retry: { max_retries: 2, delay_seconds: 2 } };
|
||||
|
||||
// 页面加载时读取历史 + 智能配置 + 可用后端
|
||||
// 页面加载时读取历史 + 智能配置 + 可用后端 + 缓存/重试设置
|
||||
window.addEventListener('load', () => {
|
||||
loadHistory(1);
|
||||
loadSmartConfig();
|
||||
loadBackends();
|
||||
loadCaptureSettings();
|
||||
});
|
||||
|
||||
function updateCacheBadge() {
|
||||
const en = captureSettings.cache && captureSettings.cache.enabled;
|
||||
document.getElementById('cacheStateBadge').textContent = en ? '✅ 开启' : '⛔ 关闭';
|
||||
document.getElementById('cacheStateBadge').style.color = en ? '#2e7d32' : '#c62828';
|
||||
const min = Math.round((captureSettings.cache && captureSettings.cache.ttl_seconds || 3600) / 60);
|
||||
document.getElementById('cacheTtlBadge').textContent = min + ' 分钟';
|
||||
}
|
||||
|
||||
async function loadCaptureSettings() {
|
||||
try {
|
||||
const res = await fetch('/api/capture/settings');
|
||||
const data = await res.json();
|
||||
if (data.success) captureSettings = data.settings;
|
||||
} catch (e) {}
|
||||
updateCacheBadge();
|
||||
}
|
||||
|
||||
function openCaptureCfg() {
|
||||
document.getElementById('cacheEnabled').checked = !!(captureSettings.cache && captureSettings.cache.enabled);
|
||||
document.getElementById('cacheTtl').value = Math.round(((captureSettings.cache && captureSettings.cache.ttl_seconds) || 3600) / 60);
|
||||
document.getElementById('retryCount').value = (captureSettings.retry && captureSettings.retry.max_retries != null) ? captureSettings.retry.max_retries : 2;
|
||||
document.getElementById('retryDelay').value = (captureSettings.retry && captureSettings.retry.delay_seconds != null) ? captureSettings.retry.delay_seconds : 2;
|
||||
document.getElementById('captureCfgResult').innerHTML = '';
|
||||
document.getElementById('captureCfgOverlay').classList.add('active');
|
||||
}
|
||||
|
||||
function closeCaptureCfg() {
|
||||
document.getElementById('captureCfgOverlay').classList.remove('active');
|
||||
}
|
||||
|
||||
async function saveCaptureCfg() {
|
||||
const el = document.getElementById('captureCfgResult');
|
||||
const settings = {
|
||||
cache: {
|
||||
enabled: document.getElementById('cacheEnabled').checked,
|
||||
ttl_seconds: (parseInt(document.getElementById('cacheTtl').value) || 60) * 60
|
||||
},
|
||||
retry: {
|
||||
max_retries: Math.max(0, parseInt(document.getElementById('retryCount').value) || 0),
|
||||
delay_seconds: Math.max(0, parseInt(document.getElementById('retryDelay').value) || 0)
|
||||
}
|
||||
};
|
||||
el.innerHTML = '<span style="color:#667eea;">⏳ 正在保存...</span>';
|
||||
try {
|
||||
const res = await fetch('/api/capture/settings', {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ settings })
|
||||
});
|
||||
const data = await res.json();
|
||||
if (data.success) {
|
||||
captureSettings = data.settings;
|
||||
updateCacheBadge();
|
||||
const en = data.settings.cache.enabled;
|
||||
el.innerHTML = `✅ 已保存:网址缓存${en ? '开启' : '关闭'}(有效期 ${Math.round(data.settings.cache.ttl_seconds / 60)} 分钟)| 失败重试 ${data.settings.retry.max_retries} 次(间隔 ${data.settings.retry.delay_seconds}s)`;
|
||||
} else {
|
||||
el.innerHTML = `<span style="color:#c62828;">❌ 保存失败:${escapeHtml(data.error || '未知错误')}</span>`;
|
||||
}
|
||||
} catch (err) {
|
||||
el.innerHTML = `<span style="color:#c62828;">❌ 保存失败:${escapeHtml(err.message)}</span>`;
|
||||
}
|
||||
}
|
||||
|
||||
async function loadBackends() {
|
||||
const sel = document.getElementById('backend');
|
||||
const hint = document.getElementById('backendHint');
|
||||
@@ -1313,6 +1441,7 @@ DELETE /api/history/<id> // 删除历史</code></pre>
|
||||
currentData = {
|
||||
url,
|
||||
action,
|
||||
refresh: document.getElementById('forceRefresh').checked,
|
||||
scroll_times: scrollTimes,
|
||||
scroll_delay: scrollDelay,
|
||||
full_page: fullPage,
|
||||
@@ -1353,9 +1482,13 @@ DELETE /api/history/<id> // 删除历史</code></pre>
|
||||
`<img src="${imageUrl}" class="result-image" alt="截图结果">`;
|
||||
} else {
|
||||
const data = await response.json();
|
||||
const cacheTag = data.from_cache
|
||||
? `<div style="margin-bottom:10px;padding:8px 14px;background:#fff8e1;border:1px solid #ffe082;border-radius:8px;font-size:13px;color:#795548;">📦 缓存命中:直接返回历史记录(#${data.history_id},提取于 ${escapeHtml(data.cached_at || '')},缓存有效期 ${data.cache_ttl_seconds / 60} 分钟)。勾选「强制刷新」可绕过缓存重新提取。</div>`
|
||||
: '';
|
||||
if (action === 'html') {
|
||||
currentBlob = new Blob([data.html], { type: 'text/html' });
|
||||
document.getElementById('resultContent').innerHTML =
|
||||
cacheTag +
|
||||
`<div class="result-code"><pre>${escapeHtml(data.html)}</pre></div>`;
|
||||
} else if (action === 'smart') {
|
||||
lastSmartHistoryId = data.history_id || null;
|
||||
@@ -1364,6 +1497,7 @@ DELETE /api/history/<id> // 删除历史</code></pre>
|
||||
`<li style="margin:4px 0;">第 <strong>${s.step}</strong> 次截图:${s.complete ? '✅ 已完整,停止滚动' : '⏳ 继续滚动'} —— ${escapeHtml(s.reason || '')}${s.model ? `<span style="color:#888;font-size:12px;">(模型:${escapeHtml(s.model)}</span>${s.endpoint ? `<span style="color:#888;font-size:12px;"> / ${escapeHtml(s.endpoint)}</span>` : ''}<span style="color:#888;font-size:12px;">)</span>` : ''}</li>`
|
||||
).join('');
|
||||
document.getElementById('resultContent').innerHTML =
|
||||
cacheTag +
|
||||
`<div style="margin-bottom:10px;color:#666;font-size:13px;">页面标题:${escapeHtml(data.title || '无')} | 🧠 AI 共滚动 <strong>${data.total_steps || 0}</strong> 次判定完成,已拼接为长图(下方展示)</div>` +
|
||||
`<img src="/api/history/${data.history_id}/file" class="result-image" alt="智能截图长图" onerror="this.style.display='none'">` +
|
||||
`<details style="margin-top:12px;"><summary style="cursor:pointer;color:#667eea;font-weight:600;">📋 查看 AI 判断过程(${(data.steps || []).length} 步)</summary>` +
|
||||
@@ -1372,7 +1506,8 @@ DELETE /api/history/<id> // 删除历史</code></pre>
|
||||
currentBlob = new Blob([data.text], { type: 'text/plain;charset=utf-8' });
|
||||
const lines = (data.text || '').split('\n').length;
|
||||
document.getElementById('resultContent').innerHTML =
|
||||
`<div style="margin-bottom:10px;color:#666;font-size:13px;">页面标题:${escapeHtml(data.title || '无')} | 共 ${lines} 行,${(data.text || '').length} 字符(已剔除标签与无效字符)</div>` +
|
||||
cacheTag +
|
||||
`<div style="margin-bottom:10px;color:#666;font-size:13px;">页面标题:${escapeHtml(data.title || '无')} | 共 ${lines} 行,${(data.text || '').length} 字符(已剔除标签与无效字符)${data.from_cache ? ` | 📦 缓存命中(历史 #${data.history_id})` : ''}</div>` +
|
||||
`<div class="result-text">${escapeHtml(data.text || '')}</div>`;
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user