Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
17431e86cf | ||
|
|
82ff8a2610 | ||
|
|
1e69e7ca57 | ||
|
|
951f941eee | ||
|
|
9c63887e10 | ||
|
|
e5ff90c46a | ||
|
|
2a9f187ce8 | ||
|
|
f7f14d7114 | ||
|
|
ef4d26f11b | ||
|
|
fc43e6e806 | ||
|
|
5783feb60d | ||
|
|
98ef180878 | ||
|
|
6b7e856e38 | ||
|
|
ccb8479242 | ||
|
|
8f9bdff00c |
@@ -31,6 +31,9 @@ captures/
|
||||
*.png
|
||||
*.html
|
||||
|
||||
# 运行时数据(历史记录数据库 + 截图文件)
|
||||
data/
|
||||
|
||||
# 环境变量
|
||||
.env
|
||||
.env.local
|
||||
+3
-3
@@ -49,10 +49,10 @@ COPY . .
|
||||
# 创建临时目录
|
||||
RUN mkdir -p /tmp/web_captures
|
||||
|
||||
EXPOSE 5000
|
||||
EXPOSE 16025
|
||||
|
||||
# 健康检查
|
||||
HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
|
||||
CMD curl -f http://localhost:5000/health || exit 1
|
||||
CMD curl -f http://localhost:16025/health || exit 1
|
||||
|
||||
CMD ["gunicorn", "-w", "4", "-b", "0.0.0.0:5000", "--timeout", "120", "app:app"]
|
||||
CMD ["gunicorn", "-w", "4", "-b", "0.0.0.0:16025", "--timeout", "120", "app:app"]
|
||||
@@ -0,0 +1,43 @@
|
||||
# Web Capture API 项目
|
||||
|
||||
## 待推送信息
|
||||
|
||||
- **仓库地址**: http://121.40.164.32:12007/hz4th_coder/web-capture-api.git
|
||||
- **当前版本**: v1.0.2
|
||||
- **提交历史**:
|
||||
- v1.0.0: 初始化项目
|
||||
- v1.0.1: 修改端口为 16025
|
||||
- v1.0.2: 修复 agent-browser socket 权限问题
|
||||
|
||||
## 操作步骤
|
||||
|
||||
### 1. 在 Git 平台创建仓库
|
||||
|
||||
请大爷登录 Git 平台 (http://121.40.164.32:12007) 并创建仓库:
|
||||
- 仓库名称: `web-capture-api`
|
||||
- 所属用户: `hz4th_coder`
|
||||
- 可见性: 公开或私有(根据需要)
|
||||
|
||||
### 2. 推送代码
|
||||
|
||||
仓库创建后,运行以下命令:
|
||||
|
||||
```bash
|
||||
cd /home/openclaw/.openclaw/workspace-hz4th_coder/works/web-capture-api
|
||||
git push -u origin master --tags
|
||||
```
|
||||
|
||||
## 项目功能
|
||||
|
||||
- 网页截图(全页或视口)
|
||||
- HTML 代码提取
|
||||
- 自动滚动加载动态内容
|
||||
- 自定义视口大小
|
||||
- Web 界面和 RESTful API
|
||||
- 支持 agent-browser 和 Playwright 双后端
|
||||
- Docker 部署支持
|
||||
|
||||
## 服务地址
|
||||
|
||||
- Web 界面: http://192.168.0.101:16025
|
||||
- API 接口: http://192.168.0.101:16025/api/capture
|
||||
@@ -104,18 +104,18 @@ API 信息接口
|
||||
|
||||
```bash
|
||||
# 基础截图
|
||||
curl -X POST http://localhost:5000/api/capture \
|
||||
curl -X POST http://localhost:16025/api/capture \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"url": "https://example.com", "action": "screenshot"}' \
|
||||
--output screenshot.png
|
||||
|
||||
# 提取HTML
|
||||
curl -X POST http://localhost:5000/api/capture \
|
||||
curl -X POST http://localhost:16025/api/capture \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"url": "https://example.com", "action": "html"}'
|
||||
|
||||
# 滚动加载后全页截图
|
||||
curl -X POST http://localhost:5000/api/capture \
|
||||
curl -X POST http://localhost:16025/api/capture \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"url": "https://news.ycombinator.com",
|
||||
@@ -127,7 +127,7 @@ curl -X POST http://localhost:5000/api/capture \
|
||||
--output full_page.png
|
||||
|
||||
# 自定义视口
|
||||
curl -X POST http://localhost:5000/api/capture \
|
||||
curl -X POST http://localhost:16025/api/capture \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"url": "https://example.com",
|
||||
@@ -144,7 +144,7 @@ import requests
|
||||
|
||||
# 截图
|
||||
response = requests.post(
|
||||
'http://localhost:5000/api/capture',
|
||||
'http://localhost:16025/api/capture',
|
||||
json={
|
||||
'url': 'https://example.com',
|
||||
'action': 'screenshot'
|
||||
@@ -155,7 +155,7 @@ with open('screenshot.png', 'wb') as f:
|
||||
|
||||
# 提取HTML
|
||||
response = requests.post(
|
||||
'http://localhost:5000/api/capture',
|
||||
'http://localhost:16025/api/capture',
|
||||
json={
|
||||
'url': 'https://example.com',
|
||||
'action': 'html'
|
||||
@@ -169,7 +169,7 @@ print(html)
|
||||
|
||||
```javascript
|
||||
// 截图
|
||||
const response = await fetch('http://localhost:5000/api/capture', {
|
||||
const response = await fetch('http://localhost:16025/api/capture', {
|
||||
method: 'POST',
|
||||
headers: {'Content-Type': 'application/json'},
|
||||
body: JSON.stringify({
|
||||
@@ -182,7 +182,7 @@ const response = await fetch('http://localhost:5000/api/capture', {
|
||||
const blob = await response.blob();
|
||||
|
||||
// 提取HTML
|
||||
const response = await fetch('http://localhost:5000/api/capture', {
|
||||
const response = await fetch('http://localhost:16025/api/capture', {
|
||||
method: 'POST',
|
||||
headers: {'Content-Type': 'application/json'},
|
||||
body: JSON.stringify({
|
||||
@@ -199,7 +199,7 @@ console.log(data.html);
|
||||
### 使用 Gunicorn
|
||||
|
||||
```bash
|
||||
gunicorn -w 4 -b 0.0.0.0:5000 app:app
|
||||
gunicorn -w 4 -b 0.0.0.0:16025 app:app
|
||||
```
|
||||
|
||||
### 使用 Systemd 服务
|
||||
@@ -230,12 +230,12 @@ RUN playwright install chromium --with-deps
|
||||
|
||||
COPY . .
|
||||
EXPOSE 5000
|
||||
CMD ["gunicorn", "-w", "4", "-b", "0.0.0.0:5000", "app:app"]
|
||||
CMD ["gunicorn", "-w", "4", "-b", "0.0.0.0:16025", "app:app"]
|
||||
```
|
||||
|
||||
```bash
|
||||
docker build -t web-capture-api .
|
||||
docker run -p 5000:5000 web-capture-api
|
||||
docker run -p 16025:16025 web-capture-api
|
||||
```
|
||||
|
||||
## 性能优化
|
||||
|
||||
@@ -23,4 +23,4 @@ echo "运行方式:"
|
||||
echo " 开发模式: python3.12 app.py"
|
||||
echo " 生产模式: gunicorn -w 4 -b 0.0.0.0:5000 app:app"
|
||||
echo ""
|
||||
echo "访问地址: http://localhost:5000"
|
||||
echo "访问地址: http://localhost:16025"
|
||||
+2
-2
@@ -6,14 +6,14 @@ services:
|
||||
image: web-capture-api:latest
|
||||
container_name: web-capture-api
|
||||
ports:
|
||||
- "5000:5000"
|
||||
- "16025:16025"
|
||||
environment:
|
||||
- TZ=Asia/Shanghai
|
||||
volumes:
|
||||
- ./captures:/tmp/web_captures
|
||||
restart: unless-stopped
|
||||
healthcheck:
|
||||
test: ["CMD", "curl", "-f", "http://localhost:5000/health"]
|
||||
test: ["CMD", "curl", "-f", "http://localhost:16025/health"]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
retries: 3
|
||||
|
||||
+2
-2
@@ -110,7 +110,7 @@ echo " cd $(pwd)"
|
||||
echo " $PYTHON app.py"
|
||||
echo ""
|
||||
echo "📖 访问地址:"
|
||||
echo " http://localhost:5000"
|
||||
echo " http://localhost:16025"
|
||||
echo ""
|
||||
echo "📚 API 文档:"
|
||||
echo " http://localhost:5000/api"
|
||||
echo " http://localhost:16025/api"
|
||||
@@ -0,0 +1,69 @@
|
||||
# 手动 Chrome 捕获指南
|
||||
|
||||
## 方案三完整流程
|
||||
|
||||
### 1️⃣ 启动真实 Chrome(带调试端口)
|
||||
|
||||
在您的本地机器或服务器上运行:
|
||||
|
||||
```bash
|
||||
# Linux
|
||||
google-chrome --remote-debugging-port=9222 https://www.techpowerup.com/gpu-specs/rtx-pro-6000-blackwell.c4272
|
||||
|
||||
# 或者使用已有的 Chrome 进程(需要先关闭所有 Chrome)
|
||||
google-chrome-stable --remote-debugging-port=9222 --user-data-dir=/tmp/chrome-debug
|
||||
```
|
||||
|
||||
### 2️⃣ 手动验证(如 Cloudflare)
|
||||
|
||||
- 在浏览器中手动完成验证过程
|
||||
- 等待页面完全加载
|
||||
- 确认可以看到正常内容
|
||||
|
||||
### 3️⃣ 连接到 Chrome 并截图
|
||||
|
||||
验证完成后,运行脚本:
|
||||
|
||||
```bash
|
||||
# 连接到 Chrome 并截图
|
||||
python3.12 /tmp/connect_chrome.py techpowerup
|
||||
|
||||
# 或者捕获当前所有页面
|
||||
python3.12 /tmp/connect_chrome.py
|
||||
```
|
||||
|
||||
### 4️⃣ 查看结果
|
||||
|
||||
```bash
|
||||
# 查看截图
|
||||
ls -lh /tmp/chrome_capture_*.png
|
||||
|
||||
# 查看HTML
|
||||
head -100 /tmp/chrome_html_0.html
|
||||
```
|
||||
|
||||
## API 集成方案
|
||||
|
||||
如果需要通过 API 获取内容,可以:
|
||||
|
||||
1. 在服务代码中添加 CDP 连接选项
|
||||
2. 用户先手动验证
|
||||
3. API 连接到已验证的浏览器实例
|
||||
|
||||
## 注意事项
|
||||
|
||||
⚠️ **重要**:
|
||||
- Chrome 必须带 `--remote-debugging-port=9222` 启动
|
||||
- 所有 Chrome 进程必须先关闭(或使用新的 user-data-dir)
|
||||
- 验证完成后才能运行脚本
|
||||
- 截图后浏览器保持打开,可以继续使用
|
||||
|
||||
## 一键脚本
|
||||
|
||||
```bash
|
||||
# 启动 Chrome 并等待用户验证
|
||||
google-chrome --remote-debugging-port=9222 "https://www.techpowerup.com/gpu-specs/rtx-pro-6000-blackwell.c4272" &
|
||||
echo "请在浏览器中完成验证,然后按 Enter 继续截图..."
|
||||
read
|
||||
python3.12 /tmp/connect_chrome.py techpowerup
|
||||
```
|
||||
+872
-25
File diff suppressed because it is too large
Load Diff
+1
-1
@@ -7,7 +7,7 @@ import requests
|
||||
import json
|
||||
import sys
|
||||
|
||||
API_URL = "http://localhost:5000"
|
||||
API_URL = "http://localhost:16025"
|
||||
|
||||
|
||||
def test_health():
|
||||
|
||||
@@ -6,7 +6,7 @@ After=network.target
|
||||
Type=simple
|
||||
User=openclaw
|
||||
WorkingDirectory=/home/openclaw/.openclaw/workspace-hz4th_coder/works/web-capture-api
|
||||
ExecStart=/home/openclaw/.openclaw/workspace-hz4th_coder/works/web-capture-api/venv/bin/gunicorn -w 4 -b 0.0.0.0:5000 app:app
|
||||
ExecStart=/home/openclaw/.openclaw/workspace-hz4th_coder/works/web-capture-api/venv/bin/gunicorn -w 4 -b 0.0.0.0:16025 app:app
|
||||
Restart=always
|
||||
RestartSec=10
|
||||
|
||||
|
||||
Reference in New Issue
Block a user