v1.4.2 屏蔽排除规则自动清理队列: 设置 exclude 后待爬队列同步移除命中 URL
- engine.url_excluded: 与 filter_links 同逻辑的排除判定辅助函数 - 保存 auto 配置时, 若 exclude 规则变化且任务未运行, 自动从 pending 队列 剔除命中规则的 URL (visited 已爬标记保留) - 排除输入框文案优化 - cnblogs-auto 已配置 exclude=[MyComments.html, OtherPosts.html, /comments], 清理 546 条用户中心页
This commit is contained in:
+1
-1
@@ -126,7 +126,7 @@
|
||||
<div class="field"><label>包含规则(每行一个,子串或正则)</label>
|
||||
<textarea name="include" rows="3" placeholder="techpowerup.com/review /news/"></textarea></div>
|
||||
<div class="field"><label>排除规则</label>
|
||||
<textarea name="exclude" rows="3" placeholder="login, signup, /tag/, /forum/"></textarea></div>
|
||||
<textarea name="exclude" rows="3" placeholder="每行一个关键词,URL 含任一关键词即不爬取 如: MyComments.html、OtherPosts.html、/comments 勾选正则后按正则匹配"></textarea></div>
|
||||
</div>
|
||||
<div class="row2">
|
||||
<div class="field check"><label><input name="same_domain" type="checkbox" checked> 仅爬同域名</label></div>
|
||||
|
||||
Reference in New Issue
Block a user