Scrapling Web Fetch
VerifiedScrapling Web Fetch — documentation and writing tool. Supports html2text, token, Supports WeChat article cleanup.
$ Add to .claude/skills/ About This Skill
Content available in Chinese
# Scrapling Web Fetch
当用户要获取网页内容、正文提取、把网页转成 markdown/text、抓取文章主体时，优先使用此技能。
默认流程 1. 使用 `python3 scripts/scrapling_fetch.py <url> <max_chars>` 2. 默认正文选择器优先级： - `article` - `main` - `.post-content` - `[class*="body"]` 3. 命中正文后，使用 `html2text` 转 Markdown 4. 若都未命中，回退到 `body` 5. 最终按 `max_chars` 截断输出
用法 ```bash python3 /Users/zzd/.openclaw/workspace/skills/scrapling-web-fetch/scripts/scrapling_fetch.py <url> 30000 ```
依赖 优先检查： - `scrapling` - `html2text`
若缺失，可安装： ```bash python3 -m pip install scrapling html2text ```
输出约定 脚本默认输出 Markdown 正文内容。 如需结构化输出，可追加 `--json`。 如需调试提取命中了哪个 selector，可查看 stderr 输出。
附加资源 - 用法参考：`/Users/zzd/.openclaw/workspace/skills/scrapling-web-fetch/references/usage.md` - 选择器策略：`/Users/zzd/.openclaw/workspace/skills/scrapling-web-fetch/references/selectors.md` - 统一入口：`/Users/zzd/.openclaw/workspace/skills/scrapling-web-fetch/scripts/fetch-web-content`
何时用这个技能 - 获取文章正文 - 抓博客/新闻/公告正文 - 将网页转成 Markdown 供后续总结 - 常规 fetch 效果差，希望提升现代网页抓取稳定性
何时不用 - 需要完整浏览器交互、点击、登录、翻页时：改用浏览器自动化 - 只是简单获取 API JSON：直接请求 API 更合适
Use Cases
- Automate browser interactions for web scraping and testing
- Extract structured data from websites using headless browser automation
- Navigate websites, fill forms, and capture screenshots programmatically
- Scrape dynamic JavaScript-rendered content that simple HTTP requests cannot access
Pros & Cons
Pros
- +Generates professional-quality output suitable for direct use
- +Structured workflow ensures consistent documentation quality
- +Clear documentation makes it easy to get started and integrate
Cons
- -Requires installing external dependencies before use
- -Generated content may need manual review and editing for accuracy
- -Template-based approach may not suit highly specialized document formats
FAQ
What does Scrapling Web Fetch do?
Scrapling Web Fetch — documentation and writing tool. Supports html2text, token, Supports WeChat article cleanup.
What platforms support Scrapling Web Fetch?
Scrapling Web Fetch is available on Claude Code, OpenClaw.
What are the use cases for Scrapling Web Fetch?
Automate browser interactions for web scraping and testing. Extract structured data from websites using headless browser automation. Navigate websites, fill forms, and capture screenshots programmatically.
100+ free AI tools
Writing, PDF, image, and developer tools — all in your browser.
Next Step
Use the skill detail page to evaluate fit and install steps. For a direct browser workflow, move into a focused tool route instead of staying in broader support surfaces.