Scrapling

What it does

When your agent uses it

• 抓取/爬取网页内容或数据（scrape
• 需要绕过 Cloudflare/WAF 等反爬保护
• 解析已有 HTML 提取结构化数据
• 用户提供 URL 并要求获取页面内容或特定元素

Example prompts

• “/scrapling”

Requirements

• Python 3
• Pre-approved tools (allowed-tools): Bash(python*), Bash(pip*), Bash(uv*), Bash(scrapling*)

What it can do on your machine

Read from SKILL.md and the folder at commit 85dd89f. It shows what the files ask for, not the result of running them.

• Tool permissions

  Pre-approves these tools, so the agent can use them without asking each time:

  • Bash(python*)
  • Bash(pip*)
  • Bash(uv*)
  • Bash(scrapling*)

  From allowed-tools in the SKILL.md frontmatter.

• Runs code

  Ships script files (Python), which the agent can run.

  Shell commands in SKILL.md call:

  • uv
  • python

  From the folder's file list and the shell code blocks in SKILL.md.

• Network

  No URLs in SKILL.md. Its commands use uv, which can reach the network depending on how they are called.

  From URLs in SKILL.md, links to its own repository left out.

• Credentials

  Names no API keys, tokens, secrets or passwords.

  From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

Context cost

Scrapling loads about 1.1k tokens when it runs, and up to ~6.4k if it reads all its reference files. Until then it costs about 64 tokens; SKILL.md has 224 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it

~64

When it runs · the whole SKILL.md, loaded when a task matches

~1.1k

With references · SKILL.md plus every file in references/, read only if the agent opens them

~6.4k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.

SKILL.md

The full file from Cedriccmh/claude-code-skill-scrapling at commit 85dd89f, republished under its MIT licence (© Cedriccmh). 224 words, ~1,096 tokens.

name

scrapling

description

使用 scrapling 进行网页抓取和数据提取。根据目标网站特征自动选择最佳 Fetcher， 生成并执行 Python 脚本完成任务。Use when: (1) 抓取/爬取网页内容或数据（scrape, crawl, fetch page, extract data） (2) 需要绕过 Cloudflare/WAF 等反爬保护 (3) 登录后抓取受保护页面 (4) 解析已有 HTML 提取结构化数据 (5) 用户提供 URL 并要求获取页面内容或特定元素 (6) 批量采集多个页面

allowed-tools

Bash(python*), Bash(pip*), Bash(uv*), Bash(scrapling*)

python -c "import scrapling; print(scrapling.__version__)"

scrapling extract get "https://example.com/article" article.md --ai-targeted
scrapling extract fetch "https://example.com/app" app.md --ai-targeted --network-idle
scrapling extract stealthy-fetch "https://protected.example.com" page.md --ai-targeted --solve-cloudflare

目标网站 →
│
├─ 已有 HTML 字符串/文件，只需解析?
│   → Selector（纯解析，无网络请求）
│   → 模板: parse_only.py
│
├─ 静态页面，无 JS 渲染，无反爬?
│   → Fetcher（最快，基于 curl_cffi）
│   → 模板: basic_fetch.py
│
├─ 需要登录（HTTP 表单，非 JS 登录）?
│   → FetcherSession（保持会话 cookie）
│   → 模板: session_login.py
│
├─ 有 Cloudflare / WAF 保护?
│   → StealthyFetcher（Camoufox 浏览器，自动过 CF）
│   → 模板: stealth_cloudflare.py
│
├─ SPA 应用（React/Vue），需要 JS 渲染?
│   → DynamicFetcher（Playwright 浏览器）
│   → 基于模板即时生成
│
└─ 不确定?
    → 先用 Fetcher 试，403/空内容 → 升级到 StealthyFetcher

1. 检查版本（步骤 0）
2. 安全预检（步骤 1）
3. 查阅 references/site-patterns.md；如果存在 references/site-patterns.local.md，也先查本地 overlay
4. 简单抽取 → 优先 CLI quick path + --ai-targeted
5. 复杂流程 → 用决策树选择 Fetcher，读取对应模板，替换参数，生成完整脚本
6. 执行脚本 / CLI → 返回最小必要结果
7. **沉淀经验（必做）**:
   - 新通用站点类型 → 去敏感化后追加到 references/site-patterns.md
   - 私有站点 / 公司站点 / 登录态细节 → 追加到 references/site-patterns.local.md（不提交）
   - 用户明确授权保存真实 cookie → 保存到 references/cookie-vault.local.md（不提交，必须 redact 输出）
   - **完成抓取后必须检查**：是否有新的 cookie 或 site pattern 需要保存，以及是否属于 local overlay

© Cedriccmh, MIT. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file

Files

SKILL.md and 15 other files (references) in the repository root of Cedriccmh/claude-code-skill-scrapling.

• SKILL.md
• .gitignore
• LICENSE
• README.md
• references/api-quick-ref.md
• references/cookie-vault.md
• references/maintenance.md
• references/security.md
• references/site-patterns.md
• references/troubleshooting.md
• references/upstream-map.md
• templates/basic_fetch.py
• templates/parse_only.py
• templates/session_login.py
• templates/stealth_cloudflare.py
• tests/test_pr1_pr3_content.py

Open the folder on GitHubat commit 85dd89f