Agent skill

Funboost Spider Crawling

by ydf0509 in ydf0509/funboost

当需要用 funboost 进行 Web 爬虫或数据采集时使用。触发场景:分布式爬虫、URL 调度、funspider、boostspider、httpx 爬虫。关键词:爬虫, spider, crawl, funspider, SimpleSpiderClient, AsyncSpiderClient, URL 调度, 分布式采集。

No licenceAuto-check passedDatabases

Install Funboost Spider Crawling

skills CLI
$ npx skills add ydf0509/funboost --skill funboost-spider-crawling -a claude-code

Project install by default; add -g for ~/.claude/skills/.

GitHub CLI
$ gh skill install ydf0509/funboost funboost-spider-crawling --agent claude-code

Project scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).

Manual copy
$ git clone --depth 1 https://github.com/ydf0509/funboost.git skills-src && mkdir -p .claude/skills && cp -r skills-src/.agents/skills/funboost-spider-crawling .claude/skills/funboost-spider-crawling && rm -rf skills-src

Use ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.

Claude Code skills documentation · loads skills from .claude/skills/

Facts

Skill name
funboost-spider-crawling
GitHub stars
893
Token cost
~3.7k tokens
SKILL.md length
486 words
Files
1
Skills in repo
19
Repo updated
First seen
Licence
None found

At a glance

当需要用 funboost 进行 Web 爬虫或数据采集时使用。触发场景:分布式爬虫、URL 调度、funspider、boostspider、httpx 爬虫。关键词:爬虫, spider, crawl, funspider, SimpleSpiderClient, AsyncSpiderClient, URL 调度, 分布式采集。

  • Works in 6 steps: 必须使用 BoosterParams 对象 — 禁止… → 获取上下文用 fct — 不用 self / bind=True → 异步模式 —… → …
  • Databases work in your project
  • SKILL.md covers 概述, 适用场景, 三种爬虫模式 and funspider 客户端用法, plus 11 more sections
  • Calls pip and python; reaches proxy.abuyun.com

What it does

Funboost Spider Crawling is an agent skill from ydf0509/funboost. 当需要用 funboost 进行 Web 爬虫或数据采集时使用。触发场景:分布式爬虫、URL 调度、funspider、boostspider、httpx 爬虫。关键词:爬虫, spider, crawl, funspider, SimpleSpiderClient, AsyncSpiderClient, URL 调度, 分布式采集。

Its SKILL.md is about 3.7k tokens, which your agent loads only when the skill is triggered. It is a single SKILL.md file with no bundled scripts.

It sits in Databases. The repository describes itself as: pip install funboost,python全功能分布式函数调度框架,funboost的功能是全面性重量级,用户能想得到的功能99%全都有;funboost的使用方式是轻量级,只有@boost一行代码需要写。支持python所有类型的并发模式和一切知名消息队列中间件,支持如 celery…

When your agent uses it

  • Databases work in your project

Example prompts

  • “/funboost-spider-crawling”

Requirements

  • Python 3

Workflow steps

6 steps, taken from the first numbered list in SKILL.md.

  1. 必须使用 BoosterParams 对象 — 禁止 @boost("queue_name", qps=5) 老式写法
  2. 获取上下文用 fct — 不用 self / bind=True
  3. 异步模式 — concurrent_mode=ConcurrentModeEnum.ASYNC 时必须 async def + await aio_push
  4. 连续启动消费 — func1.consume(); func2.consume(),不要用 threading.Thread 包装
  5. AI 运行脚本 — 必须设 PYTHONPATH=项目根目录,并用 timeout 或 os._exit() 终止(consume() 永不退出)
  6. 禁止 flush Redis — 去重依赖 Redis,不要清空

What it can do on your machine

Read from SKILL.md and the folder at commit 2b866e5. It shows what the files ask for, not the result of running them.

  • Tool permissions

    Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.

    From allowed-tools in the SKILL.md frontmatter.

  • Runs code

    Shell commands in SKILL.md call:

    • pip
    • python

    From the folder's file list and the shell code blocks in SKILL.md.

  • Network

    Hosts in commands or code, which the agent is likely to contact:

    • proxy.abuyun.com

    From URLs in SKILL.md, links to its own repository left out.

  • Credentials

    Names no API keys, tokens, secrets or passwords.

    From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

Context cost

Funboost Spider Crawling loads about 3.7k tokens when it runs. Until then it costs about 49 tokens; SKILL.md has 486 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it
~49
When it runs · the whole SKILL.md, loaded when a task matches
~3.7k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Auto-check passed

The automated check found no risky patterns in SKILL.md.

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.

SKILL.md

Without a licence we can't republish the file, so here is its outline and opening line. It has 486 words (~3,668 tokens).

name
funboost-spider-crawling

Read the full SKILL.md on GitHub

Files

Just SKILL.md in .agents/skills/funboost-spider-crawling of ydf0509/funboost.

Open the folder on GitHubat commit 2b866e5

Compare with similar skills

Funboost Spider Crawling next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.

Funboost Spider Crawling compared with similar skills
SkillStarsUsed inTokensAuto-checkLicenceRepo updated
Funboost Spider Crawling this skillydf0509/funboost893—~3.7kAutomated safety check: PassNone
Keeper Stress AnalysisClickHouse/ClickHouse50k—~4.7kAutomated safety check: PassApache-2.0
Perf ComparisonClickHouse/ClickHouse50k—~3.9kAutomated safety check: NotesApache-2.0
Patch Release CheckClickHouse/ClickHouse50k—~4kAutomated safety check: NotesApache-2.0
Evolving The Data ModelTriliumNext/Trilium38k—~2.1kAutomated safety check: PassAGPL-3.0
Hybrid Cloud Outboxesgetsentry/sentry46k—~4.8kAutomated safety check: PassCustom licence

Similar skills

  • Keeper Stress Analysis

    ClickHouse/ClickHouse

    Analyze ClickHouse Keeper stress-test results from play.clickhouse.com / keeperstresstests data warehouse.

    50k GitHub stars~4.7k tokensUpdated today
    DatabasesAuto-check passed
  • Perf Comparison

    ClickHouse/ClickHouse

    Evaluate ClickHouse performance test results from existing CI/dashboard data or local perf.py runs.

    50k GitHub stars~3.9k tokensUpdated today
    DatabasesAuto-check: notes
  • Patch Release Check

    ClickHouse/ClickHouse

    Check whether ClickHouse's supported versions (last 3 majors + latest LTS) have recent stable patch releases, diagnose why the scheduled AutoReleases pipeline failed, and identify which releases…

    50k GitHub stars~4k tokensUpdated today
    DatabasesAuto-check: notes
  • Evolving The Data Model

    TriliumNext/Trilium

    A skill your agent uses when adding a DB migration or a new column/field to a Becca entity in Trilium ("add a migration", "new column on notes/attributes", "ALTER TABLE", "add a field to…

    38k GitHub stars~2.1k tokensUpdated today
    DatabasesAuto-check passed
  • Hybrid Cloud Outboxes

    getsentry/sentry

    Official

    Guide for creating and maintaining outbox-based eventually consistent operations in Sentry.

    46k GitHub stars~4.8k tokensUpdated today
    DatabasesAuto-check passed
  • Adaptive Spin Wait

    microsoft/garnet

    Official

    Selects Garnet spin-wait policies based on the slow-path cost.

    12k GitHub stars~1k tokensUpdated today
    DatabasesAuto-check passed

More from ydf0509/funboost

All 19 skills in this repo
  • 当需要在 funboost 中连续启动多个函数消费(多个消费者同时运行)时使用。触发场景:一个脚本里同时跑多个 @boost 任务函数的消费、为不同队列启动多个消费者。关键词:连续启动, 多个消费者, consume, multi consume, 同时消费, 启动多个, 多个队列消费。

    893 GitHub stars~502 tokensUpdated 1 mo ago
    Auto-check passed
  • 当需要为 funboost 框架添加新的消息中间件时使用。触发场景:实现自定义 Publisher/Consumer 类、使用 registercustombroker 注册新 broker、使用 overridecls 定制现有 broker 行为。关键词:new broker, AbstractPublisher, AbstractConsumer…

    893 GitHub stars~1.7k tokensUpdated 1 mo ago
    Auto-check passed
  • 当需要为 funboost 创建 Consumer 或 Publisher 的 Mixin 扩展类时使用。触发场景:添加监控、熔断、限流、链路追踪等横切关注点,编写自定义前置/后置处理钩子。关键词:mixin, consumeroverridecls, publisheroverridecls, ConsumerMixin, 自定义消费者, hook, 拦截器, 熔断器, 监控…

    893 GitHub stars~2.1k tokensUpdated 1 mo ago
    Auto-check passed
  • 当需要为 funboost 框架编写测试、运行测试脚本、或验证新 broker/mixin 实现时使用。触发场景:编写测试脚本、验证消费者/发布者功能、AI Agent 运行 funboost 脚本。关键词:test funboost, 写测试, 运行测试, testframe, regression test, 验证消费, AI testing, os.exit, timeout。

    893 GitHub stars~1.5k tokensUpdated 1 mo ago
    Auto-check passed
  • Funboost Advanced Retry

    ydf0509/funboost

    当需要配置 funboost 的高级重试策略时使用。触发场景:指数退避重试、死信队列、熔断器、任务去重过滤、自定义错误处理。关键词:retry, maxretrytimes, dead letter, DLX, exponential backoff, circuit breaker, isusingadvancedretry…

    893 GitHub stars~1.6k tokensUpdated 1 mo ago
    Auto-check passed
  • 当需要在 funboost 中使用异步(async/await)编程时使用。触发场景:async def 消费函数、aiopush/aiopublish、concurrentmode=ASYNC、event loop 报错。关键词:async, await, asyncio, aiopush, aiopublish, ConcurrentModeEnum.ASYNC, 异步消费, 协程。

    893 GitHub stars~3.4k tokensUpdated 1 mo ago
    Auto-check passed

Categories

Questions about Funboost Spider Crawling

What does Funboost Spider Crawling do?

当需要用 funboost 进行 Web 爬虫或数据采集时使用。触发场景:分布式爬虫、URL 调度、funspider、boostspider、httpx 爬虫。关键词:爬虫, spider, crawl, funspider, SimpleSpiderClient, AsyncSpiderClient, URL 调度, 分布式采集。. Funboost Spider Crawling is an agent skill from ydf0509/funboost.

When should I use Funboost Spider Crawling?

Funboost Spider Crawling fits situations like: databases work in your project.

How do I install Funboost Spider Crawling in Claude Code?

Run `npx skills add ydf0509/funboost --skill funboost-spider-crawling -a claude-code`. Or copy the skill folder (.agents/skills/funboost-spider-crawling in ydf0509/funboost) into .claude/skills/funboost-spider-crawling in your project. Claude Code loads it when a task matches its description.

How do I install Funboost Spider Crawling in Codex?

Run `npx skills add ydf0509/funboost --skill funboost-spider-crawling -a codex`. Or copy the skill folder (.agents/skills/funboost-spider-crawling in ydf0509/funboost) into .agents/skills/funboost-spider-crawling in your project. Codex loads it when a task matches its description.

Can I use Funboost Spider Crawling in Cursor, Gemini CLI or GitHub Copilot?

Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add ydf0509/funboost --skill funboost-spider-crawling -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/funboost-spider-crawling, .gemini/skills/funboost-spider-crawling, .github/skills/funboost-spider-crawling and .opencode/skills/funboost-spider-crawling in your project.

What does Funboost Spider Crawling need to run?

Going by SKILL.md and its folder, Funboost Spider Crawling needs the command-line tools its instructions call (pip and python). Our summary lists: Python 3.

Does Funboost Spider Crawling access the network?

SKILL.md names 1 domain. In commands or code: proxy.abuyun.com; the agent is likely to contact it when it follows the instructions. This is read from the text; nothing was executed.

Is Funboost Spider Crawling safe to install?

Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. Review the folder before installing.

What licence does Funboost Spider Crawling use?

No licence was found for Funboost Spider Crawling or its repository. Without one, default copyright applies: ask the author before reusing or redistributing it.

How many tokens does Funboost Spider Crawling use?

About 3.7k tokens (SKILL.md is roughly 15k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full.

What are the alternatives to Funboost Spider Crawling?

Skills that share tags, products or a category with Funboost Spider Crawling: Keeper Stress Analysis (ClickHouse/ClickHouse, 50k stars), Perf Comparison (ClickHouse/ClickHouse, 50k stars), Patch Release Check (ClickHouse/ClickHouse, 50k stars) and Evolving The Data Model (TriliumNext/Trilium, 38k stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.

Who maintains Funboost Spider Crawling?

ydf0509 (a GitHub user) maintains it in ydf0509/funboost, which has 893 GitHub stars. The repository holds 19 skills in this directory. The repository was last updated on August 11, 2026.

Source: ydf0509/funboost on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.