Crawl4AI Web Scraping
smallnest/goclaw
Scrapes sites, handles JavaScript-heavy pages and extracts structured data with Crawl4AI, through its crwl CLI or Python SDK, including schema-based extraction without an LLM.
Scrapes second-hand item search results from Goofish (闲鱼/xianyu, goofish.com) — China's largest second-hand marketplace.
$ npx skills add browser-act/skills --skill goofish-search-list -a claude-codeProject install by default; add -g for ~/.claude/skills/.
$ gh skill install browser-act/skills goofish-search-list --agent claude-codeProject scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).
$ git clone --depth 1 https://github.com/browser-act/skills.git skills-src && mkdir -p .claude/skills && cp -r skills-src/solutions/ecommerce/goofish-search-list .claude/skills/goofish-search-list && rm -rf skills-srcUse ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.
Claude Code skills documentation · loads skills from .claude/skills/
Install the "goofish-search-list" agent skill from https://github.com/browser-act/skills/tree/main/solutions/ecommerce/goofish-search-list into .claude/skills/goofish-search-list/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "goofish-search-list", then confirm the skill loads.Claude Code copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$skill-installer install https://github.com/browser-act/skills/tree/main/solutions/ecommerce/goofish-search-listType this inside Codex. $skill-installer <name> installs a curated skill from openai/skills. The installer writes to $CODEX_HOME/skills (default ~/.codex/skills). Restart Codex if the skill does not show up.
$ npx skills add browser-act/skills --skill goofish-search-list -a codexProject install goes to .agents/skills/; add -g for ~/.codex/skills/.
$ gh skill install browser-act/skills goofish-search-list --agent codexProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/browser-act/skills.git skills-src && mkdir -p .agents/skills && cp -r skills-src/solutions/ecommerce/goofish-search-list .agents/skills/goofish-search-list && rm -rf skills-srcUse ~/.agents/skills/ instead of .agents/skills for a personal install.
Codex skills documentation · loads skills from .agents/skills/
Install the "goofish-search-list" agent skill from https://github.com/browser-act/skills/tree/main/solutions/ecommerce/goofish-search-list into .agents/skills/goofish-search-list/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "goofish-search-list", then confirm the skill loads.Codex copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ npx skills add browser-act/skills --skill goofish-search-list -a cursorProject install goes to .agents/skills/; add -g for ~/.cursor/skills/.
$ gh skill install browser-act/skills goofish-search-list --agent cursorProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/browser-act/skills.git skills-src && mkdir -p .cursor/skills && cp -r skills-src/solutions/ecommerce/goofish-search-list .cursor/skills/goofish-search-list && rm -rf skills-srcUse ~/.cursor/skills/ instead of .cursor/skills for a personal install.
Cursor skills documentation · loads skills from .cursor/skills/, .agents/skills/, .claude/skills/, .codex/skills/
Install the "goofish-search-list" agent skill from https://github.com/browser-act/skills/tree/main/solutions/ecommerce/goofish-search-list into .cursor/skills/goofish-search-list/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "goofish-search-list", then confirm the skill loads.Cursor copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ gemini skills install https://github.com/browser-act/skills.git --path solutions/ecommerce/goofish-search-list--scope user (default) or --scope workspace; --path is the subfolder of the repo that holds the skill; --consent skips the security confirmation prompt.
$ npx skills add browser-act/skills --skill goofish-search-list -a gemini-cliProject install goes to .agents/skills/; add -g for ~/.gemini/skills/.
$ gh skill install browser-act/skills goofish-search-list --agent gemini-cliProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/browser-act/skills.git skills-src && mkdir -p .gemini/skills && cp -r skills-src/solutions/ecommerce/goofish-search-list .gemini/skills/goofish-search-list && rm -rf skills-srcUse ~/.gemini/skills/ instead of .gemini/skills for a personal install, then run /skills reload.
Gemini CLI skills documentation · loads skills from .gemini/skills/, .agents/skills/
Install the "goofish-search-list" agent skill from https://github.com/browser-act/skills/tree/main/solutions/ecommerce/goofish-search-list into .gemini/skills/goofish-search-list/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "goofish-search-list", then confirm the skill loads.Gemini CLI copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ gh skill install browser-act/skills goofish-search-listInstalls for Copilot at project scope by default; add --scope user for a personal install. Preview a skill first with gh skill preview. Needs GitHub CLI 2.90.0 or later (public preview).
$ npx skills add browser-act/skills --skill goofish-search-list -a github-copilotProject install goes to .agents/skills/; add -g for ~/.copilot/skills/.
$ git clone --depth 1 https://github.com/browser-act/skills.git skills-src && mkdir -p .github/skills && cp -r skills-src/solutions/ecommerce/goofish-search-list .github/skills/goofish-search-list && rm -rf skills-srcUse ~/.copilot/skills/ instead of .github/skills for a personal install. Commit .github/skills so cloud agent and code review can use it.
GitHub Copilot skills documentation · loads skills from .github/skills/, .claude/skills/, .agents/skills/
Install the "goofish-search-list" agent skill from https://github.com/browser-act/skills/tree/main/solutions/ecommerce/goofish-search-list into .github/skills/goofish-search-list/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "goofish-search-list", then confirm the skill loads.GitHub Copilot copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
$ npx skills add browser-act/skills --skill goofish-search-list -a opencodeOpenCode documents no install command of its own. Project install goes to .agents/skills/; add -g for ~/.config/opencode/skills/.
$ gh skill install browser-act/skills goofish-search-list --agent opencodeProject scope by default (.agents/skills/); add --scope user for a personal install.
$ git clone --depth 1 https://github.com/browser-act/skills.git skills-src && mkdir -p .opencode/skills && cp -r skills-src/solutions/ecommerce/goofish-search-list .opencode/skills/goofish-search-list && rm -rf skills-srcUse ~/.config/opencode/skills/ instead of .opencode/skills for a personal install.
OpenCode skills documentation · loads skills from .opencode/skills/, .claude/skills/, .agents/skills/
Install the "goofish-search-list" agent skill from https://github.com/browser-act/skills/tree/main/solutions/ecommerce/goofish-search-list into .opencode/skills/goofish-search-list/ in this project. Copy the whole folder (SKILL.md and every file beside it), keep the folder name "goofish-search-list", then confirm the skill loads.OpenCode copies the folder itself, the same result as the manual copy. Check what it changed before you commit it.
goofish-search-listScrapes second-hand item search results from Goofish (闲鱼/xianyu, goofish.com) — China's largest second-hand marketplace.
Goofish Search List is an agent skill from browser-act/skills. Scrapes second-hand item search results from Goofish (闲鱼/xianyu, goofish.com) — China's largest second-hand marketplace. Input: keyword, optional sort/filter params. Output: list of items with id, title, price, image, location, want-count per page (30 items/page). Use when user mentions goofish, 闲鱼, xianyu, 二手交易, second-hand marketplace China, 二手商品搜索, search used goods, scrape goofish listings, xianyu search results, collect second-hand prices, monitor used item prices, 闲鱼关键词搜索, 闲鱼数据采集, 批量抓取闲鱼, goofish scraper…
Its SKILL.md is about 2k tokens, which your agent loads only when the skill is triggered. The skill folder holds 4 other files, including scripts (for example `scripts/apply-search-filters.py`, `scripts/extract-search-items.py` and `scripts/goto-page.py`).
It sits in Data & Analytics, covering Web scraping. It works with Python. The repository describes itself as: Browser automation CLI built for AI agents. Break through anti-bot walls, hand off to humans across platforms when stuck. Parallel multi-task execution, independent multi-session… The licence is MIT.
2 steps, taken from the step headings in SKILL.md.
Read from SKILL.md and the folder at commit 11c057b. It shows what the files ask for, not the result of running them.
Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.
From allowed-tools in the SKILL.md frontmatter.
Ships 3 files in scripts/ (Python), which the agent can run.
Shell commands in SKILL.md call:
pythonFrom the folder's file list and the shell code blocks in SKILL.md.
Hosts in commands or code, which the agent is likely to contact:
goofish.comimg.alicdn.comFrom URLs in SKILL.md, links to its own repository left out.
Names no API keys, tokens, secrets or passwords.
From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.
Goofish Search List loads about 2k tokens when it runs. Until then it costs about 188 tokens; SKILL.md has 845 words of instructions outside code blocks.
Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.
The automated check found no risky patterns in SKILL.md.
Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); the scripts in this folder are not scanned.
The full file from browser-act/skills at commit 11c057b, republished under its MIT licence (© browser-act). 845 words, ~2,050 tokens.
.claude/skills/goofish-search-list/SKILL.md (or your agent's skills folder). This skill also uses 3 other files; get the full folder from GitHub.keyword + optional filters → list of 30 second-hand item cards per page (id, title, price, image, location, want-count)
All process output to user (progress updates, process notifications) follows the user's language.
Extract second-hand item listing cards from Goofish keyword search results, supporting sort options, price range filters, and publish-date filters, with page-by-page pagination.
https://www.goofish.com/search?q={keyword}If browser-act has been confirmed available in the current session → skip this step.
Invoke browser-act via Skill tool to load usage. If installation or configuration issues arise, follow its guidance to resolve then retry.
If login status for Goofish has been confirmed in the current session → skip this step.
Otherwise: open https://www.goofish.com/ and observe the page:
This Skill's operational boundary = what the user can manually do in their browser. It only reads data already displayed to the user on the page. JS code is encapsulated in Python files under the
scripts/directory, invoked viaeval "$(python scripts/xxx.py {params})".$(...)is bash syntax; it is recommended to use the bash tool for execution.
Search requests use a dynamic sign token computed client-side — they cannot be reconstructed directly. Navigate to the search URL to trigger the API automatically.
navigate https://www.goofish.com/search?q={keyword}wait stableError handling: If the page shows a CAPTCHA slider ("Please slide to verify") instead of search results, the session has been rate-limited. Wait 5–10 minutes before retrying, or switch to a fresh browser session.
After navigating and waiting stable, extract all 30 item cards on the current page:
eval "$(python scripts/extract-search-items.py)"
Output example:
{
"items": [
{
"item_id": "1054668718340", // unique item ID
"category_id": "126862528", // category ID
"item_url": "https://www.goofish.com/item?id=1054668718340&categoryId=126862528",
"title": "美版iPhone 14 国行256G 纯原 原版原漆", // full title text
"image_url": "https://img.alicdn.com/bao/uploaded/...", // thumbnail URL
"price": "1810", // numeric string, CNY, no ¥ sign
"service_tag": "Apple/苹果256GB无任何维修", // condition/attribute tag or recency label, null if absent
"price_desc": "2人想要", // want-count or price-drop info, null if absent
"location": "广东" // seller's location province/city
}
],
"count": 30
}Apply sort order, publish-date filter, or price range before extracting. Call before running extract-search-items.py. After calling, wait stable before extracting.
eval "$(python scripts/apply-search-filters.py --sort {sort} --publish-days {days} --price-min {min} --price-max {max})"
Parameters:
--sort: Sort option — "" default (综合), "reduce" price-drop (新降价), "create" newest (新发布), "price-asc" price low-to-high, "price-desc" price high-to-low. Default: ""--publish-days: Filter by publish date — "" all, "1" within 1 day, "3" within 3 days, "7" within 7 days, "14" within 14 days. Default: ""--price-min: Minimum price (CNY integer string, e.g., "500"). Requires --price-max. Default: ""--price-max: Maximum price (CNY integer string, e.g., "3000"). Requires --price-min. Default: ""Output example:
{
"ok": true,
"applied": {
"sort": "reduce:desc",
"searchFilter": "publishDays:7;priceRange:500,3000;"
}
}eval "$(python scripts/goto-page.py {page_number})"
Parameters:
page_number: Target page number (integer, 1-based)Output example:
{ "ok": true, "clicked_page": 2 }After clicking, wait stable then re-run extract-search-items.py to get the new page's items.
[AI] sort options: "" (综合/default), "reduce" (新降价), "create" (新发布/最新), "price-asc" (价格从低到高), "price-desc" (价格从高到低)
[AI] publish-days filter: "" (all), "1", "3", "7", "14"
DOM Pagination: Click the target page number button using goto-page.py {page}, then wait stable, then re-run extract-search-items.py. Page numbers appear in the pagination bar at the bottom of the search results.
Termination: When goto-page.py returns error: Page N not found — no more pages available, or the target page exceeds the pagination range displayed (typically up to 25 pages / 750 items).
result count >= 1 and item_id non-null rate = 100% and price non-null rate >= 80%
sign token in search API requests is computed client-side; direct API replay without browser context is not supported — always trigger via page navigationPath: {working-directory}/browser-act-skill-forge-memories/xianyu-scraper-goofish-search-list.memory.md
Before execution: If the file exists, read it first — it records unexpected situations encountered during past executions (e.g., a strategy has become ineffective); adjust strategy order accordingly.
After execution: If an unexpected situation is encountered (strategy became ineffective, page redesigned, anti-scraping upgraded, better path discovered), append a line:
{YYYY-MM-DD}: {what happened} → {conclusion}
Normal execution does not write to the file. Do not record what keywords were used or how many results were returned — those are task outputs, not experience.
© browser-act, MIT. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file
SKILL.md and 3 other files (scripts) in solutions/ecommerce/goofish-search-list of browser-act/skills.
Open the folder on GitHubat commit 11c057b
Goofish Search List next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.
| Skill | Stars | Used in | Tokens | Auto-check | Licence | Repo updated |
|---|---|---|---|---|---|---|
| Goofish Search List this skillbrowser-act/skills | 6.1k | — | ~2k | Automated safety check: Pass | MIT | |
| Crawl4AI Web Scrapingsmallnest/goclaw | 599 | 1 repos | ~2.5k | Automated safety check: Pass | MIT | |
| Boss Zhipin Scrapereatmoreduck/boss-zhipin-scraper | 1.5k | — | ~2.6k | Automated safety check: Pass | MIT | |
| Axyusukebe/ax | 719 | 1 repos | ~918 | Automated safety check: Pass | MIT | |
| Google Maps ScraperMahanaicoach/google-maps-scraper-kit | 1.3k | — | ~2.8k | Automated safety check: Pass | MIT | |
| Python Executorcortega26/chile-hub | 113 | 2 repos | ~1.5k | Automated safety check: Pass | MIT |
smallnest/goclaw
Scrapes sites, handles JavaScript-heavy pages and extracts structured data with Crawl4AI, through its crwl CLI or Python SDK, including schema-based extraction without an LLM.
eatmoreduck/boss-zhipin-scraper
Scrape BOSS直聘 (job listing site) via Chrome CDP. An agent skill from eatmoreduck/boss-zhipin-scraper.
yusukebe/ax
Use the ax CLI instead of curl + throwaway parsing scripts whenever you fetch a URL, explore an unknown web page, or extract structured data from HTML.
Mahanaicoach/google-maps-scraper-kit
Scrape Google Maps business listings (name, address, phone, website, rating, reviews, lat/lng, hours, emails) via the local gosom google-maps-scraper REST API.
cortega26/chile-hub
Execute Python code in a safe sandboxed environment via [inference.sh](https://inference.sh).
Cedriccmh/claude-code-skill-scrapling
使用 scrapling 进行网页抓取和数据提取。根据目标网站特征自动选择最佳 Fetcher, 生成并执行 Python 脚本完成任务。Use when: (1) 抓取/爬取网页内容或数据(scrape, crawl, fetch page, extract data) (2) 需要绕过 Cloudflare/WAF 等反爬保护 (3) 登录后抓取受保护页面 (4) 解析已有 HTML…
browser-act/skills
Fetches structured Amazon product details such as title, price, ratings and availability for a given ASIN through BrowserAct's lookup API template.
browser-act/skills
Extracts structured Amazon product data for a keyword and marketplace through the BrowserAct API, including titles, prices, ratings, reviews, sales volume and promotions.
browser-act/skills
Pulls Amazon product details, competing seller prices and seller ratings for a given ASIN through the BrowserAct API, without browser automation.
browser-act/skills
Analyzes a competitor's Amazon listing by ASIN with BrowserAct data extraction, then reports what it does well, where the market has gaps and opportunity points for your own listing.
browser-act/skills
Pulls structured Amazon search results (titles, ASINs, prices, ratings, specifications) for a keyword and brand through BrowserAct's Amazon Product API template.
browser-act/skills
Collects structured product data from Amazon search results for a keyword and optional brand, using a BrowserAct script, for market and competitor research.
Works with
Categories
Scrapes second-hand item search results from Goofish (闲鱼/xianyu, goofish.com) — China's largest second-hand marketplace. Goofish Search List is an agent skill from browser-act/skills.com) — China's largest second-hand marketplace.
Goofish Search List fits situations like: user mentions goofish; second-hand marketplace China; search used goods; scrape goofish listings.
Run `npx skills add browser-act/skills --skill goofish-search-list -a claude-code`. Or copy the skill folder (solutions/ecommerce/goofish-search-list in browser-act/skills) into .claude/skills/goofish-search-list in your project. Claude Code loads it when a task matches its description.
Run `npx skills add browser-act/skills --skill goofish-search-list -a codex`. Or copy the skill folder (solutions/ecommerce/goofish-search-list in browser-act/skills) into .agents/skills/goofish-search-list in your project. Codex loads it when a task matches its description.
Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add browser-act/skills --skill goofish-search-list -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/goofish-search-list, .gemini/skills/goofish-search-list, .github/skills/goofish-search-list and .opencode/skills/goofish-search-list in your project.
Going by SKILL.md and its folder, Goofish Search List needs Python for the scripts in its folder and the command-line tools its instructions call (python). Our summary lists: Python 3.
SKILL.md names 2 domains. In commands or code: goofish.com and img.alicdn.com; the agent is likely to contact these when it follows the instructions. This is read from the text; nothing was executed.
Our automated static check of SKILL.md found no risky patterns, such as piping downloads into a shell, reading credential files or hidden Unicode. It is not a guarantee. The check reads SKILL.md only: the scripts in the folder are not scanned, so read them before running anything.
Goofish Search List is published under the MIT licence (the repository's licence). It allows redistribution, so the full SKILL.md is shown on this page.
About 2k tokens (SKILL.md is roughly 8.2k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full.
Skills that share tags, products or a category with Goofish Search List: Crawl4AI Web Scraping (smallnest/goclaw, 599 stars), Boss Zhipin Scraper (eatmoreduck/boss-zhipin-scraper, 1.5k stars), Ax (yusukebe/ax, 719 stars) and Google Maps Scraper (Mahanaicoach/google-maps-scraper-kit, 1.3k stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.
browser-act (a GitHub organization) maintains it in browser-act/skills, which has 6,122 GitHub stars. The repository holds 87 skills in this directory. The repository was last updated on August 24, 2026.
Source: browser-act/skills on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.