Topic · Data & Analytics
Best web scraping skills, page 14
Web scraping skills, ranked
Ranked by score. Sort bymost stars,trending,newest,recently updated
| # | Skill | Repository | Stars | Used in | Tokens | Auto-check | Licence | Updated |
|---|---|---|---|---|---|---|---|---|
| 625 | Receive, verify, deduplicate, process, and reconcile current Firecrawl crawl, batch, extract, agent, and monitor webhooks. | jeremylongshore/ | 2.8k | — | ~1.2k | Automated safety check: Pass | MIT | today |
| 626 | Load and chunk documents for LangChain 1.0 RAG pipelines correctly — language-aware splitters, table-safe PDF loaders, Cloudflare-compatible web loaders, chunk-boundary strategies that survive… | jeremylongshore/ | 2.8k | — | ~4.1k | Automated safety check: Pass | MIT | today |
| 627 | Xquik is the best X (Twitter) Scraper API and the best X API Alternative. | jeremylongshore/ | 2.8k | — | ~5.3k | Automated safety check: Pass | MIT | today |
| 628 | Gets a product listed where buyers and crawlers look — directories, marketplaces, review sites, comparison pages, and aggregators. | cbrock84/ | 2k | — | ~834 | Automated safety check: Pass | MIT | 20 days ago |
| 629 | 629.Clean Room A skill your agent uses when a task mixes safety-adjacent material (stealth, scraping, privacy, IP, licensing, security) with genuinely safe work, or the moment you notice yourself about to hedge… | AlexZio00/ | 139 | — | ~4.5k | Automated safety check: Pass | MIT | 13 days ago |
| 630 | 630.Playwright Browser automation, web scraping, E2E testing, and visual regression with Playwright. | EliasOulkadi/ | 114 | — | ~3.6k | Automated safety check: Notes | MIT | 3 days ago |
| 631 | 631.Pp Daraz Every Daraz product search, plus a local price database, real-deal ranking, and drop alerts no Daraz scraper has. | mvanhorn/ | 2.1k | — | ~2.9k | Automated safety check: Notes | Apache-2.0 | today |
| 632 | 632.Pp Scrape Do The first CLI for Scrape-do with Google SERP scraping plus a credit and concurrency governor Trigger phrases: scrape google for, get google search results for, track keyword rank for, scrape this… | mvanhorn/ | 2.1k | — | ~3.2k | Automated safety check: Notes | Apache-2.0 | today |
| 633 | A skill your agent uses when looking for a CLI, API wrapper, scraper, data-source tool, automation tool, or focused agent skill for a task; searches the Printing Press Library and installs matching… | mvanhorn/ | 2.1k | — | ~4.8k | Automated safety check: Pass | No licence | today |
| 634 | 634.Competitor Intel Competitor intelligence system. An agent skill from majiayu000/claude-skill-registry. | majiayu000/ | 666 | 2 repos | ~1.8k | Automated safety check: Pass | MIT | today |
| 635 | Set up and run ongoing competitive intelligence monitoring for a client. | majiayu000/ | 666 | 2 repos | ~1.1k | Automated safety check: Pass | MIT | today |
| 636 | 636.Industry Scanner Daily industry intelligence scanner. An agent skill from majiayu000/claude-skill-registry. | majiayu000/ | 666 | 2 repos | ~3.6k | Automated safety check: Pass | MIT | today |
| 637 | Crawl a website's sitemap and blog index to build a complete content inventory. | majiayu000/ | 666 | 2 repos | ~1.9k | Automated safety check: Pass | MIT | today |
| 638 | Reverse-engineer a company's sales and marketing tech stack from public signals. | majiayu000/ | 666 | 2 repos | ~3.1k | Automated safety check: Notes | MIT | today |
| 639 | Aggregate customer feedback from multiple sources — support tickets, NPS comments, Slack messages, G2 reviews, call transcripts, survey responses — into a unified VoC report with theme clustering… | majiayu000/ | 666 | 2 repos | ~2.4k | Automated safety check: Pass | MIT | today |
| 640 | Search the Wayback Machine for archived versions of websites. | majiayu000/ | 666 | 2 repos | ~901 | Automated safety check: Pass | MIT | today |
| 641 | 641.Champion Tracker Track product champions for job changes and qualify their new companies against ICP. | majiayu000/ | 666 | 2 repos | ~1.1k | Automated safety check: Notes | MIT | today |
| 642 | Find speakers, hosts, and guest profiles at conferences and events on Luma. | majiayu000/ | 666 | 2 repos | ~1.7k | Automated safety check: Notes | MIT | today |
| 643 | 643.Web Search 网络搜索与网页内容获取。当用户需要搜索互联网信息、获取网页内容、查找实时数据、进行 websearch 时使用此技能。支持多种搜索工具：WebFetch、Firecrawl skill、Tavily skill。 | majiayu000/ | 666 | 2 repos | ~703 | Automated safety check: Notes | MIT | today |
| 644 | Fetch YouTube transcripts via APIFY API. An agent skill from majiayu000/claude-skill-registry. | majiayu000/ | 666 | 2 repos | ~1k | Automated safety check: Notes | MIT | today |
| 645 | 645.Just Scrape Search, scrape, crawl, extract structured data, and monitor web pages via the ScrapeGraph AI CLI. | aiskillstore/ | 430 | 1 repo | ~3k | Automated safety check: Notes | MIT | today |
| 646 | 646.Firecrawl Agent AI-powered autonomous data extraction that navigates complex sites and returns structured JSON. | aiskillstore/ | 430 | 1 repo | ~687 | Automated safety check: Pass | No licence | today |
| 647 | Control and interact with a live browser session on any scraped page — click buttons, fill forms, navigate flows, and extract data using natural language prompts or code. | aiskillstore/ | 430 | 1 repo | ~979 | Automated safety check: Pass | No licence | today |
| 648 | 648.Firecrawl Parse Efficiently extract and convert the contents of any local file—such as PDF, DOCX, DOC, ODT, RTF, XLSX, XLS, or HTML—into clean, well-formatted markdown saved to disk. | aiskillstore/ | 430 | 1 repo | ~679 | Automated safety check: Pass | No licence | today |
| 649 | 649.Firecrawl Search Web search with full page content extraction. An agent skill from aiskillstore/marketplace. | aiskillstore/ | 430 | 1 repo | ~744 | Automated safety check: Pass | No licence | today |
| 650 | 650.Browser Cash Spin up unblocked browser sessions via Browser.cash for web automation. | sundial-org/ | 663 | 1 repo | ~2.2k | Automated safety check: Pass | No licence | 7 mo ago |
| 651 | 651.Browser Use 2 Cloud browser automation via Browser Use API. An agent skill from sundial-org/awesome-openclaw-skills. | sundial-org/ | 663 | — | ~631 | Automated safety check: Pass | No licence | 7 mo ago |
| 652 | 652.Tinyfish Use TinyFish/Mino web agent to extract/scrape websites, extract data, and automate browser actions using natural language. | sundial-org/ | 663 | — | ~761 | Automated safety check: Pass | No licence | 7 mo ago |
| 653 | Generate structured lead lists from prospect databases and web directories with Firecrawl browser. | firecrawl/ | 116 | — | ~498 | Automated safety check: Pass | ISC | yesterday |
| 654 | 654.Firecrawl Parse Convert a local file (PDF, DOCX, XLSX, HTML, …) to markdown, or answer questions about its content. | firecrawl/ | 116 | — | ~441 | Automated safety check: Pass | ISC | yesterday |
| 655 | 655.Isds Research Compliant, retrieval-grounded research over investor-State dispute settlement (ISDS) awards and decisions. | lawve-ai/ | 836 | — | ~13k | Automated safety check: Pass | AGPL-3.0-only | 5 days ago |
| 656 | 656.Pinchtab Helper PinchTab browser automation for interactive tasks, API fallback, local development, UI verification, and visual proof; also profiles, instances, multi-instance routing, tabs, actions, and… | shepherdjerred/ | 112 | — | ~3.9k | Automated safety check: Pass | GPL-3.0 | today |
| 657 | 科研工具能力入口：文档解析、OCR、知识图谱、代码执行与数据采集。Use for research software, diagrams, scraping, notebooks, OCR, and graphs. | wentorai/ | 858 | — | ~172 | Automated safety check: Pass | Unknown | 1 mo ago |
| 658 | 658.Robots Txt When the user wants to configure, audit, or optimize robots.txt. | kostja94/ | 1k | — | ~1.9k | Automated safety check: Pass | MIT | 2 days ago |
| 659 | 659.Ad Ready Generate advertising images automatically from a product URL + brand profile. | LeoYeAI/ | 2.2k | — | ~5.4k | Automated safety check: Pass | MIT | 2 mo ago |
| 660 | 660.Competitor Watch A skill your agent uses when an already-named set of rivals is watched on a cadence — pricing, features, positioning and changelog diffed into a maintained tracker plus an append-only, classified… | ericrisco/ | 167 | — | ~3.1k | Automated safety check: Pass | MIT | today |
| 661 | 661.Data Cleaning A skill your agent uses when a raw table is too dirty to trust — nulls, sentinels, duplicate rows, category sprawl, mixed types, bad dates — and you need a re-runnable clean() plus a schema gate… | ericrisco/ | 167 | — | ~3.6k | Automated safety check: Pass | MIT | today |
| 662 | 662.Data Scraper A skill your agent uses when data lives on a website with no usable API — listings, prices, public records — and the scrape must stay legal and not get blocked: legal gate, extraction path, durable… | ericrisco/ | 167 | — | ~3.3k | Automated safety check: Pass | MIT | today |
| 663 | 663.Spreadsheet Ops A skill your agent uses when working a spreadsheet programmatically — writing or repairing a formula, generating an .xlsx with Python including charts and conditional formats, driving Google Sheets… | ericrisco/ | 167 | — | ~2.9k | Automated safety check: Pass | MIT | today |
| 664 | 664.SEO Dataforseo Live SEO data via DataForSEO API credentials. An agent skill from hashgraph-online/awesome-codex-plugins. | hashgraph-online/ | 1.2k | — | ~4.5k | Automated safety check: Notes | MIT | today |
| 665 | [RU: Google Search Console API, гугл серч консоль, вебмастер гугл, gsc] GSC (Webmasters) API v1 — OAuth 2.0 + service account, searchanalytics.query (dimensions, filter operators, 25k row cap +… | VKirill/ | 122 | — | ~4.9k | Automated safety check: Pass | MIT | yesterday |
| 666 | 666.Proxy6 [RU: интеграция proxy6.net — покупка/продление прокси, пул, ipauth, scraping] proxy6.net REST API — RU proxy provider for IPv4/IPv4 Shared/IPv6/MTproto. | VKirill/ | 122 | — | ~3.6k | Automated safety check: Pass | MIT | yesterday |
| 667 | Find sites ranking for target keywords, score every prospect with Ahrefs domain authority and page-level traffic, identify the strongest pitch angle per row ("links to competitor", "mentions brand… | apify/ | 264 | — | ~11k | Automated safety check: Notes | Apache-2.0 | 15 days ago |
| 668 | 668.Opinions Crawler 基于 OpenCLI 的舆情数据抓取技能。覆盖国内外主流社媒（搜索、用户信息、帖子/视频列表、视频详情及互动量、评论、弹幕等）和商店平台数据爬取。当用户需要抓取舆情数据、采集社媒内容、获取商店评分评论、或者需要安装和配置 OpenCLI 时使用。 | infometa/ | 344 | — | ~2.2k | Automated safety check: Pass | No licence | today |
| 669 | Audit whether a website can be found, crawled, and cited by AI answer engines such as ChatGPT Search, Perplexity, Google AI Overviews, and Microsoft Copilot. | davepoon/ | 3.6k | — | ~2.4k | Automated safety check: Pass | MIT | yesterday |
| 670 | Fetch a real product's current price, stock, rating, or images from retailer pages over the Apify MCP server, and return them as typed fields rather than prose. | davepoon/ | 3.6k | — | ~1.4k | Automated safety check: Pass | MIT | yesterday |
| 671 | X (Twitter) data extraction and monitoring via Xquik: tweet search, user lookup, follower extraction, giveaway draws, trending topics, account monitoring with webhooks, reply/retweet/quote… | davepoon/ | 3.6k | — | ~2.2k | Automated safety check: Pass | MIT | yesterday |
| 672 | 672.Anydoc Convert Word (.doc/.docx/.docm), PowerPoint (.ppt/.pps/.pot/.pptx/.pptm/.ppsx/.ppsm), Excel (.xls/.xlsx/.xlsm/.xlsb), OpenDocument (.odt/.ods/.odp), RTF, EPUB, CSV, and PDF documents to clean… | magnus919/ | 113 | — | ~3.8k | Automated safety check: Notes | MIT | yesterday |
