Agent skill

Firecrawl Scraper

by secondsky in secondsky/claude-skills

Firecrawl v2.5 API for web scraping/crawling to LLM-ready markdown.

MITAuto-check: notesData & Analytics

Install Firecrawl Scraper

skills CLI
$ npx skills add secondsky/claude-skills --skill firecrawl-scraper -a claude-code

Project install by default; add -g for ~/.claude/skills/.

GitHub CLI
$ gh skill install secondsky/claude-skills firecrawl-scraper --agent claude-code

Project scope by default; add --scope user for a personal install. Needs GitHub CLI 2.90.0 or later (public preview).

Manual copy
$ git clone --depth 1 https://github.com/secondsky/claude-skills.git skills-src && mkdir -p .claude/skills && cp -r skills-src/plugins/firecrawl-scraper/skills/firecrawl-scraper .claude/skills/firecrawl-scraper && rm -rf skills-src

Use ~/.claude/skills/ instead of .claude/skills for a personal install. The folder must contain SKILL.md.

Claude Code skills documentation · loads skills from .claude/skills/

Facts

Skill name
firecrawl-scraper
GitHub stars
227
Used in
1 other repo
Token cost
~1.9k tokens
SKILL.md length
586 words
Files
7 (incl. references)
Skills in repo
169
Repo updated
First seen
Licence
MIT

At a glance

Firecrawl v2.5 API for web scraping/crawling to LLM-ready markdown.

  • Works in 4 steps: /v2/scrape - Single Page Scraping → /v2/crawl - Full Site Crawling → /v2/map - URL Discovery → …
  • Site extraction
  • SKILL.md covers What is Firecrawl?, API Endpoints, Authentication and SDK Quick Start, plus 9 more sections
  • Runs Python and TypeScript scripts from its folder; calls pip and bun; reaches api.firecrawl.dev; needs FIRECRAWL_API_KEY

What it does

Firecrawl Scraper is an agent skill from secondsky/claude-skills. Firecrawl v2.5 API for web scraping/crawling to LLM-ready markdown. Use for site extraction, dynamic content, or encountering JavaScript rendering, bot detection, content loading errors.

Its SKILL.md is about 1.9k tokens, which your agent loads only when the skill is triggered. The skill folder holds 8 other files, including reference files (for example `references/common-patterns.md`, `references/endpoints.md` and `templates/firecrawl-crawl-example.py`).

It sits in Data & Analytics, covering Web scraping. It works with Firecrawl and JavaScript. The repository describes itself as: Production-ready skills for Claude Code CLI - Cloudflare, React, Tailwind v4, and AI integrations. The licence is MIT.

When your agent uses it

  • Site extraction
  • Dynamic content
  • Encountering JavaScript rendering
  • Content loading errors

Example prompts

  • “/firecrawl-scraper”

Requirements

  • Python 3
  • Node.js
  • A credential in FIRECRAWL_API_KEY

Workflow steps

4 steps, taken from the step headings in SKILL.md.

  1. /v2/scrape - Single Page Scraping
  2. /v2/crawl - Full Site Crawling
  3. /v2/map - URL Discovery
  4. /v2/extract - Structured Data Extraction

What it can do on your machine

Read from SKILL.md and the folder at commit 8837836. It shows what the files ask for, not the result of running them.

  • Tool permissions

    Pre-approves nothing: there is no allowed-tools line, so your agent's usual permission prompts apply.

    From allowed-tools in the SKILL.md frontmatter.

  • Runs code

    Ships script files (Python and TypeScript), which the agent can run.

    Shell commands in SKILL.md call:

    • pip
    • bun

    From the folder's file list and the shell code blocks in SKILL.md.

  • Network

    Hosts in commands or code, which the agent is likely to contact:

    • api.firecrawl.dev

    Also links to:

    • docs.firecrawl.dev
    • firecrawl.dev
    • github.com

    From URLs in SKILL.md, links to its own repository left out.

  • Credentials

    Names these keys or tokens, usually read from environment variables:

    • FIRECRAWL_API_KEY

    From names ending in _API_KEY, _TOKEN, _SECRET, _KEY or _PASSWORD in SKILL.md.

Context cost

Firecrawl Scraper loads about 1.9k tokens when it runs, and up to ~9k if it reads all its reference files. Until then it costs about 51 tokens; SKILL.md has 586 words of instructions outside code blocks.

Always · name and description, kept in context so the agent knows when to use it
~51
When it runs · the whole SKILL.md, loaded when a task matches
~1.9k
With references · SKILL.md plus every file in references/, read only if the agent opens them
~9k

Estimates: characters ÷ 4, the usual rule of thumb; real counts depend on the model's tokenizer. Scripts and assets cost tokens only if the agent reads them.

Safety

Auto-check: notes

The automated check noted patterns worth knowing about, such as sudo or a known installer.

  • NoteMentions a .env fileSKILL.md:121
    # .env file
  • NoteMentions a .env fileSKILL.md:126
    # .env.local (for local development)

Automated static check — not a guarantee. Review scripts before installing. It scans the text of SKILL.md for risky patterns (piping downloads into a shell, reading credential files, hidden Unicode, destructive commands); files beside SKILL.md are not scanned.

SKILL.md

The full file from secondsky/claude-skills at commit 8837836, republished under its MIT licence (© secondsky). 586 words, ~1,865 tokens.

Download SKILL.mdSave it as .claude/skills/firecrawl-scraper/SKILL.md (or your agent's skills folder). This skill also uses 6 other files; get the full folder from GitHub.
name
firecrawl-scraper
description
Firecrawl v2.5 API for web scraping/crawling to LLM-ready markdown. Use for site extraction, dynamic content, or encountering JavaScript rendering, bot detection, content loading errors.
metadata.keywords
firecrawl, firecrawl api, web scraping, web crawler, scrape website, crawl website, extract content, html to markdown, site crawler, content extraction, web…
license
MIT

Firecrawl Web Scraper Skill

Status: Production Ready ✅ Last Updated: 2026-08-03 Official Docs: https://docs.firecrawl.dev API Version: v2.5


What is Firecrawl?

Firecrawl is a Web Data API for AI that turns entire websites into LLM-ready markdown or structured data. It handles:

  • JavaScript rendering - Executes client-side JavaScript to capture dynamic content
  • Anti-bot bypass - Gets past CAPTCHA and bot detection systems
  • Format conversion - Outputs as markdown, JSON, or structured data
  • Screenshot capture - Saves visual representations of pages
  • Browser automation - Full headless browser capabilities

API Endpoints

1. /v2/scrape - Single Page Scraping

Scrapes a single webpage and returns clean, structured content.

Use Cases:

  • Extract article content
  • Get product details
  • Scrape specific pages
  • Convert HTML to markdown

Key Options:

  • formats: ["markdown", "html", "screenshot"]
  • onlyMainContent: true/false (removes nav, footer, ads)
  • waitFor: milliseconds to wait before scraping
  • actions: browser automation actions (click, scroll, etc.)
2. /v2/crawl - Full Site Crawling

Crawls all accessible pages from a starting URL.

Use Cases:

  • Index entire documentation sites
  • Archive website content
  • Build knowledge bases
  • Scrape multi-page content

Key Options:

  • limit: max pages to crawl
  • maxDepth: how many links deep to follow
  • allowedDomains: restrict to specific domains
  • excludePaths: skip certain URL patterns
3. /v2/map - URL Discovery

Maps all URLs on a website without scraping content.

Use Cases:

  • Find sitemap
  • Discover all pages
  • Plan crawling strategy
  • Audit website structure
4. /v2/extract - Structured Data Extraction

Uses AI to extract specific data fields from pages.

Use Cases:

  • Extract product prices and names
  • Parse contact information
  • Build structured datasets
  • Custom data schemas

Key Options:

  • schema: Zod or JSON schema defining desired structure
  • systemPrompt: guide AI extraction behavior

Authentication

Firecrawl requires an API key for all requests.

Get API Key
  1. Sign up at https://www.firecrawl.dev
  2. Go to dashboard → API Keys
  3. Copy your API key (starts with fc-)
Store Securely

NEVER hardcode API keys in code!

bash
# .env file
FIRECRAWL_API_KEY=fc-your-api-key-here
bash
# .env.local (for local development)
FIRECRAWL_API_KEY=fc-your-api-key-here

SDK Quick Start

Python
bash
pip install firecrawl-py  # v4.5.0+
python
from firecrawl import FirecrawlApp
import os

app = FirecrawlApp(api_key=os.environ.get("FIRECRAWL_API_KEY"))
result = app.scrape_url("https://example.com", params={"formats": ["markdown"], "onlyMainContent": True})
print(result.get("markdown"))
TypeScript/Node.js
bash
bun add @mendable/firecrawl-js  # v4.32.0+
typescript
import FirecrawlApp from '@mendable/firecrawl-js';

const app = new FirecrawlApp({ apiKey: process.env.FIRECRAWL_API_KEY });
const result = await app.scrapeUrl('https://example.com', { formats: ['markdown'], onlyMainContent: true });
console.log(result.markdown);

See: templates/ for crawl, extract, and advanced examples


Common Use Cases

Use CaseEndpointKey Options
Documentation scrapingcrawl_url()limit: 500, allowedDomains
Product data extractionextract()Zod schema + systemPrompt
News article scrapingscrape_url()onlyMainContent: true, removeBase64Images
URL discoverymap()Find all pages before crawling

See: references/common-patterns.md for complete examples.


Show full SKILL.md (235 more words)Show less

Error Handling

python
# Python
try:
    result = app.scrape_url("https://example.com")
except FirecrawlException as e:
    print(f"Firecrawl error: {e}")
typescript
// TypeScript
try {
  const result = await app.scrapeUrl('https://example.com');
} catch (error) {
  console.error('Error:', error.message);
}

Rate Limits & Best Practices

Best PracticeWhy
Use onlyMainContent: trueReduces credits, cleaner output
Set reasonable limitAvoid excessive costs
Use map endpoint firstPlan crawling strategy
Cache resultsAvoid re-scraping
Batch extract callsMore efficient for multiple URLs

Credits: Free tier = 500/month, paid tiers higher.


Cloudflare Workers Integration

⚠️ SDK cannot run in Workers (Node.js dependencies). Use direct REST API:

typescript
const response = await fetch('https://api.firecrawl.dev/v2/scrape', {
  method: 'POST',
  headers: {
    'Authorization': `Bearer ${env.FIRECRAWL_API_KEY}`,
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({ url, formats: ['markdown'], onlyMainContent: true })
});

See: references/common-patterns.md for complete Workers example with caching.


When to Use This Skill

✅ Use Firecrawl❌ Don't Use
Modern JS-rendered sitesSimple static HTML (use cheerio)
Clean markdown for LLMsExisting Puppeteer setup works
RAG/chatbot contentDirect API available
Structured data extractionBudget constraints
Bot protection bypass

Common Issues

IssueCauseFix
"Invalid API Key"Key not setCheck $FIRECRAWL_API_KEY starts with fc-
"Rate limit exceeded"Monthly credits usedCheck dashboard, upgrade plan
"Timeout error"Page slow to loadAdd waitFor: 10000
"Content is empty"JS loads lateAdd actions: [{type: "wait", milliseconds: 3000}]

Advanced Features

FeatureUsage
Browser actionsactions: [{type: "click", selector: "button"}]
Custom headersheaders: {"User-Agent": "Custom Bot"}
Webhookswebhook: "https://your-domain.com/webhook"
Screenshotsformats: ["screenshot"]

See: references/endpoints.md for complete API reference.


When to Load References

ReferenceLoad When...
endpoints.mdNeed complete API endpoint documentation
common-patterns.mdCloudflare Workers, caching, batch processing, error handling

Package Versions

PackageVersion
firecrawl-py4.5.0+
@mendable/firecrawl-js4.32.0+
APIv2

Note: Node.js SDK requires Node.js >=22.0.0, cannot run in Workers.


Official Docs: https://docs.firecrawl.dev | GitHub: https://github.com/mendableai/firecrawl

Token Savings: ~60% | Production Ready: ✅

© secondsky, MIT. Rendered from Markdown: HTML in the file is shown as text, images as links, and headings moved down two levels. Raw file

Files

SKILL.md and 6 other files (references) in plugins/firecrawl-scraper/skills/firecrawl-scraper of secondsky/claude-skills.

  • SKILL.md
  • references/common-patterns.md
  • references/endpoints.md
  • templates/firecrawl-crawl-example.py
  • templates/firecrawl-scrape-python.py
  • templates/firecrawl-scrape-typescript.ts
  • templates/firecrawl-worker-fetch.ts

Open the folder on GitHubat commit 8837836

Used in 1 other repository

We found 1 copy of this SKILL.md (exact, near-identical or edited) in other folders, from 1 other GitHub owner. This page covers the copy in secondsky/claude-skills, which our catalogue first saw on October 7, 2026.

Compare with similar skills

Firecrawl Scraper next to the 5 skills that share the most tags, products or categories with it. Stars are the repository's; “used in” counts other GitHub owners with a copy.

Firecrawl Scraper compared with similar skills
SkillStarsUsed inTokensAuto-checkLicenceRepo updated
Firecrawl Scraper this skillsecondsky/claude-skills2271 repos~1.9kAutomated safety check: NotesMIT
Firecrawl Scraperynulihao/AgentSkillOS6171 repos~4.1kAutomated safety check: NotesMIT
Firecrawlaiskillstore/marketplace4301 repos~2.6kAutomated safety check: PassNone
Firecrawl Scraperlamm-mit/scienceclaw244—~425Automated safety check: PassApache-2.0
Web Scrapingplatonai/Browser41.2k—~1.1kAutomated safety check: PassApache-2.0
Selenium Opinion Crawler123321kk/opinion-agent-ultimate107—~631Automated safety check: PassNone

Similar skills

  • Firecrawl Scraper

    ynulihao/AgentSkillOS

    Complete knowledge domain for Firecrawl v2 API - web scraping and crawling that converts websites into LLM-ready markdown or structured data.

    617 GitHub starsUsed in 1 repo~4.1k tokens
    Data & AnalyticsAuto-check: notes
  • Firecrawl

    aiskillstore/marketplace

    Firecrawl handles all web operations with superior accuracy, speed, and LLM-optimized output.

    430 GitHub starsUsed in 1 repo~2.6k tokens
    Data & AnalyticsAuto-check passed
  • Firecrawl Scraper

    lamm-mit/scienceclaw

    Web scraping of JavaScript-rendered scientific websites using Firecrawl API

    244 GitHub stars~425 tokensUpdated 1 mo ago
    Data & AnalyticsAuto-check passed
  • Web Scraping

    platonai/Browser4

    Extracts data from web pages using browser automation and CSS/JavaScript selectors.

    1.2k GitHub stars~1.1k tokensUpdated today
    Data & AnalyticsAuto-check passed
  • Selenium Opinion Crawler

    123321kk/opinion-agent-ultimate

    browser-based page capture and text extraction for public-opinion research.

    107 GitHub stars~631 tokensUpdated 6 mo ago
    Data & AnalyticsAuto-check passed
  • Chrome Devtools

    einverne/dotfiles

    Browser automation, debugging, and performance analysis using Puppeteer CLI scripts.

    121 GitHub starsUsed in 1 repo~1.6k tokens
    Data & AnalyticsAuto-check: notes

More from secondsky/claude-skills

All 169 skills in this repo
  • Tanstack AI

    secondsky/claude-skills

    TanStack AI (alpha) provider-agnostic type-safe chat with streaming for OpenAI, Anthropic, Gemini, Ollama.

    227 GitHub starsUsed in 1 repo~3.6k tokens
    Auto-check: notes
  • Auto Animate

    secondsky/claude-skills

    AutoAnimate (@formkit/auto-animate) zero-config animations for React.

    227 GitHub stars~2.9k tokensUpdated 9 days ago
    Auto-check passed
  • Base UI React

    secondsky/claude-skills

    MUI Base UI unstyled React components with Floating UI. An agent skill from secondsky/claude-skills.

    227 GitHub stars~1.9k tokensUpdated 9 days ago
    Auto-check passed
  • Cloudflare Images

    secondsky/claude-skills

    This skill should be used when the user asks to "upload images to Cloudflare", "implement direct creator upload", "configure image transformations", "optimize WebP/AVIF", "create image variants"…

    227 GitHub stars~3.6k tokensUpdated 9 days ago
    Auto-check: notes
  • Cloudflare Nextjs

    secondsky/claude-skills

    Deploy Next.js to Cloudflare Workers via the OpenNext adapter (@opennextjs/cloudflare).

    227 GitHub stars~5.3k tokensUpdated 9 days ago
    Auto-check: notes
  • Cloudflare Sandbox

    secondsky/claude-skills

    Cloudflare Sandboxes SDK for secure code execution in Linux containers at edge.

    227 GitHub stars~4.5k tokensUpdated 9 days ago
    Auto-check passed

Questions about Firecrawl Scraper

What does Firecrawl Scraper do?

Firecrawl v2.5 API for web scraping/crawling to LLM-ready markdown. Firecrawl Scraper is an agent skill from secondsky/claude-skills.5 API for web scraping/crawling to LLM-ready markdown.

When should I use Firecrawl Scraper?

Firecrawl Scraper fits situations like: site extraction; dynamic content; encountering JavaScript rendering; content loading errors.

How do I install Firecrawl Scraper in Claude Code?

Run `npx skills add secondsky/claude-skills --skill firecrawl-scraper -a claude-code`. Or copy the skill folder (plugins/firecrawl-scraper/skills/firecrawl-scraper in secondsky/claude-skills) into .claude/skills/firecrawl-scraper in your project. Claude Code loads it when a task matches its description.

How do I install Firecrawl Scraper in Codex?

Run `npx skills add secondsky/claude-skills --skill firecrawl-scraper -a codex`. Or copy the skill folder (plugins/firecrawl-scraper/skills/firecrawl-scraper in secondsky/claude-skills) into .agents/skills/firecrawl-scraper in your project. Codex loads it when a task matches its description.

Can I use Firecrawl Scraper in Cursor, Gemini CLI or GitHub Copilot?

Cursor, Gemini CLI, GitHub Copilot and OpenCode also load SKILL.md folders. With the skills CLI, run `npx skills add secondsky/claude-skills --skill firecrawl-scraper -a cursor` (or -a gemini-cli, github-copilot or opencode for the others). To copy it by hand, put the folder in .cursor/skills/firecrawl-scraper, .gemini/skills/firecrawl-scraper, .github/skills/firecrawl-scraper and .opencode/skills/firecrawl-scraper in your project.

What does Firecrawl Scraper need to run?

Going by SKILL.md and its folder, Firecrawl Scraper needs Python and TypeScript for the scripts in its folder, the command-line tools its instructions call (pip and bun) and credentials named FIRECRAWL_API_KEY. Our summary lists: Python 3; Node.js; A credential in FIRECRAWL_API_KEY.

Does Firecrawl Scraper access the network?

SKILL.md names 4 domains. In commands or code: api.firecrawl.dev; the agent is likely to contact it when it follows the instructions. As links in the text: docs.firecrawl.dev, firecrawl.dev and github.com. This is read from the text; nothing was executed.

Is Firecrawl Scraper safe to install?

Our automated static check of SKILL.md found notes only (mentions a .env file), nothing it rates as a warning. It is not a guarantee. Review the folder before installing.

What licence does Firecrawl Scraper use?

Firecrawl Scraper is published under the MIT licence (declared in SKILL.md). It allows redistribution, so the full SKILL.md is shown on this page.

How many tokens does Firecrawl Scraper use?

About 1.9k tokens (SKILL.md is roughly 7.5k characters). Agents keep only the skill's name and description in context until a task matches; then they load SKILL.md in full. Its references folder adds about 7.1k tokens, read only when the agent opens those files.

What are the alternatives to Firecrawl Scraper?

Skills that share tags, products or a category with Firecrawl Scraper: Firecrawl Scraper (ynulihao/AgentSkillOS, 617 stars), Firecrawl (aiskillstore/marketplace, 430 stars), Firecrawl Scraper (lamm-mit/scienceclaw, 244 stars) and Web Scraping (platonai/Browser4, 1.2k stars). The comparison table on this page puts their stars, adoption, token cost, safety result and licence side by side.

Who maintains Firecrawl Scraper?

secondsky (a GitHub user) maintains it in secondsky/claude-skills, which has 227 GitHub stars. The repository holds 169 skills in this directory. The repository was last updated on September 28, 2026.

Source: secondsky/claude-skills on GitHub. Facts on this page come from the repository at the commit we read; the author's words are quoted as theirs.