Competitor Analysis

Competitor Analysis

When to Use

Use when the user needs structured competitor research with Browserbase discovery, enrichment lanes, screenshots, comparison matrices, and a final HTML report.

Source: browserbase/skills (MIT).

Analyze a user's competitors. Uses Browserbase Search API for discovery and a 4-lane Plan→Research→Synthesize pattern for enrichment — outputting an HTML report with overview, per-competitor deep dives, a side-by-side feature/pricing matrix, and a chronological mentions feed.

Required: BROWSERBASE_API_KEY env var and the browse CLI installed (npm install -g browse).

First-run setup: On the first run you'll be prompted to approve browse cloud fetch, browse cloud search, cat, mkdir, sed, etc. Select "Yes, and don't ask again for: browse cloud fetch:*" (or equivalent) for each. To permanently approve, add these to your ~/.claude/settings.json under permissions.allow:

"Bash(browse:*)", "Bash(bunx:*)", "Bash(bun:*)", "Bash(node:*)",
"Bash(cat:*)", "Bash(mkdir:*)", "Bash(sed:*)", "Bash(head:*)", "Bash(tr:*)", "Bash(rm:*)"

Path rules: Always use full literal paths in Bash — NOT ~ or $HOME. Resolve the home directory once and use it everywhere. When building subagent prompts, replace {SKILL_DIR} with the full literal path.

Output directory: All output goes to ~/Desktop/{company_slug}_competitors_{YYYY-MM-DD}/. This directory contains one .md file per competitor plus the generated HTML views and CSV.

CRITICAL — Tool restrictions (applies to main agent AND all subagents):

• All web searches: use browse cloud search. NEVER WebSearch.
• All page fetches: use browse cloud fetch --allow-redirects (returns markdown by default; add --format raw if you need the original HTML, then pipe through sed ... | tr -s ' \n' to extract text). NEVER WebFetch. 1 MB response limit — fall back to browse get markdown (after browse open <url> --remote) for JS-heavy pages.
• All research output: subagents write one markdown file per competitor to {OUTPUT_DIR}/{competitor-slug}.md using bash heredoc. NEVER use the Write tool or python3 -c. See references/example-research.md for the file format.
• Report compilation: use node {SKILL_DIR}/scripts/compile_report.mjs {OUTPUT_DIR} --user-company "{user_company}" --open — generates index.html, competitors/*.html, matrix.html, mentions.html, results.csv in one step and opens overview.
• URL deduplication: node {SKILL_DIR}/scripts/list_urls.mjs /tmp --prefix competitor.
• Subagents must use ONLY the Bash tool.
• Main agent NEVER reads raw discovery JSON batch files.

CRITICAL — Minimize permission prompts:

• Subagents MUST batch ALL file writes into a SINGLE Bash call using chained heredocs.
• Batch ALL searches and ALL fetches into single Bash calls via && chaining.

Pipeline Overview

Follow these 8 steps in order. Do not skip or reorder.

1. User Company Research — Deeply understand the user's company, produce precise_category + category_include_keywords + exclusion_list
2. Depth Mode + Seed Input — Choose depth, accept optional seed competitor URLs
3. Discovery (3 parallel waves) — Wave A (alternatives), Wave B (precise category), Wave C (comparison-page graph via "X vs Y" title parsing)
4. Gate — scripts/gate_candidates.mjs fetches each candidate's hero text (via browse cloud fetch) and drops wrong-category URLs
5. Confirm enrichment set with the user — Present PASS / UNKNOWN / rejected-brand-matches via AskUserQuestion. User ticks the real ones, adds any the discovery missed. Skipping this step is wasteful because enrichment is expensive (25 subagents × depth budget) and the gate is imperfect (JS-heavy homepages, Cloudflare challenges, semantic-variant taglines)
6. Deep Enrichment (5 subagents per competitor in deep/deeper modes) — Marketing, Discussion, Social, News, Technical — each lane a separate subagent writing to partials/; then merge_partials.mjs consolidates. In deep/deeper modes, Step 5d adds a 6th Battle Card synthesis lane AFTER Step 5c fact-check completes — produces per-competitor Landmines / Objection Handlers / Talk Tracks grounded in cited evidence.
7. Screenshots — capture_screenshots.mjs via the browse CLI captures a 1280×800 homepage hero per competitor
8. HTML Report — Overview + per-competitor (with embedded hero screenshot + Battle Card card) + matrix + mentions views

Step 0: Setup Output Directory

OUTPUT_DIR=~/Desktop/{company_slug}_competitors_{YYYY-MM-DD}
mkdir -p "$OUTPUT_DIR"

Replace {company_slug} with the user's company name (lowercase, hyphenated) and {YYYY-MM-DD} with today's date. Pass {OUTPUT_DIR} as a full literal path to every subagent.

Clean up discovery batch files from prior runs:

rm -f /tmp/competitor_discovery_batch_*.json

Re-runs must start from a clean $OUTPUT_DIR. compile_report.mjs ingests every {slug}.md in the directory, and merge_partials.mjs only overwrites the slugs in the current set — it never deletes ones dropped from a new enrichment set. Since the directory is keyed by date, a same-day re-run with a different competitor set would leave stale competitors in the overview, matrix, CSV, and screenshots. Either use a fresh directory or clear the prior per-competitor files first:

rm -f "$OUTPUT_DIR"/*.md && rm -rf "$OUTPUT_DIR"/partials "$OUTPUT_DIR"/screenshots

Step 1: User Company Research

This step sets the baseline for what "competitor" means AND produces the verified data the Step 5b matrix will use for the userCompany row.

Rule: The user's company gets the same 5-lane research depth as competitors. Do NOT fill userCompany in matrix.json from memory — it will ship false claims to the user's own team. On a search-API run (user company Exa, 2026-04-23), skipping this step produced a matrix that claimed Exa had a "published uptime SLA" (there is no numeric public SLA — only a status page) and marked its MIT-licensed Python SDK as open-source: false (the repo is github.com/exa-labs/exa-py, LICENSE confirmed MIT). Both errors would have surfaced in the "Where you're winning" card as fabricated moats.

Process:

1. Ask the user for their company name or URL.

2. Check for an existing profile at {SKILL_DIR}/profiles/{company-slug}.json. If it exists, load it and confirm with the user: "I have your profile from {researched_at}. Still accurate?" — if yes, skip to Step 2 BUT still run the partial-lane enrichment below so matrix synthesis has fresh feature evidence. The profile format is shared with company-research (same shape). If a user already has a profile saved under company-research/profiles/, you may copy it into this skill's profiles directory rather than re-researching.

3. Run the full 5-lane enrichment on the user's company — identical to the competitor pattern in Step 5. For each lane, spawn a Bash-only subagent that writes to {OUTPUT_DIR}/partials/{user-slug}.{lane}.md:

   • marketing — tagline, positioning, pricing tiers, features, integrations, open-source components (SDK repos + licenses), regions offered, compliance (SOC 2 / HIPAA / trust portal URL)
   • technical — REST + streaming API support (with docs URLs), SDK languages, MCP server URL, neural vs keyword retrieval modes, reranking / highlights / live-crawl specifics, published uptime SLA (actual %, not status page), third-party retrieval-quality benchmarks
   • discussion, social, news — optional in quick mode, recommended in deep+ See references/research-patterns.md → "Self-Research" for sub-questions. Each finding MUST cite a URL.

4. Run merge_partials.mjs on the user's partials too — produces {OUTPUT_DIR}/{user-slug}.md, the canonical source Step 5b reads from for userCompany flags.

5. Synthesize into a profile: Company, Product, Existing Customers, Competitors (seed list), Use Cases, precise_category, category_include_keywords, exclusion_list. Do NOT include ICP — this skill doesn't need it.

   • precise_category: one sentence describing the category. e.g., "AI web search API for agents with neural + keyword retrieval". Avoid vague words like "tools" / "platform".
   • category_include_keywords: 8-15 phrases a direct competitor's marketing would likely contain (hero or title). Include semantic variants.
   • exclusion_list: phrases that indicate a different category — used by the gate to reject false positives (e.g. antidetect browser, scraping api, screenshot api, residential proxy). See references/research-patterns.md → "Synthesis Output" for the exact format and Exa as a worked example.

6. Present the profile + the user-company .md to the user for confirmation. Do not proceed until confirmed.

7. Save the confirmed profile to {SKILL_DIR}/profiles/{company-slug}.json.

Step 2: Depth Mode + Seed Input

Ask clarifying questions via AskUserQuestion with checkboxes:

• Known competitors? Text area for URLs/names (optional — discovery will find more).
• Depth mode?
  • quick — marketing surface only, many competitors, ~2-3 tool calls each
  • deep — + external signal (mentions, reviews, news), ~5-8 tool calls each
  • deeper — + public benchmarks + strategic diff vs user's company, ~10-15 tool calls each
• Target count? Rough number of competitors to research (e.g., 10 / 20 / 50).

This is the ONLY user interaction. After this, execute silently until the report is ready.

Step 3: Discovery (3 parallel waves)

Formula: ceil(target_count / 20) queries per wave. Over-discover ~3x because the gate drops ~40-60%.

Evaluation on a search-API run shows all three waves are additive — skip any and you lose real competitors:

Wave A — Generic alternatives (broad; heavy aggregator noise, filtered out later)

• "alternatives to {user_company}"
• "{user_company} competitors"

Wave B — Precise category (uses precise_category from the profile)

• "{precise_category}" verbatim
• 2-3 queries composed from the most distinctive tokens (e.g. "web search api for ai agents", "retrieval API for LLMs")

Wave C — Comparison-page graph (highest precision)

• "{user_company} vs"
• "{seed1} vs", "{seed2} vs", "{seed3} vs" (seeds from the profile's competitors list)
• After the searches, run scripts/extract_vs_names.mjs to parse "X vs Y" patterns from result titles — this uniquely surfaces competitors that don't appear as URL hits.

Process:

1. Issue 3 parallel browse cloud search Bash calls (one per wave) in a SINGLE message — NOT subagents. Each Bash call chains its 2-4 queries with &&. See references/workflow.md → "Discovery — parallel Bash, not subagents" for the exact recipe. Subagents are too heavy for a workload of 6-12 browse cloud search calls.
2. After all waves complete:

   node {SKILL_DIR}/scripts/list_urls.mjs /tmp --prefix competitor > /tmp/competitor_urls.txt
   node {SKILL_DIR}/scripts/extract_vs_names.mjs /tmp --prefix competitor \
     --seed "{user_company},{seed1},{seed2},{seed3}" \
     > /tmp/competitor_vs_names.jsonl

3. Filter /tmp/competitor_urls.txt — remove blog posts, news, AI-tool directories (seektool.ai, respan.ai, agentsindex.ai, toolradar.com, aitoolsatlas.ai, vibecodedthis.com, etc.), review aggregators (g2.com, capterra.com), databases (crunchbase.com, tracxn.com), user's own domain. See references/workflow.md for the full noise-domain list.
4. For vs_names entries that have a resolved domain, add them. For unresolved names, optionally run browse cloud search "{name}" --num-results 3 and pick the top root domain.
5. Merge with user-provided seed URLs. Dedup by hostname → /tmp/competitor_candidates.txt.

Step 4: Gate (category-fit filter)

Drop candidates whose marketing identifies them as a different category before enrichment burns tool calls on them.

cat /tmp/competitor_candidates.txt \
  | node {SKILL_DIR}/scripts/gate_candidates.mjs \
      --include "{profile.category_include_keywords joined with commas}" \
      --exclude "{profile.exclusion_list joined with commas}" \
      --concurrency 6 \
  > /tmp/competitor_gated.jsonl

grep '"status":"PASS"' /tmp/competitor_gated.jsonl \
  | node -e 'require("fs").readFileSync(0,"utf-8").split("\n").filter(Boolean).forEach(l => { try { console.log(JSON.parse(l).url); } catch {} })' \
  > /tmp/competitor_passed.txt

The gate fetches each candidate's homepage via browse cloud fetch --allow-redirects --format raw, extracts the first 800 chars of visible text, and classifies position-aware: exclude in <title> → REJECT; include in <title> → PASS; hybrid title → hero200 tiebreak; otherwise fall through.

Evaluated on a search-API run with 12 mixed candidates: 7/7 real competitors passed, 4/4 wrong-category rejected, 1 known-hybrid edge case rejected.

Step 4.5: Confirm enrichment set with the user

This step is mandatory. Do NOT skip to enrichment just because the gate ran.

Enrichment is expensive: 5 competitors × 5 lane-subagents = 25 subagents, ~10-15 minutes of wall clock, ~300 browse cloud calls. Running it on the wrong set wastes all of that. The gate also has known blind spots:

• JS-heavy homepages (e.g. Tavily, Firecrawl) — browse cloud fetch returns near-empty text, so keyword matching has nothing to match on → REJECT or UNKNOWN
• Cloudflare challenge pages (e.g. Perplexity) — title becomes "Just a moment..." → no category signal
• Semantic variants — "search foundation" / "retrieval backbone" don't lexically match a list centered on "search API"
• Domain ambiguity — brave.com (the browser) vs api-dashboard.search.brave.com (the actual API product) can confuse classification

The user almost always has domain knowledge the skill lacks. Ask them.

Process — the main agent:

1. Read /tmp/competitor_gated.jsonl and group rows:

   • PASS bucket: everything with status=PASS.
   • UNKNOWN bucket: status=UNKNOWN (fetch failed — always surface, these are the silent misses).
   • Rejected-brand bucket: top ~10 REJECT rows whose title mentions a well-known brand pattern (e.g. contains the token from a user-supplied seed list, or appears frequently in the Wave C "X vs Y" graph).

2. Present the buckets to the user, one table per bucket, with URL + title + reason (for rejects).

3. Use AskUserQuestion with a checkbox list of all candidates across the three buckets, plus a free-text "add more" field. The prompt should be explicit:

   "Here are the gate's picks plus a few it was unsure about. Tick the ones that are real competitors in your space, and paste any URLs I missed (comma-separated). Enrichment will run on ONLY the ticked set."

4. Write the confirmed set to /tmp/competitor_enrichment_set.txt (one URL per line). This is the input for Step 5 — not /tmp/competitor_passed.txt.

If the user doesn't respond or explicitly says "just run it", fall back to /tmp/competitor_passed.txt as-is, but warn in chat that the run may waste budget on wrong-category hits.

Exa test, 2026-04-24: gate auto-passed 22 of 101 candidates but missed Tavily (generic title), Jina AI (semantic mismatch — "search foundation"), Firecrawl (JS-heavy fetch failure), and Perplexity (Cloudflare challenge). All four are real direct competitors. This step catches them.

Step 5: Deep Enrichment

Two modes. See references/workflow.md for prompt templates and wave management. See references/research-patterns.md for the lane-by-lane methodology.

Quick mode — single subagent per batch

• Input: /tmp/competitor_enrichment_set.txt (user-confirmed set from Step 4.5), ~8 competitors per subagent.
• One subagent runs Lane A only (marketing surface). 2-3 tool calls each.
• Writes directly to {OUTPUT_DIR}/{slug}.md.

Deep / Deeper mode — 5 subagents PER competitor (parallel lane fan-out)

For each competitor, launch 5 parallel subagents, one per lane:

• A. Marketing (marketing): pricing, features, positioning, integrations, customers, team, funding, HQ. Owns canonical frontmatter.
• B. Discussion (discussion): Reddit, HN, forums, Dev.to, Hashnode. Broad queries beyond site: — also "{competitor}" review 2026, "{competitor}" issues OR problems, "{competitor}" discussion.
• C. Social (social): LinkedIn posts, YouTube videos, Twitter/X. Snippets only — do NOT fetch.
• D. News & Comparisons (news): TechCrunch, Verge, VentureBeat, Forbes, Businesswire, Substack, blog reviews. Every mention needs a date.
• E. Technical & Benchmarks (technical): GitHub benchmark repos/PRs, performance posts. Writes Benchmarks + technical Findings.

Budget per lane: deep = 5-8 tool calls, deeper = 10-15. Launch ALL competitor × lane subagents in a SINGLE Agent tool message. For 10 competitors × 5 lanes = 50 parallel Agent calls in one message. Do NOT split into batches per competitor or per lane — wall clock collapses to the slowest single agent (~3-5 min). Splitting into 5 rounds of 10 cost 25 minutes of wall clock vs 5 minutes parallel on a real measured run; do not do it.

Each subagent writes a partial to {OUTPUT_DIR}/partials/{slug}.{lane}.md.

Critical: Pass the user's company name, product, and key features verbatim into every subagent prompt so the technical lane can do strategic diffing. Pass the full literal {OUTPUT_DIR} path to every subagent.