Apify AI Search Visibility Tracker

AI Search Visibility Tracker

Four workflows covering the full AI visibility lifecycle: discover which prompts matter → find citation opportunities → audit your site → track over time.

All workflows use apify/google-search-scraper for AI search. Workflow C also uses apify/website-content-crawler.

Recommended flow: Run Workflow A to discover prompts → Workflow B to find citation opportunities → Workflow C to audit your site → Workflow D to track everything on a schedule.

Workflow A — Competitor Prompt Discovery

Goal: Find which queries surface a competitor in AI search answers, so you know which prompts are worth monitoring.

Inputs to collect

Workflow

1. Generate 15–30 candidate queries from seed keywords using these templates:

   • best [topic], [topic] tools, how to [topic], [topic] for [use case]
   • [topic] vs [competitor brand], [competitor brand] alternative
   • [topic] API, [topic] pricing, [topic] tutorial

2. Run apify/google-search-scraper for each candidate query. For each result, extract:

   • aiOverview.sources[], aiMode.sources[], chatGptAnswer.sources[], perplexityAnswer.sources[], copilotAnswer.sources[], geminiAnswer.sources[]
   • Also check answer_text / aiOverview.text for competitor brand name mentions (word-boundary match: \bBrand\b)

3. For each (query, source) pair where the competitor domain or brand appears: record a hit.

4. Output a prompt-major table sorted by total hit count descending:

| Query | ChatGPT | Perplexity | AI Overviews | AI Mode | Copilot | Gemini | Total |
|-------|---------|------------|--------------|---------|---------|--------|-------|
| "best web scraping API" | ✓ | ✓ | — | ✓ | — | ✓ | 4 |
| "how to scrape Google" | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | 6 |

5. Deliver the top-N queries (default 10) as a ready-to-paste list for Workflow D's config.json prompts.

Workflow B — Citation Opportunity Finder

Goal: For a target topic, identify which domains and content types AI engines most often cite — revealing where to publish or pitch content.

Inputs to collect

Workflow

1. Run apify/google-search-scraper for each seed query across selected AI sources.

2. Collect every URL from sources[] across all results. Normalise to registrable domain (blog.example.com → example.com).

3. Aggregate:

   • By domain: count citations, list which AI sources cite it, list which queries triggered it
   • By content type: infer from URL path patterns (docs → /docs/, /reference/; blog → /blog/; news → known news domains)

4. Rank by total citation count. Output:

Top-cited domains for "web scraping" (42 queries × 6 sources):
| Domain | Citations | AI Sources | Inferred type |
|--------|-----------|------------|--------------|
| docs.apify.com | 38 | ChatGPT, Perplexity, AI Mode | Documentation |
| scraperapi.com/blog | 21 | AI Overviews, Gemini | Long-form blog |

5. If deep-crawl enabled: run apify/website-content-crawler on the top-3 cited URLs per domain. From the markdown output, extract:

   • First heading that directly answers the query
   • Presence of code blocks in first 500 words
   • Word count
   • Whether an H2/H3 contains the exact query phrase

6. Summarise patterns: "AI engines in this topic prefer [long-form docs / short direct-answer posts]. Typical cited page: [word count range], [has/lacks direct-answer H2], [has/lacks code example above the fold]."

Workflow C — GEO Website Audit

Goal: Check whether a specific website's content is structured for AI citation; compare it against what AI engines actually cite for your target prompts.

Inputs to collect

Workflow

1. Run apify/google-search-scraper for each target prompt. For each (prompt × source) record whether your registrable domain appears in sources[].

2. For prompts where your domain is not cited: identify the top-cited competitor URL for that prompt.

3. Run apify/website-content-crawler on:

   • Your most relevant page(s) for each un-cited prompt
   • The top-cited competitor page for each un-cited prompt

4. For each un-cited prompt, produce a gap card:

Prompt: "how to scrape Google search results"
Your page: apify.com/blog/scraping-google  →  NOT cited on ChatGPT, Perplexity, AI Mode
Top-cited: docs.brightdata.com/scraping/google (cited 5/6 sources)

Structural gaps:
  ✗ Your page: answer buried after 900 words, no direct-answer H2
  ✓ Competitor: H2 "How to scrape Google in 3 steps" at word 120 + code block at word 180

Recommended actions (priority order):
  1. Add H2 that mirrors the query phrase within first 300 words
  2. Move code example above the fold
  3. Add a "Quick answer" summary box at the top

5. Deliver: per-prompt gap cards + a consolidated action table ranked by expected impact.

Workflow D — Recurring Visibility Tracker

Goal: Snapshot brand citations and mentions across all six AI surfaces on a recurring schedule and track changes over time.

Prerequisites

(No need to check upfront)

• APIFY_TOKEN saved in a .env file next to config.json (the runner auto-loads it).
• Python 3.9+ on PATH. pip3 install requests (only third-party dependency); pip3 install tldextract recommended for accurate registrable-domain matching on multi-part TLDs.
• For automated daily runs: macOS / Linux with launchd or cron available (the installer handles both). Windows users get printed schtasks instructions.

Steps

Copy this checklist and track progress:

Task Progress:
- [ ] Step 1: Load or collect the seven required inputs
- [ ] Step 2: Confirm AI sources and cadence
- [ ] Step 3: Write config.json + .env, then install the OS schedule
- [ ] Step 4: Run a snapshot now so the user sees the first report
- [ ] Step 5: Deliver the history report (diff vs. all prior runs)

Step 1: Load or Collect the Seven Required Inputs

If config.json exists in the user's working directory, load it and skip to Step 4 unless the user asks to reconfigure. On first run, ask all seven anchors before any Actor call:

After those seven, ask optional follow-ups: countryCode, languageCode, location (UULE), preferred run hour (default 09:00 local).

Then one verbosity question -- save as config.json:include_full_answers:

• on_demand (default): report shows short quoted snippets around each surface-form match. Full LLM answers live in the named KV store; user can ask later.
• always: report embeds the full LLM answer verbatim whenever any entity is mentioned. Useful for one prompt; gets unwieldy at 5+ prompts.