Pp Scrape Do

<!-- GENERATED FILE — DO NOT EDIT.
     This file is a verbatim mirror of library/developer-tools/scrape-do/SKILL.md,
     regenerated post-merge by tools/generate-skills/. Hand-edits here are
     silently overwritten on the next regen. Edit the library/ source instead.
     See the repository agent guide, section "Generated artifacts: registry.json, cli-skills/". -->

Scrape.do — Printing Press CLI

Prerequisites: Install the CLI

This skill drives the scrape-do-pp-cli binary. You must verify the CLI is installed before invoking any command from this skill. If it is missing, install it first:

1. Install via the Printing Press installer:

   npx -y @mvanhorn/printing-press-library install scrape-do --cli-only

2. Verify: scrape-do-pp-cli --version
3. Ensure the directory the npx installer placed the binary in is on $PATH. On Windows this is %LOCALAPPDATA%\Programs\PrintingPress\bin; on macOS/Linux it is typically $HOME/.local/bin or a platform-specific location printed by the installer. If you used the Go fallback instead, add $GOPATH/bin (or $HOME/go/bin).

If the npx install fails (no Node, offline, etc.), fall back to a direct Go install (requires Go 1.26.6 or newer):

go install github.com/mvanhorn/printing-press-library/library/developer-tools/scrape-do/cmd/scrape-do-pp-cli@latest

If --version reports "command not found" after install, the install step did not put the binary on $PATH. Do not proceed with skill commands until verification succeeds.

Every Scrape.do surface — the core scraper plus the whole Google family (search, maps, news, shopping, flights, hotels, trends) — wrapped with an offline SQLite history. The governor estimates credit cost before every call with cost, debits a local ledger from the authoritative cost header with budget, and gates concurrent requests against your plan's live ceiling so an agent swarm never 429s itself or burns the monthly budget. SERPs become queryable history: drift and movers surface rank changes offline with no re-spend.

When to Use This CLI

Reach for this CLI whenever a task needs Google search results, structured SERP data, or any proxied web scrape via Scrape.do — and especially when several agents share one Scrape.do account. Its governor (cost/budget/batch) keeps concurrent usage inside the plan's limits and the monthly credit budget, and its offline SERP history (drift/movers) answers rank-change questions without re-spending credits. Prefer it over raw curl calls when you care about credit cost, concurrency safety, or comparing results over time.

When Not to Use This CLI

Do not activate this CLI for requests that require creating, updating, deleting, publishing, commenting, upvoting, inviting, ordering, sending messages, booking, purchasing, or changing remote state. No command mutates remote target state. Note, however, that scrape, google, and batch make billed GET requests that consume Scrape.do account credits — use cost to estimate before spending and budget to track and cap spend.

Unique Capabilities

These capabilities aren't available in any other tool for this API.

Offline SERP intelligence

• drift — Diff a Google query's two most recent stored SERPs and see exactly which results moved, appeared, or dropped — entirely offline, no credits spent.

  When an agent needs to know whether a ranking changed since last check, this answers it without re-spending a 10-credit SERP call.

  scrape-do-pp-cli drift "best crm software" --json

• movers — Scan every tracked query's latest-versus-previous SERP snapshot and surface only the queries whose top positions moved past a threshold.

  Turns hundreds of tracked keywords into a short 'what changed this week' list an agent can act on.

  scrape-do-pp-cli movers --threshold 3 --agent

Credit & concurrency governance

• batch — Dispatch a list of URLs or queries through a shared concurrency lease and per-call credit ledger, auto-retrying only the non-billed 429/502/510 classes and stopping before a credit ceiling.

  Lets an agent swarm hammer one account at full speed without 429 storms or blowing the monthly budget.

  scrape-do-pp-cli batch --input urls.txt --max-credits 500 --agent

• budget — Attribute spend by mode and query-family from a local ledger debited off the authoritative per-call cost header, joined with cached account state to forecast burn-rate against days remaining.

  Tells an agent how much budget is left and which workloads are eating it before the account hits a hard 401.

  scrape-do-pp-cli budget --agent

• cost — Print the exact credit cost a request will incur — accounting for render, super-proxy, Google endpoints, and per-domain overrides — before spending a single credit.

  Lets an agent compare the cost of cheap vs expensive scrape modes and pick the cheapest path that works.

  scrape-do-pp-cli cost --url https://www.linkedin.com/company/example --render --super

Command Reference

account — Live Scrape.do account state: subscription status, concurrency allowance and headroom, monthly credit cap and remaining credits.

• scrape-do-pp-cli account — Fetch live account state: IsActive, ConcurrentRequest, RemainingConcurrentRequest, MaxMonthlyRequest

Finding the right command

When you know what you want to do but not which command does it, ask the CLI directly:

scrape-do-pp-cli which "<capability in your own words>"

which resolves a natural-language capability query to the best matching command from this CLI's curated feature index. Exit code 0 means at least one match; exit code 2 means no confident match — fall back to --help or use a narrower query.

Recipes

Narrow a deep SERP payload for an agent

scrape-do-pp-cli google search "coffee makers" --agent --select organic_results.position,organic_results.title,organic_results.link

The SERP JSON is large and deeply nested; --select with dotted paths returns just rank, title, and link so an agent doesn't burn context parsing the full payload.

Estimate cost before an expensive scrape

scrape-do-pp-cli cost --url https://www.linkedin.com/company/example --render --super

Prints the expected credits (LinkedIn domain override + render + super proxy) with no API spend, so you can choose the cheapest mode that works.

Fan out a URL list under the concurrency cap

scrape-do-pp-cli batch --input urls.txt --max-credits 500 --agent

Dispatches every URL through the shared concurrency lease, retries only the non-billed 429/502/510 classes, and stops before the 500-credit ceiling.

See what ranked-changes happened this week

scrape-do-pp-cli movers --threshold 3 --agent

Compares each tracked query's two latest stored SERPs and lists only the queries whose top positions moved by 3 or more — entirely offline.