Multi Account Scraping

Multi-Account Scraping

Two problems, one mechanism.

"This site only gives me the data when I am logged in, and I run twelve accounts." Each account gets its own browser profile with its own fingerprint, cookies and exit IP, and one command runs across all of them at once.

"My scraper broke again" or "the site sends my scraper a captcha but works fine in a browser." The request comes from a real browser session, and the per-site adapter lives in a shared repository, so a markup change is fixed once for everyone instead of in every caller's codebase.

The unit is a recipe: one file that turns one site into one command. The caller asks for reddit/hot and gets structured JSON, instead of a browser handle and a scraping problem.

• Recipes live in their own public repository: https://github.com/antibrow/recipes
• Both SDKs read the same registry, so one recipe produces the same output from Node or Python
• Requires anti-detect-browser 2.26.0 or newer (npm), or antibrow 0.20.0 or newer (PyPI)
• Full browser/profile reference: the anti-detect-browser skill

Authorized use only. Read sites and accounts you own or are permitted to operate, and public pages. Recipes read; they do not post, vote, send or buy. Respect each site's terms and rate limits - see Acceptable use.

What this does not claim. A recipe does not get you past bot management. Whether a strict site answers at all depends on the exit IP's reputation, the request cadence and the history the profile carries. Several published recipes need a clean residential exit, and they say so; see Which recipes need a good exit IP.

When to use

• An agent needs data, not a browser. One command, JSON out, no selectors to invent per run.
• The same task across many identities. fanout runs one recipe on N profiles concurrently, each with its own persona, cookie jar and exit IP.
• A site that refuses plain HTTP scraping. The recipe runs inside a real browser session, so the request carries a real client and that profile's cookies.
• A scraper that keeps breaking. The adapter lives in a shared repo, so a markup change is fixed once for everyone rather than in each caller's codebase.

Do not reach for this when the site has a public API you can call directly, or when you need many pages of one site with no login and no identity requirement - a plain HTTP scraper with a proxy pool is cheaper and faster than a browser per identity.

First run

npm install -g anti-detect-browser          # or: pip install antibrow
export ANTIBROW_API_KEY=...                 # get one at https://antibrow.com

anti-detect-browser recipe update           # pull the registry, pin every recipe by sha256
anti-detect-browser recipe list
anti-detect-browser recipe info reddit/hot  # args, hosts, identity, review state
anti-detect-browser recipe run reddit/hot --temporary --json

Python is the same set of subcommands: python -m antibrow recipe list|info|run|fanout|test|scaffold|guide.

Commands

Always pass --jq, or the whole payload lands in your context. The filter is a small documented subset: .items[].title, .items[0], .items[1:3], .a.b, ["odd key"], length, keys, joined with |. Anything outside it is rejected by name rather than approximated, so a filter never silently returns null.

From code

import { runRecipe, fanoutRecipe } from 'anti-detect-browser'

const { value, blockedHosts } = await runRecipe({
  id: 'reddit/hot',
  key: process.env.ANTIBROW_API_KEY!,
  temporary: true,
  args: { limit: 5 },
})

const { results } = await fanoutRecipe({
  id: 'amazon/search',
  key: process.env.ANTIBROW_API_KEY!,
  profiles: ['shopper-01', 'shopper-02', 'shopper-03'],
  args: { query: 'usb c cable' },
  concurrency: 3,
})

from antibrow import run_recipe, fanout_recipe

result = run_recipe("reddit/hot", temporary=True, args={"limit": 5})
print(result.value, result.blocked_hosts)

fanned = fanout_recipe("amazon/search", ["shopper-01", "shopper-02"], args={"query": "usb c cable"})
for row in fanned.rows:
    print(row.profile, row.result.value if row.ok else row.error)

run_recipe_async() is the asyncio mirror. Runs default to focusWindow: false and restoreTabs: false, so a fanout does not steal focus and no restored tab from a previous session is visible to the site.

Fanout and the concurrency cap

fanout reads the plan's concurrent-browser limit before queueing anything and clamps to it, rather than launching into a refusal halfway through - a refused launch mid-run looks like a broken recipe. Free plans allow one browser at a time, so a fanout there is serial but still works.

One profile failing does not take the rest down. Each row is { profile, ok, value | error }, ordered as you asked for them, so a caller diffing two identities does not have to sort first.

What a recipe may do

export const meta = {
  id: 'reddit/hot',
  summary: 'Hot posts from the front page or one subreddit.',
  domains: ['www.reddit.com'],      // every host it may reach
  entry: 'https://www.reddit.com/', // the page the runtime opens
  identity: 'any',                  // 'any' | 'logged-in' | 'anonymous'
  args: [{ name: 'limit', type: 'number', default: 25, max: 100 }],
}

export async function run(ctx, args) {
  const res = await ctx.fetchJson(`/hot.json?limit=${args.limit}`)
  return { items: res.data.children.map((c) => ({ id: c.data.id, title: c.data.title })) }
}

run() executes inside the page, so a relative fetch carries that profile's session for that site and the SDK never touches a cookie. document and the site's own JavaScript are available; there is no fs, no process, and no navigation (a recipe cannot navigate, because that would destroy the context it is running in).

ctx is the whole surface: fetchJson(path, init?), fetchText(path, init?), log(message), sleep(ms) (capped at 30s per call), profileName.

meta.entry may interpolate arguments - https://www.google.com/search?q={query} - for a page that only exists per query. Reach for this when a site only renders what you want on a real navigation: Google answers an in-page fetch for its own result page with a redirect interstitial, so fetching it can never work. The host is never interpolated; it is what the allowlist is checked against.

Why a third-party recipe is safe to run in a logged-in profile

A recipe is community code executing in a browser profile that may hold live sessions, so treat it the way you would a browser extension. Three things make that workable, and they are enforced client-side rather than promised by the repo:

1. meta.domains is enforced at the network layer. A request to any host the recipe did not declare is aborted, so a Reddit recipe cannot reach your mail. This is deliberately not implemented inside ctx.fetchJson: code running in a page can always call fetch itself. Blocked hosts come back in the result, which is also how you learn what a new recipe still needs to declare.
2. Recipes are pinned by SHA-256, and the file must agree with the registry row that advertised it (id, entry, identity, domains). A row claiming a narrower allowlist than the file declares is refused rather than run.
3. Only reviewed recipes run by default. --allow-unreviewed opts in, and only on a --temporary profile - those are local-only, so nothing an unreviewed recipe collects travels to another machine.

Read the recipe you are about to run. It is one screen of code, and recipe info prints its declared hosts first.