Capture

Traffic Capture (Phase 1)

Assess the site, then capture comprehensive HTTP traffic. This skill combines site assessment with full traffic recording in a single browser session.

Copy this checklist and check off items as you complete them:

Phase 1 Progress:
- [ ] Prerequisites: playwright-cli available, checkpoint restored if one exists
- [ ] Step 1: Browser open on target URL
- [ ] Step 2: Fingerprint run + auth handled + profile classified + assessment.md written
- [ ] Step 3: Full capture — exploration targets met for the profile
- [ ] Step 4: Trace parsed, validate-capture.py PASSED (the gate)
- [ ] Step 5: Browser closed, checkpoint + phase-state marked complete

CRITICAL EXECUTION RULES

NEVER use run_in_background: true for ANY playwright-cli command. All playwright-cli commands must run in the foreground with appropriate timeouts. Background execution causes task ID tracking failures — the command completes before you can read the output. See references/playwright-cli-commands.md for the timeout table.

NEVER use eval for complex expressions. eval fails silently on ternaries, comma operators, and multi-branch logic with "not well-serializable" errors. Use run-code instead. See references/framework-detection.md for details.

ESM context — no require(). run-code uses ESM. Use await import('fs') instead of require('fs'). See references/playwright-cli-commands.md.

Prerequisites (Hard Gate)

Do NOT start unless:

• playwright-cli is available (npx @playwright/cli@latest --version)
• Target URL is known

Default capture method: playwright-cli tracing (standard workflow below).

Optional --mitmproxy mode

When the --mitmproxy flag was passed to /cli-anything-web (or the capture needs untruncated bodies / enhanced metadata), follow this workflow with the step substitutions in references/mitmproxy-mode.md — Steps 1, 3, and 4 differ; everything else is identical. When playwright-cli itself is unavailable (e.g., no Node), fall back to chrome-devtools-mcp via launch-chrome-debug.sh — see HARNESS.md Tool Hierarchy.

Public API Shortcut

If the target site has a documented public REST/JSON API (e.g., Hacker News Firebase API, Dev.to API, Reddit API, Wikipedia API), browser capture is optional:

1. Probe the API endpoints directly with httpx or curl
2. Save responses as <app>/traffic-capture/raw-traffic.json
3. Skip to Phase 2 (methodology)

This applies when:

• API docs exist (OpenAPI/Swagger, developer docs page, /api/ prefix)
• The API is publicly accessible without browser-specific auth
• Endpoints return JSON (not HTML)

If unsure whether a public API exists, proceed with browser capture as normal.

Resume from Checkpoint

Before starting, check if a previous capture session exists:

python ${CLAUDE_PLUGIN_ROOT}/scripts/capture-checkpoint.py restore <app>

If a checkpoint exists, read the guidance field and resume from the last completed step instead of starting over. This prevents duplicate work when sessions are interrupted.

Step 1: Setup

# Create output directory
mkdir -p <app>/traffic-capture

# Clear any stale sessions
npx @playwright/cli@latest kill-all 2>/dev/null || true

npx @playwright/cli@latest -s=<app> open <url> --headed --persistent
# Note: heavy SPAs (Next.js, React) may show "TimeoutError: page._snapshotForAI" on open.
# This is non-fatal — verify with: npx @playwright/cli@latest list
#
# IMPORTANT — "Browser opened with pid..." in command output means the daemon
# RE-ATTACHED to the existing browser, NOT that a new session was created.
# Do NOT re-navigate or restart when you see this. The session is still open.

--mitmproxy mode: use the Step 1 substitution in references/mitmproxy-mode.md (start the proxy, then open through it).

Do NOT ask the user to log in yet — Step 2 will determine if auth is needed.

Step 2: Site Fingerprint (Single Command)

Run the all-in-one site fingerprint command instead of individual eval calls. This is faster, more reliable, and detects framework + protection + iframes + auth requirements in one shot.

Use the script file — multi-line JS with arrow functions and optional chaining fails in playwright-cli's single-line command parser. The script file approach has been tested and works reliably:

npx @playwright/cli@latest -s=<app> run-code "$(grep -v '^\s*//' ${CLAUDE_PLUGIN_ROOT}/scripts/site-fingerprint.js | tr '\n' ' ')"

IMPORTANT: The site-fingerprint.js script must be loaded via the command above. Do NOT copy-paste the JS inline — it will fail with SyntaxError. The grep -v strips comments and tr joins lines for single-line execution.

Interpret fingerprint results

The fingerprint returns four groups: framework, protection, auth, iframes. Map each true flag to the next action:

Claude-facing shortcuts:

• googleBatch: true → generate rpc/ subpackage (batchexecute protocol).
• cloudflareManagedChallenge: true → tier 4 (camoufox) is required; curl_cffi alone will fail.
• awsWaf: true → capture aws-waf-token cookie; use curl_cffi for GraphQL, cookie-only for SSR.
• akamai: true or datadome: true → 1–2 s delays between requests are mandatory.
• serviceWorker: true → note in assessment.md; generated CLI uses service_workers="block".
• iframeCount > 0 → re-run the fingerprint inside the iframe. Google Labs apps (Stitch / MusicFX / ImageFX) follow this pattern — parent has WIZ_global_data, iframe has the real app.

Note: snapshot and click <ref> auto-resolve iframes. Only drop down to run-code for iframe interaction when built-in commands fail.

Auth detection (BEFORE exploration)

Check the fingerprint auth fields:

If auth is needed:

1. Tell the user: "This site requires login. Please log in in the browser window."
2. Wait for user confirmation
3. Save auth state and tighten permissions (CLAUDE.md mandates chmod 600):

npx @playwright/cli@latest -s=<app> state-save <app>/traffic-capture/<app>-auth.json
chmod 600 <app>/traffic-capture/<app>-auth.json

If NO auth is needed: Skip directly to Step 2b.

2b. Classify Site Profile

Based on fingerprint results AND what you see in the UI, classify the site:

2c. Quick API Probe (Force SPA Navigation Trick)

Start a SHORT trace, click 3-4 internal links, stop. This reveals hidden API endpoints that SSR hides on initial page load.

npx @playwright/cli@latest -s=<app> tracing-start
npx @playwright/cli@latest -s=<app> click <internal-link-1>
npx @playwright/cli@latest -s=<app> click <internal-link-2>
npx @playwright/cli@latest -s=<app> click <internal-link-3>
npx @playwright/cli@latest -s=<app> tracing-stop

# Quick parse to see what endpoints appeared (saved alongside the full capture
# so it survives the session — don't output to /tmp).
python ${CLAUDE_PLUGIN_ROOT}/scripts/parse-trace.py .playwright-cli/traces/ --latest \
  --output <app>/traffic-capture/probe-traffic.json

This probe trace is separate from the full capture in Step 3 — Step 3 will start a fresh trace that overwrites the .network file in .playwright-cli/traces/. The parsed probe-traffic.json is kept in traffic-capture/ so it stays available for cross-referencing during Step 4.

Check the probe results — what API patterns did you find? See references/api-discovery.md for the priority chain and decision tree.

2d. Write Assessment Summary

Create <app>/traffic-capture/assessment.md to consolidate all findings:

# Site Assessment: <app>

- **URL**: <url>
- **Framework**: <detected framework or "none/custom">
- **Protocol**: <REST / GraphQL / batchexecute / HTML scraping / hybrid>
- **Protection**: <none / cloudflare / captcha / aws-waf / etc.>
- **Auth required**: <yes (type: Google SSO / cookie / JWT / API key) / no>
- **Iframes**: <yes (N frames, app in frame N at <url>) / no>
- **Site profile**: <Auth+CRUD / Auth+Generation / Auth+Read-only / No-auth+CRUD / No-auth+Read-only>
- **Capture strategy**: <API-first / SSR+API hybrid / batchexecute / HTML scraping / protected-manual>
- **Key observations**: <any quirks, localized UI, rate limits, special patterns>