Apify Lead Scoring Enrichment

Lead Scoring & Enrichment

Turn a CSV of company URLs into a scored, contact-enriched pitch list. The agent asks the user for scoring rules in plain English ("+10 if using Shopify", "-5 if company size <10"), picks an enrichment path (departments or copywriters), and orchestrates six Apify Actors through four helper scripts.

Prerequisites

• Apify account with an active APIFY_TOKEN (Console → Settings → Integrations)
• Node.js 20.6+ (needed for native --env-file support)
• A .env file at the skill root containing APIFY_TOKEN=apify_api_...
• One-time inside scripts/: npm install (installs csv-parse, csv-stringify)

Optional but recommended: the Apify CLI (npm i -g apify-cli) for ad-hoc Actor calls. The helper scripts hit the REST API directly and do not need the CLI.

Workflow

Copy this checklist and track progress:

Task Progress:
- [ ] Step 1: Collect CSV path and validate required column (company_url)
- [ ] Step 2: Collect scoring rules per source (tech / content / metadata)
- [ ] Step 3: Collect enrichment path (departments OR copywriters)
- [ ] Step 4: Run scoring Actors (writes scoring.json)
- [ ] Step 5: Apply scoring rules per lead → assign per-source scores + outreach_hook (writes scored.json)
- [ ] Step 5b: Compute theoretical min/max score, ask user for qualification threshold, filter leads → qualified_leads.csv
- [ ] Step 6: Run enrichment path against qualified_leads.csv (writes enrichment.json)
- [ ] Step 7: Merge scoring + enrichment onto the ORIGINAL CSV → leads.enriched.csv (qualified column marks who made the cut)

Step 1: CSV intake

Ask the user for the CSV path. Required column: company_url. Recognized optional columns pass through untouched: company_name, first_name, last_name, role, department. Reject the run if company_url is missing. Trim to a bare domain (strip trailing slash, www. optional) when feeding downstream Actors that expect a domain.

Step 2: Scoring rules (per source)

Ask one question per source so it's obvious to the user (and to you at Step 5) which data each rule tests. Ask only for the sources the user wants to fetch — each source has a matching --enable-* flag in Step 4.

2a. Tech-stack rules — applied against scoring.json[url].tech (BuiltWith output). Only ask if the user wants --enable-tech. Example rules to show:

• +10 if the company uses Shopify or WooCommerce (we sell a Shopify integration).
• +5 if the tech stack includes HubSpot, Marketo, or Segment (marketing-ops ICP).
• -3 if no analytics or CDP is detected (likely too early-stage).

2b. Website-content rules — applied against scoring.json[url].content (Website Content Crawler markdown/text). Only ask if the user wants --enable-content. Also ask for maxCrawlDepth here (default 0 = homepage only; higher = more $). Example rules:

• +8 if the homepage describes a SaaS or platform business.
• -5 if the homepage describes a services agency (not our ICP).
• +3 if the homepage mentions "developers" or "API" (technical buyer).

2c. Company-metadata rules — applied against scoring.json[url].metadata (Contact Info Scraper metadata). Only ask if the user wants --enable-metadata. Example rules:

• +3 if industry is e-commerce, retail, or B2C.
• -3 if company size is under 10 employees (too small to buy).
• +5 if the company has a LinkedIn presence (bigger operation).

Store each rule block verbatim, tagged with its source. If the user folds a metadata rule into the tech block (e.g. "+3 if size >50" under tech), re-file it to the correct block before Step 5 and tell them why.

Any source the user has no rules for should also be dropped from the Step 4 --enable-* flags — no point paying for a signal you won't score on. Full example rule sets: examples/scoring-rules.example.md.

Step 3: Enrichment path (pick one)

Ask: "Which enrichment path?

• (A) Department contacts — find named people (with title + email) in a specific team at each company. Uses Contact Info Scraper's "Business leads enrichment" add-on, falls back to Bulk Email Finder for any lead without a discovered email.
• (B) Copywriter hunt — for each domain, Google-search site:{domain} blog, extract author names from top posts, find their emails. Good for guest-post outreach."

For Path A, collect two more inputs:

1. Department(s) — one or more from this enum (comma-separated): c_suite, product, engineering_technical, design, education, finance, human_resources, information_technology, legal, marketing, medical_health, operations, sales, consulting. Example: marketing,sales.
2. Max leads per domain — integer. Recommend 3–5 for typical SDR work. ⚠️ This is a cost multiplier: 5 leads × 500 domains = up to 2500 billed leads. Apify only charges for leads successfully found. Warn the user before running if max_leads × domain_count > 500.

For Path B no additional input is needed.

Step 4: Run scoring Actors

node --env-file=.env scripts/run_scoring.js \
  --input leads.csv \
  --output scoring.json \
  --enable-tech --enable-content --enable-metadata \
  --content-crawl-depth 0

run_scoring.js batches all URLs into a single call per enabled Actor (not one call per lead), then reshapes the datasets into a per-URL sidecar so the agent can look up every signal by company_url. Actors that weren't --enable-*'d are skipped. Read the resulting scoring.json — its shape is { "https://acme.com": { "tech": {...}, "content": {...}, "metadata": {...} }, ... }.

Step 5: Apply scoring rules (per source, then sum)

For each lead in scoring.json, run one pass per source using only that source's rules from Step 2. This keeps the score auditable — if content_score = -5 on a lead the user expected to convert, you can inspect exactly which content rule fired without re-deriving the whole computation.

Produce five fields per lead:

• tech_score (number, or null if --enable-tech was off) — sum of Step 2a rule deltas against scoring.json[url].tech.
• content_score (number, or null if --enable-content was off) — sum of Step 2b rule deltas against scoring.json[url].content.
• metadata_score (number, or null if --enable-metadata was off) — sum of Step 2c rule deltas against scoring.json[url].metadata.
• score (number) — sum of the three above, treating null as 0.
• outreach_hook (string, one sentence) — the single most-personalizable signal across all sources: a specific CMS ("uses Shopify"), a named analytics tool, an industry match, a hiring signal in the copy — whatever a human sales rep would open the email with.

The null vs 0 distinction matters: a source that wasn't fetched must not be conflated with a source that was fetched and simply scored zero. Downstream CSV columns render null as blank, 0 as "0".

Store scored rows as an intermediate scored.json (agent writes it directly, keyed by canonical https://domain), then pass it to filter_qualified.js at Step 5b and to merge_output.js at Step 7.

Step 5b: Qualification threshold gate

Enrichment is the expensive part — running it on unqualified leads burns credits with no ROI. Gate it with a user-set threshold before you call any enrichment Actor.

1. Compute the theoretical score range from the Step 2 rules the user gave. For each source's rule set, sum every positive delta into max_source and every negative delta into min_source. Then min_total = min_tech + min_content + min_metadata and same for max_total. This is a hard bound: no lead can score outside it.

2. Also compute the observed range from scored.json — the actual minimum and maximum score values across all leads. Often the observed range is much narrower than the theoretical one.

3. Present both to the user, plus a rough tiering suggestion:

   *"Theoretical range: {min_total} to {max_total}. Observed range in your list: {observed_min} to {observed_max} across {n_leads} leads. Distribution: {count above 75th percentile} / {count above 50th percentile} / {count above 25th percentile} at those thresholds. What threshold do you want? Leads scoring at or above the threshold move to enrichment; everything below is flagged in the final CSV as qualified=false and skipped."*

   Recommend the 75th-percentile score as a starting point if the user is unsure — enrichment cost drops ~75% while keeping the top of the funnel. Warn if their chosen threshold would qualify 0 leads or qualify all of them (no filtering).

4. Mark qualified: true|false on every row in scored.json based on the chosen threshold (write it back), then run:

   node scripts/filter_qualified.js \
     --leads leads.csv \
     --scores scored.json \
     --output qualified_leads.csv

   filter_qualified.js is a pure-Node CSV filter — it reads scored.json, keeps only rows where qualified === true, and writes them to qualified_leads.csv preserving all original columns. The full lead list (including unqualified rows) still lives in the original leads.csv — Step 7's merge uses that as the join base.