API Docs Quality Report

API Docs Audit Skill

Crawls every API reference endpoint page on a documentation site, scores each against 5 quality checks, detects site-wide patterns, and outputs an interactive HTML report matching the style of previously-generated audits in this session.

---

STEP 0 — COLLECT INPUT

You need exactly one input: the docs URL.

If the user hasn't provided it, ask:

Which API docs URL should I audit?

If the URL is provided, strip trailing slashes and confirm it before proceeding.

Special case — "do the same for X": If the user says "do the same" or "same audit for X", they want the identical workflow and HTML report format applied to the new URL. Proceed directly to Step 1.

---

STEP 1 — DISCOVER ALL ENDPOINT PAGES

1a. Try llms.txt first

Fetch <docs_url>/llms.txt.

If found:

• Extract every line matching the pattern - [Page Title](URL): description
• Filter to API reference pages only — look for paths containing any of: api-reference, api-reference/, /api/, /endpoints/, /rest/, /docs/, /reference/ or versioned paths (e.g., /v1/)
• Exclude: overview/index pages (no HTTP method implied), SDK docs, CLI docs, Terraform docs, guides, quickstart pages
• Store as ENDPOINT_PAGES[] — list of {title, url, description}

If not found or returns 404:

• Fetch the docs homepage
• Look for a nav sidebar or sitemap link referencing API/endpoints
• Extract all API reference page URLs from the nav structure
• Note in the report that llms.txt was unavailable

1b. Identify categories

Group discovered pages by their URL path prefix or nav section:

• e.g. /api-reference/agents/ → Agents category
• e.g. /api-reference/executions/ → Executions category
• Pages at the root level of api-reference → Uncategorized or infer from title

Store CATEGORIES[] — list of {name, page_count, pages[]}.

Report discovery summary:

Found N endpoint pages across K categories. Fetching now...

---

STEP 2 — CHECK OPENAPI SPEC AVAILABILITY

Before fetching individual pages, check if a standalone OpenAPI spec exists.

Try these discovery methods in order (stop at first success):

1. Inspect the Link HTTP header (RFC 8288) on the homepage or docs root for rel="openapi" or rel="describedby" — this is the standard discovery mechanism for API specifications
2. <docs_url>/openapi.json
3. <docs_url>/openapi.yaml
4. <docs_url>/api/openapi.json
5. Any URL referenced in llms.txt under "## OpenAPI Specs"

Result:

• If accessible: SPEC_AVAILABLE = true, store the URL
• If 404 on all attempts and no Link header found: SPEC_AVAILABLE = false

A missing spec is itself a top-priority finding — record it as Issue #1.

---

STEP 3 — FETCH AND PARSE ALL ENDPOINT PAGES

Fetch each URL in ENDPOINT_PAGES[]. For each page, extract:

3a. Page-level prose description

The text that appears above the OpenAPI block (or between the # title and the first ## OpenAPI heading). This is the human-written description.

3b. OpenAPI block

Look for a fenced code block with language hint yaml or json that contains an OpenAPI spec. Extract:

• HTTP_METHOD — e.g. GET, POST, PUT, DELETE, PATCH
• HTTP_PATH — e.g. /api/v1/agents/{agent_id}/execute
• REQUEST_BODY_SCHEMA — the requestBody schema section (if any)
• PARAMETERS — path + query params (if any)
• RESPONSES — the full responses object
• RESPONSE_200_SCHEMA — the schema under responses.200 or 201

If no OpenAPI block found, mark SPEC_INLINE = false.

---

STEP 4 — SCORE EACH ENDPOINT (5 CHECKS)

For every endpoint, evaluate all 5 checks. Store results as {check_name: "pass"|"warn"|"fail", current_state: string, fix_guidance: string}.

Read references/scoring-rules.md for the exact scoring criteria per check.

Check 1 — Endpoint Description Check 2 — OpenAPI Spec Check 3 — Body Param Descriptions Check 4 — Response Codes Check 5 — Response Schema

---

STEP 5 — DETECT SITE-WIDE PATTERNS

After scoring all endpoints, look for patterns that repeat across many pages.

A pattern qualifies if it affects ≥ 60% of endpoints:

For each detected pattern, write a single paragraph summarising:

• What the pattern is
• How many endpoints it affects
• Root cause (e.g. FastAPI auto-generation, missing middleware docs)
• Impact on developer experience

Store as SITE_WIDE_PATTERNS[].

---