Documentation Research Methodology

3.4 Handling Version-Specific Content

1. Identify the documentation version from version selectors, URL path segments (/docs/v2/), or version badges.
2. When the documentation version differs from the target version, mark affected claims as inferred and add a note: "documented for v{X}, target is v{Y}."
3. When no version is stated, assume latest version. Note this assumption.
4. Check changelogs between documentation version and target version for behavioral changes that would invalidate claims.

3.5 Deduplication

When the same behavioral claim appears on multiple pages:

• Keep one entry in the claims file.
• Cite the most authoritative source (API reference over tutorial).
• If both sources are equally authoritative and agree, cite either and add corroborated_by to escalate confidence.

4. Output Structure

4.1 Raw Files (workspace/public/docs/raw/)

One file per topic area. Multiple pages about the same topic merge into one raw file.

# {Topic Name}

## Source
- **URL:** {primary source URL}
- **Additional URLs:** {other pages merged into this file}
- **Fetched:** {ISO 8601 timestamp}
- **Doc version:** {version if identifiable, otherwise "assumed latest"}

## Content Summary

[Concise structured extraction of key information from this topic area.
NOT verbatim reproduction. Organized by subtopic.]

## Key Behavioral Claims

- {claim text}
  <!-- cite: source=official-docs, ref={URL}#{section}, confidence={level}, agent=doc-researcher -->

- {claim text}
  <!-- cite: source=official-docs, ref={URL}#{section}, confidence={level}, agent=doc-researcher -->

4.2 Claims File (workspace/public/docs/claims/claims-by-topic.md)

The primary output. All behavioral claims organized by topic with full provenance.

# Behavioral Claims from Public Documentation

## Metadata
- **Target:** {product name}
- **Agent:** doc-researcher
- **Date:** {ISO 8601}
- **Total claims:** {count}
- **By confidence:** confirmed: {n}, inferred: {n}, assumed: {n}
- **Sources consulted:** {n} documentation pages, {n} community resources

---

## {Topic Area 1}

### CLAIM-DOC-001: {Short Descriptive Title}
{One or two sentences stating the behavioral claim precisely.}
<!-- cite: source={source-type}, ref={URL}, confidence={level}, agent=doc-researcher -->

### CLAIM-DOC-002: {Short Descriptive Title}
{Claim text.}
<!-- cite: source={source-type}, ref={URL}, confidence={level}, agent=doc-researcher -->

---

## {Topic Area 2}

### CLAIM-DOC-003: {Short Descriptive Title}
{Claim text.}
<!-- cite: source={source-type}, ref={URL}, confidence={level}, agent=doc-researcher -->

Claim ID format: CLAIM-DOC-{NNN} where NNN is a zero-padded three-digit sequence. IDs are assigned in the order claims are written and are never reused.

4.3 Gaps File (workspace/public/docs/gaps.md)

# Documentation Gaps

## Metadata
- **Target:** {product name}
- **Agent:** doc-researcher
- **Date:** {ISO 8601}
- **Topics documented:** {n}
- **Topics expected but missing:** {n}

---

## Missing Documentation

### GAP-001: {Topic}
**Expected:** {what documentation should exist for this topic}
**Found:** {what was actually found, or "No documentation found"}
**Impact:** {what analysis mode or manual effort could fill this gap}

---

## Authenticated Documentation (Skipped)

| URL | Auth Type | Notes |
|-----|-----------|-------|

## Non-English Documentation (Skipped)

| URL | Language | Notes |
|-----|----------|-------|

## Unfetched Pages (Budget Exhausted)

| URL | Reason | Priority |
|-----|--------|----------|

## Documentation Contradictions

| Topic | Page A | Page B | Contradiction |
|-------|--------|--------|---------------|

5. Termination Criteria

Stop the research process when ALL of the following are true:

1. Official site exhausted. The official documentation site has been fully traversed -- all pages in the table of contents or sitemap have been fetched, or the page budget has been reached.
2. All Tier 1 patterns executed. Every search pattern in Tier 1 has been run.
3. Tier 2 attempted. At least 3 relevant Tier 2 patterns have been tried.
4. Community reviewed. At least the top 5 Stack Overflow results have been reviewed.
5. Diminishing returns. No new behavioral claims have been extracted from the last 3 pages fetched.

If the page budget (50 pages) is reached before these criteria are met, stop fetching but still write the claims and gaps files from what was gathered. Record the unmet criteria in gaps.md.

6. Gap Analysis

After all extraction is complete, assess documentation completeness by comparing what you found against what a complete documentation suite should include for the target product type.

Expected Topics by Product Type

CLI tool:

• Installation and setup
• All commands and subcommands
• All flags and arguments (including global flags)
• Configuration file format and location
• Environment variables
• Authentication and credentials
• Error messages and exit codes
• Input/output formats
• Shell completion

Web API / REST service:

• Authentication and authorization
• All endpoints (method, path, parameters, request/response body)
• Rate limiting
• Pagination
• Error response format and error codes
• Versioning strategy
• Webhooks (if applicable)
• SDKs and client libraries

Library / SDK:

• Installation
• Public API surface (classes, functions, types)
• Configuration and initialization
• Error handling patterns
• Lifecycle and resource management
• Thread safety / concurrency model
• Compatibility matrix (language versions, platforms)

Desktop / mobile application:

• Installation and system requirements
• Features and UI flows
• Keyboard shortcuts
• File formats
• Configuration and preferences
• Integration points (plugins, extensions, APIs)

For each expected topic that is missing or insufficiently documented, create a GAP entry in gaps.md. Note which other intelligence source (SDK analysis, runtime observation, source code analysis) could fill the gap.

7. Provenance Discipline

This section restates the rules from the provenance-methodology skill as they apply specifically to documentation research. Follow both this section and the full provenance-methodology skill.

Cite As You Go

Every time you write a behavioral claim, the very next thing you write is the citation. Do not batch citations. Do not defer them. Write the claim, write the citation, then move on.

Citation Format

<!-- cite: source={source-type}, ref={URL}, confidence={level}, agent=doc-researcher -->

Source Type Selection

• Use official-docs for: official documentation pages, README files, man pages, API references, changelogs, published standards, RFCs
• Use community-knowledge for: Stack Overflow answers, GitHub Discussions, third-party blog posts, tutorials by non-maintainers, conference talks

Confidence Assignment

• confirmed: Two or more independent sources agree on the same behavioral claim. Example: official docs state a timeout is 30 seconds AND a Stack Overflow answer with maintainer confirmation says the same.
• inferred: One authoritative source states the claim and nothing contradicts it. This is the most common confidence level for doc-researcher output.
• assumed: No direct source states this, but it follows from convention or reasoning. Rare for this agent. If you find yourself writing many assumed claims, you may be speculating rather than extracting.

Standards and RFCs

When official documentation references a standard or RFC:

• Note the standard name and relevant section numbers in the claim.
• Use source type official-docs (the documentation itself is the source; the standard is additional context).
• If you fetch the standard/RFC directly, cite it separately.