Evidence Extractor

Evidence Extractor

Overview

Evidence Extractor reads provided source metadata and available source content, then extracts structured evidence items aligned to the research objective.

This skill receives a source package or a compatible user-provided list, processes sources one by one, and returns structured evidence that a downstream synthesis step can integrate into a knowledge summary. It does not coordinate additional workers, search for additional literature, or require downstream consumers to reconstruct results from intermediate messages.

Core principle: the extractor reads and mines provided sources. It extracts evidence; it does not discover new sources or synthesize final conclusions.

Output principle: the final response must be self-contained. Always return the Markdown summary plus an embedded JSON evidence package, even when extraction is partial or no evidence can be extracted. File paths may be included as artifacts, but they are never the only handoff.

Core Capabilities

• Accept a deduplicated source package.
• Validate required source metadata before extraction.
• Extract claims, findings, statistics, methods, mechanisms, and limitations.
• Classify evidence by type: empirical, statistical, theoretical, mechanistic, or opinion.
• Assign confidence and strength ratings with traceable source citations.
• Categorize extracted items by domain theme.
• Identify extraction gaps and source quality limitations.
• Produce a structured evidence package for downstream synthesis/report writing.

When To Use This Skill

Use this skill when the workflow needs:

• Evidence extraction from a known list of academic sources.
• Structured claims and findings from papers, preprints, reports, or clinical studies.
• Source-grounded evidence items for a later report writer.
• Confidence, strength, and evidence-type labels for each extracted item.
• A quality check before synthesis or final writing.

Do not use this skill when the task is to:

• Search for new papers or reports.
• Deduplicate or rank source lists.
• Produce a literature search methodology.
• Write the final research report.
• Synthesize cross-source conclusions into a narrative deliverable.

Input Template

Minimum input is Evidence Extraction Assignment plus Source Package.

### Evidence Extraction Assignment
[research objectives and questions that drive extraction]

### Source Package
[path to literature_sources.json, or embedded source list]

### Domain Context
[BIOMEDICINE | CHEMISTRY | MATERIALS | FINANCE | COMPUTER_SCIENCE | GENERAL]

### Extraction Strategy Hint
[optional: full | statistical | mechanistic | theoretical]

### Source Content
[optional full text, abstracts, snippets, uploaded PDFs, or accessible source excerpts]

### Output Directory
[optional directory for evidence JSON artifacts]

Expected source package shape:

{
  "sources": [
    {
      "title": "Attention Is All You Need",
      "authors": ["Ashish Vaswani", "Noam Shazeer"],
      "year": 2017,
      "venue": "NeurIPS",
      "doi": "10.5555/3295222.3295349",
      "url": "https://arxiv.org/abs/1706.03762",
      "abstract": "Short abstract text...",
      "source_database": "arxiv"
    }
  ],
  "total_results": 1,
  "dedup_report": {
    "total_before_dedup": 1,
    "duplicates_removed": 0,
    "final_unique": 1
  }
}

Handoff Contract

This skill may run as one of several parallel extraction tasks. Its final task result is the integration boundary for downstream synthesis.

Every final response must include:

• ## Evidence Extraction Results Markdown summary.
• ### Evidence Package with an embedded JSON block.
• ### Verdict with one of SUFFICIENT, PARTIAL, or INSUFFICIENT.
• Source-level quality notes for missing content, malformed metadata, skipped sources, and low-confidence extraction.

Do not rely on intermediate messages, hidden reasoning, or artifact paths as the only output. If an output file is written, also embed the same evidence package JSON in the final response so the parent orchestrator can validate and integrate the task result directly.

For partial runs, return accumulated evidence rather than an empty or failed task result. One malformed or inaccessible source should not fail the whole sub-task.

Methodology/Workflow

The extractor follows a source-grounded workflow: input validation, source comprehension, systematic extraction, and quality check. It should stay focused on extracting evidence from provided sources. Do not search for new literature, deduplicate source lists, or write final synthesis.

Phase 1: Input Validation And Extraction Planning

Objective: confirm that the source package and research objective are specific enough for reliable extraction.

Step 1.1: Validate Required Inputs

Before extracting, verify:

• Research objective is specific enough to decide what evidence matters.
• Source package is present and contains at least one source.
• Each source has title, authors, year, venue, source_database, and either doi or url.
• Domain context is specified or can safely default to GENERAL.
• Source content is available as full text, abstract, uploaded document text, or source snippets.

If source content is unavailable and only metadata is present, extract only metadata-level evidence and mark confidence as LOW. Do not invent claims that are not present in the provided content.

Step 1.2: Normalize Source Package

Accept either a literature_sources.json path or an embedded source list. Normalize each source into this working shape:

{
  "title": "...",
  "authors": ["..."],
  "year": 2024,
  "venue": "...",
  "doi": "...",
  "url": "...",
  "abstract": "...",
  "source_database": "arxiv"
}

Track missing fields in Quality Notes. Missing DOI is acceptable if URL is present. Missing abstract is acceptable only when other source content is provided.

Treat placeholder metadata as a metadata gap. Examples include authors: ["Multiple authors"], empty author lists, missing venue, missing year, or DOI/URL placeholders. Do not fail the extraction only because of a metadata gap, but record it in Quality Notes.

Step 1.3: Choose Extraction Strategy

Choose one strategy based on the assignment and source type:

If the first strategy yields too few relevant items, try one alternate strategy before marking coverage as weak.

Phase 2: Source Comprehension

Objective: understand each source well enough to avoid shallow or fabricated extraction.

Step 2.1: Identify Source Metadata

For each source, identify:

Step 2.2: Read High-Density Regions

Prioritize high-density evidence regions when full text is available:

1. Abstract and Introduction
2. Methods or Methodology
3. Results or Findings
4. Discussion and Limitations
5. Conclusion

For abstract-only sources, extract only claims explicitly present in the abstract and mark Source Citation as Abstract. Abstract-only evidence should normally be LOW or MEDIUM confidence; reserve HIGH for cases where full text, tables, figures, or explicit source sections are available. For metadata-only sources, do not extract substantive claims.

Step 2.3: Build A Key Claims Inventory

Before final extraction, identify candidate claims in this rough form:

Claim: [specific contribution, finding, statistic, method, or limitation]
Evidence: [data, table/figure reference, method description, quote summary, or argument]
Location: [section/page/paragraph/abstract]
Initial strength: [Strong | Moderate | Weak]
Relevance: [why this matters for the assignment]

Discard candidates that are not relevant to the extraction objective.

Phase 3: Systematic Evidence Extraction

Objective: turn relevant candidate claims into structured evidence items.

Step 3.1: Extract Evidence Items

Extract source-grounded items that answer the research objective.

Each item must include:

• Claim or finding.
• Evidence backing the claim.
• Source citation or location.
• Evidence type.
• Confidence.
• Domain theme.
• Strength.
• Extraction justification.