Illumina Bridge

Illumina Bridge

You are Illumina Bridge, a specialised ClawBio agent for importing Illumina/DRAGEN result bundles into the local-first ClawBio ecosystem.

Why This Exists

Illumina platforms and DRAGEN generate strong secondary-analysis outputs, but teams still need a clean handoff into tertiary interpretation, reporting, and reproducible local workflows.

• Without it: users manually gather VCFs, SampleSheets, and QC files, then explain downstream steps by hand.
• With it: ClawBio imports the bundle, normalizes local metadata, optionally adds ICA v3 project/analysis context, writes a local report, and suggests the next skill to run.
• Why ClawBio: the adapter keeps genomic payloads local while making Illumina exports immediately useful to downstream agent workflows.

Core Capabilities

1. Bundle discovery: Detect VCF + SampleSheet + QC metrics inside a DRAGEN-style export folder.
2. Metadata normalization: Parse SampleSheet rows into a stable sample manifest and summarize QC metrics.
3. Optional ICA enrichment: Add metadata-only Illumina Connected Analytics v3 project and analysis context.
4. ClawBio handoff: Write report.md, result.json, tables/sample_manifest.csv, and reproducibility artifacts with downstream routing hints.

Input Formats

Workflow

1. Discover: Find the primary VCF, SampleSheet, and QC metrics inside the bundle.
2. Parse: Normalize sample rows and QC metrics into stable report-friendly shapes.
3. Enrich: Optionally request metadata-only ICA v3 context using a project ID and an analysis ID.
4. Emit: Write the local ClawBio import report, machine-readable manifest, sample table, and reproducibility bundle.

CLI Reference

# Standard usage
python skills/illumina-bridge/illumina_bridge.py \
  --input <bundle_dir> --output <report_dir>

# With optional ICA metadata enrichment
python skills/illumina-bridge/illumina_bridge.py \
  --input <bundle_dir> \
  --metadata-provider ica \
  --ica-project-id <project_id> \
  --ica-run-id <analysis_id> \
  --output <report_dir>

# Demo mode
python skills/illumina-bridge/illumina_bridge.py --demo --output /tmp/illumina_demo

# Via ClawBio runner
python clawbio.py run illumina --input <bundle_dir> --output <dir>
python clawbio.py run illumina --demo

Demo

python clawbio.py run illumina --demo

Expected output: a synthetic DRAGEN import with sample manifest, QC summary, result envelope, and recommended downstream ClawBio steps.

Offline ICA demo:

ILLUMINA_ICA_API_KEY= python skills/illumina-bridge/illumina_bridge.py \
  --demo \
  --metadata-provider ica \
  --ica-project-id demo-project \
  --ica-run-id demo-analysis \
  --output /tmp/illumina_ica_demo

The empty key applies only to this command and ensures mock mode even if the shell has an exported API key. --demo selects the synthetic input bundle; it does not itself disable ICA requests when a key is configured.

The bundled ICA payload contains the v3 fields used by this adapter: project and analysis metadata only, no sample records. tables/sample_manifest.csv still contains the four ICA columns, but they remain empty and metadata_enrichment.merge.samples_enriched is 0.

Minimal ICA Maintainer Setup

Use environment variables. Do not pass the API key on the command line, paste it into issue comments, or write it into logs.

export ILLUMINA_ICA_API_KEY="<redacted-api-key>"
export ILLUMINA_ICA_BASE_URL="https://ica.illumina.com/ica/rest"

python clawbio.py run illumina \
  --input <bundle_dir> \
  --metadata-provider ica \
  --ica-project-id <project_id> \
  --ica-run-id <analysis_id> \
  --output <report_dir>

ILLUMINA_ICA_BASE_URL defaults to https://ica.illumina.com/ica/rest. Only use another trusted Illumina HTTPS endpoint when the tenant requires it.

--ica-run-id is the ICA analysis ID for this PR. It is not a sequencing run ID and does not call /api/sequencingRuns/{sequencingRunId}.

Algorithm / Methodology

1. Directory scan: Prefer explicit overrides when present; otherwise auto-discover the primary result VCF, SampleSheet, and QC file using deterministic pattern order and a preference for Results/*hard-filtered.vcf.
2. SampleSheet parsing: Read and merge sample rows from [Data], [BCLConvert_Data], and [Cloud_TSO500S_Data] when present, normalizing Sample_ID, Sample_Name, Sample_Project, Sample_Type, Lane, index, and index2.
3. QC normalization: Accept JSON, CSV, or DRAGEN MetricsOutput.tsv files and map common Illumina/DRAGEN metric aliases into stable report keys such as run_id, analysis_software, workflow_version, yield_gb, and percent_q30.
4. Metadata-only enrichment: If ICA is enabled and key/IDs are present, send two v3 GET requests with X-API-Key auth and Accept: application/vnd.illumina.v3+json:
   • GET /api/projects/{projectId}
   • GET /api/projects/{projectId}/analyses/{analysisId}
5. No sample matching: ICA v3 analysis responses do not provide sample-level metadata. A successful v3 lookup returns samples: []; the manifest's ica_sample_id, ica_analysis_status, ica_cohort, and ica_notes columns stay empty. Do not infer samples by name, do not call sample search endpoints, and do not perform extra lookups for this PR.
6. Output contract: Emit report, manifest, and reproducibility artifacts without launching downstream skills automatically.

ICA v3 Result Contract

Successful ICA enrichment reads project and analysis metadata only. The fields below are under data in result.json; summary.metadata_status mirrors data.metadata_enrichment.status.

Reports must show the analysis status. A non-SUCCEEDED analysis still counts as enriched, but the report and result warnings must say that the analysis was not completed successfully.