Bio Sra Data

Version Compatibility

Reference examples tested with: sra-tools 3.0+ (fasterq-dump, prefetch, vdb-validate, vdb-config), pysradb 2.2+, ENA portal API 2.0+

Before using code patterns, verify installed versions match. If versions differ:

• CLI: fasterq-dump --version, prefetch --version
• Python: pip show pysradb

If a flag is unrecognized or behavior changes, run <tool> --help and adapt.

SRA Data

"Download FASTQ from this SRA accession" -> Two paths exist in 2026: the SRA toolkit (NCBI's official, with prefetch + fasterq-dump) and the ENA mirror (EMBL-EBI's mirror with direct FASTQ download, often faster). For >1 TB workflows, a third path: AWS Open Data (STRIDES program) where same-region EC2 pulls SRA data with zero egress cost.

The single most impactful decision is where to pull from. SRA-direct is the default but ENA is faster more often than not, and AWS Open Data is the right answer for cloud-native analysis pipelines.

• CLI: prefetch SRR..., fasterq-dump SRR..., vdb-validate SRR... (sra-tools)
• CLI: curl https://ftp.sra.ebi.ac.uk/... (ENA mirror; direct FASTQ)
• CLI: aws s3 cp s3://sra-pub-run-odp/sra/SRR.../SRR... ./SRR....sra ... (STRIDES; object is unsuffixed; same-region free)
• Python: pysradb for metadata; subprocess for download

Required Setup

# sra-tools (toolkit)
conda install -c bioconda sra-tools           # 3.0+
fasterq-dump --version                        # confirm

# Configure cache location (default ~/ncbi/ -- often too small)
vdb-config --cfg                              # show current config
vdb-config --set /repository/user/main/public/root=/data/sra_cache

# Optional: pysradb for metadata
pip install pysradb

For STRIDES cloud:

# AWS CLI (no NCBI auth needed for public buckets)
aws s3 ls s3://sra-pub-run-odp/sra/SRR12345678/ --no-sign-request

Decision matrix: where to pull from

Default recommendation: ENA mirror for off-cloud, STRIDES (AWS/GCP) for in-cloud analysis. SRA-direct only when neither is available or when SRA format itself is needed (e.g. for re-extraction of technical reads).

SRA accession hierarchy

Conversion is via SRA metadata: pysradb metadata <ID> or efetch -db sra -id <UID> -rettype runinfo.

The actual download unit is SRR/ERR/DRR (runs). The BioProject (PRJNA...) is the convenient top-level handle for "pull all data for paper X".

fasterq-dump vs fastq-dump

fasterq-dump (sra-tools 2.10+) is the multi-threaded successor. Always prefer it, with two exceptions noted below.

The uncompressed-scratch trap: fasterq-dump writes uncompressed FASTQ first, then leaves it uncompressed. A 100 GB compressed FASTQ needs ~300 GB of scratch space + 300 GB of final output. Either compress post-hoc with pigz or use --mem to control RAM/disk tradeoff.

prefetch and the --max-size trap

prefetch downloads .sra files to the configured cache before extraction. Default --max-size 20G silently skips runs larger than 20 GB.

# Wrong: silently skips runs >20 GB
prefetch SRR12345678

# Right: set max-size explicitly to your largest expected size
prefetch SRR12345678 --max-size 100G -p

For unknown-size queues, set max-size to a generous upper bound (e.g. --max-size 200G) or query metadata first with pysradb metadata.

ENA mirror: direct FASTQ URLs

ENA stores FASTQ files directly (no SRA-format intermediate). Discover URLs via the ENA portal API:

curl 'https://www.ebi.ac.uk/ena/portal/api/filereport?accession=SRR12345678&result=read_run&fields=fastq_ftp,fastq_md5,read_count&format=tsv'

Returns TSV with semicolon-separated paired-end URLs and md5 checksums.

Direct download:

curl -O 'https://ftp.sra.ebi.ac.uk/vol1/fastq/SRR123/078/SRR12345678/SRR12345678_1.fastq.gz'

ENA's mirror is typically faster than SRA's because (a) it's hosted on Aspera-aware servers, (b) the FASTQ is pre-compressed (no SRA->FASTQ conversion needed), (c) EMBL-EBI's bandwidth is generous. For most downloads in 2026, ENA is the right default.

Single-cell / 10x quirks

10x Genomics records include "technical reads" (cell barcodes, UMIs) interleaved with biological reads. Default fasterq-dump (or fastq-dump) skips them. To get all reads:

# fasterq-dump with technical reads
fasterq-dump SRR12345678 --include-technical --split-files -p -O ./fastq/

# Some 10x records require fastq-dump -- check sra-stat first
sra-stat --xml SRR12345678 | grep -E '(spotCount|baseCount|tag)'

For 10x v3, expect 3 files per run: R1 (barcode+UMI), R2 (cDNA), I1 (index). For 10x v2: R1 (barcode), R2 (UMI+cDNA), I1.

MD5 / vdb-validate

Always verify downloads.

# vdb-validate for SRA-format files (toolkit path)
vdb-validate SRR12345678

# md5sum for ENA FASTQ files
md5sum -c <(echo "<expected_md5>  SRR12345678_1.fastq.gz")

ENA provides md5 in the portal API response. SRA-toolkit's vdb-validate is the equivalent for .sra files (different file format).

Cloud (STRIDES) access

NCBI's STRIDES initiative mirrored SRA data to AWS Open Data (us-east-1) and GCP (us-central1). Same-region pulls have zero egress cost.

# List SRA cloud-hosted files (no NCBI auth needed)
aws s3 ls s3://sra-pub-run-odp/sra/SRR12345678/ --no-sign-request

# Direct copy to EC2 in us-east-1. The STRIDES object is named without a `.sra`
# suffix (just SRR12345678); rename on copy to keep fasterq-dump happy.
aws s3 cp s3://sra-pub-run-odp/sra/SRR12345678/SRR12345678 ./SRR12345678.sra --no-sign-request

# Then fasterq-dump locally
fasterq-dump ./SRR12345678.sra -p -e 8

For cloud-native analysis pipelines (Nextflow on AWS Batch, Cromwell, etc.), STRIDES is the right path.

Code patterns

Single SRR via ENA mirror (preferred default)

Goal: Download paired-end FASTQ for one SRR; verify md5; minimal dependencies.

Approach: Query ENA portal API for FASTQ URLs and md5; download with curl; verify with md5sum.

Reference (ENA portal API 2.0+, curl):

#!/bin/bash
SRR="${1:-SRR12345678}"
OUT="${2:-./fastq}"
mkdir -p "${OUT}"

# Get FASTQ URLs + md5 from ENA portal API
META=$(curl -s "https://www.ebi.ac.uk/ena/portal/api/filereport?accession=${SRR}&result=read_run&fields=fastq_ftp,fastq_md5&format=tsv" | tail -1)
URLS=$(echo "${META}" | cut -f1 | tr ';' '\n')
MD5S=$(echo "${META}" | cut -f2 | tr ';' '\n')

i=0
while read url; do
    fname="${OUT}/$(basename ${url})"
    expected_md5=$(echo "${MD5S}" | sed -n "$((i+1))p")
    echo "Downloading ${fname}"
    curl -sL -o "${fname}" "https://${url}"
    actual_md5=$(md5sum "${fname}" | awk '{print $1}')
    if [ "${actual_md5}" != "${expected_md5}" ]; then
        echo "MD5 MISMATCH ${fname}: expected ${expected_md5}, got ${actual_md5}"
        exit 1
    fi
    echo "  md5 OK"
    i=$((i+1))
done <<< "${URLS}"

prefetch + fasterq-dump (SRA toolkit, classic)

#!/bin/bash
SRR="${1:-SRR12345678}"
OUT="${2:-./fastq}"
THREADS="${3:-8}"
mkdir -p "${OUT}"

# prefetch with explicit max-size (default 20G silently skips larger)
prefetch "${SRR}" --max-size 100G -p

# Validate SRA file
vdb-validate "${SRR}" || { echo "Validation FAILED"; exit 1; }

# Extract FASTQ (multi-threaded; uncompressed scratch ~3x final size)
fasterq-dump "${SRR}" -O "${OUT}" -e "${THREADS}" -p --split-files

# Compress post-hoc (fasterq-dump does NOT compress)
pigz -p "${THREADS}" "${OUT}/${SRR}"_*.fastq

# Cleanup SRA cache if you don't need it
# rm -rf ~/ncbi/sra/${SRR}.sra