Research Paper Downloader

Source: https://github.com/aipoch/medical-research-skills

When to Use

• The user provides a DOI (e.g., 10.1038/s41586-020-2649-2) and wants the corresponding open-access PDF.
• The user provides an arXiv ID (e.g., 1706.03762) and wants the preprint PDF.
• The user asks to search papers by keywords (e.g., "attention mechanism") and download available open-access PDFs.
• The user requests downloading an academic/research paper and expects the tool to try multiple open-access providers automatically.
• The user needs the final local save path for the downloaded PDF(s).

Key Features

• Multi-source open-access retrieval with priority-based fallback until a PDF is found.
• Supports three input modes:
  • DOI download
  • arXiv ID download
  • Keyword search + download (with max_results)
• Configurable output directory and network behavior (timeouts, retries, preferred source order).
• Clear failure modes:
  • invalid identifiers
  • no open-access location found
  • network timeouts / source unavailable
• Safe file writing: downloads are saved only under the configured output directory.
• Consistent naming convention: {first_author}_{year}_{title}.pdf (truncated to a maximum length).

Dependencies

• Python >= 3.8
• requests >= 2.0.0
• Internet access (HTTPS) to the following services:
  • api.semanticscholar.org
  • api.openalex.org
  • api.unpaywall.org
  • arxiv.org
  • pubmed.ncbi.nlm.nih.gov
  • api.crossref.org

Example Usage

CLI (scripts/download.py)

Download by DOI:

python scripts/download.py --doi "10.1038/s41586-020-2649-2" --output "~/Downloads/ResearchPapers"

Download by arXiv ID:

python scripts/download.py --arxiv "1706.03762" --output "~/Downloads/ResearchPapers"

Search by keywords and download up to 3 results:

python scripts/download.py --search "attention mechanism" --max-results 3 --output "~/Downloads/ResearchPapers"

Python (import and call)

from pathlib import Path

# These functions are expected to be provided by the skill implementation (e.g., scripts/download.py or a module it exposes).
from download import download_by_doi, download_by_arxiv, search_download

output_dir = Path.home() / "Downloads" / "ResearchPapers"
output_dir.mkdir(parents=True, exist_ok=True)

# 1) Download by DOI
pdf_path = download_by_doi(doi="10.1016/j.clinimag.2022.03.004", output_dir=str(output_dir))
print("Saved to:", pdf_path)

# 2) Download by arXiv ID
pdf_path = download_by_arxiv(arxiv_id="2310.12345", output_dir=str(output_dir))
print("Saved to:", pdf_path)

# 3) Keyword search and download
pdf_paths = search_download(query="transformer attention", max_results=5, output_dir=str(output_dir))
for p in pdf_paths:
    print("Saved to:", p)

Implementation Details

Source Priority and Fallback

The downloader attempts open-access sources in priority order and stops at the first successful PDF retrieval:

1. Semantic Scholar (Open Access API) — api.semanticscholar.org
2. OpenAlex (Open Academic Graph) — api.openalex.org
3. Unpaywall (OA location finder) — api.unpaywall.org
4. arXiv (preprint server) — arxiv.org
5. PubMed (biomedical database) — pubmed.ncbi.nlm.nih.gov
6. Crossref (DOI registry/metadata) — api.crossref.org

If a source is unavailable, returns no OA link, or times out, the implementation should automatically proceed to the next source.

Configuration (config.json)

Recommended configuration keys:

• output_dir: default download directory (e.g., ~/Downloads/ResearchPapers)
• timeout: request timeout in seconds (recommended range: 30-120)
• max_retries: maximum retry attempts for transient failures
• preferred_sources: ordered list of sources to try (overrides default priority)

File Output Rules

• Path restriction: all files must be written only inside output_dir.
• Default directory: ~/Downloads/ResearchPapers (if not overridden).
• Filename format: {first_author}_{year}_{title}.pdf
• Maximum filename length: 80 characters (truncate safely).
• The tool should return (or print) the full absolute path of each saved PDF.

Error Handling Expectations

• Invalid DOI/arXiv ID: return a clear error and suggest correcting the identifier format.
• No open-access version found: report that no OA PDF is available and suggest alternatives (library access or contacting the authors).
• Timeouts: retry according to max_retries, then fall back to the next source.
• Source unavailable: skip to the next source and report which sources failed if all fail.

Safety Constraints

• No paywall bypassing or restricted-content circumvention.
• No credential handling is required (sources are expected to work without API keys).
• The downloader must not execute external code; it only performs network requests and writes files to disk.