Literature Management

Source: https://github.com/aipoch/medical-research-skills

When to Use

• You have a batch of locally downloaded papers (PDF/BibTeX/RIS/CSV/TXT) that must be organized into a consistent library structure.
• You need offline deduplication using DOI or normalized Title + Year to avoid repeated entries.
• You want to apply manual tags at import time and also derive tags from available metadata keywords.
• You need a local, appendable search index (index.jsonl) for later retrieval and filtering.
• You must operate in an environment where network access is not allowed and all processing must remain local.

Key Features

• Local-only import into a managed literature library directory.
• Deterministic deduplication with clear priority rules (DOI → Title+Year → file hash).
• Tagging support:
  • manual tags via CLI flags
  • automatic tags from metadata keywords (when present)
• Searchable index generation and maintenance via index.jsonl (one JSON object per line).
• Predictable file organization by year and journal, with safe handling of naming conflicts.
• Security/compliance-friendly behavior: no external APIs, no credentials, no network calls.

Dependencies

• Python 3.9+
• Python packages (install via requirements file):
  • pip install -r scripts/requirements.txt

Example Usage

# 1) Choose a source directory containing local literature files
SOURCE_DIR="/path/to/downloads"

# 2) Choose (or create) a target library directory managed by this skill
LIBRARY_DIR="/path/to/literature-library"

# 3) Import with optional manual tags (repeatable)
python scripts/import_library.py \
  --source-dir "$SOURCE_DIR" \
  --library-dir "$LIBRARY_DIR" \
  --tag "survey" \
  --tag "to-read"

After running, verify:

• Organized files under:
  • "$LIBRARY_DIR/files/<Year>/<Journal>/..."
• Index file:
  • "$LIBRARY_DIR/index.jsonl"
• The import/deduplication/error summary printed by the script.

Additional examples may be available in: references/examples.md.

Implementation Details

Inputs

• Source directory: contains one or more of .pdf, .bib, .ris, .csv, .txt
• Target library directory: the managed library root
• Manual tags (optional): provided via repeated --tag "<tag>"

Outputs

• Organized literature files written into the target library directory
• index.jsonl created/appended in the library root
• A summary of imported items, deduplicated items, and errors

Index Data Model (index.jsonl)

Each line is a single JSON record with (at minimum) the following fields:

• id, title, year, journal, authors, keywords, doi, tags
• source_type, source_path, file_path
• dedup_key, dedup_rule, imported_at

Deduplication Algorithm

Deduplication is applied in the following priority order:

1. DOI (primary)
   • Case-insensitive comparison after normalization.
2. Title + Year (secondary)
   • Title is normalized (e.g., whitespace/case normalization) and combined with year.
3. File hash (fallback)
   • Used only when DOI and Title+Year are unavailable.

The chosen rule is recorded in dedup_rule, and the computed key is stored in dedup_key.

Tagging Rules

• All --tag values are always applied to imported records.
• If metadata includes keywords, they are converted into tags and merged with manual tags.

File Organization Rules

• Target path pattern:
  • <library>/files/<Year>/<Journal>/
• Unknown values are mapped to:
  • UnknownYear, UnknownJournal
• Filenames are preserved when possible; if a conflict occurs, a suffix is appended to avoid overwriting.

Security / Compliance Constraints

• No network access is used or required.
• No external APIs or credentials are used.
• The tool only reads from the specified --source-dir.
• The tool only writes within the specified --library-dir (no writes outside the library root).

When Not to Use

• Do not use this skill when the required source data, identifiers, files, or credentials are missing.
• Do not use this skill when the user asks for fabricated results, unsupported claims, or out-of-scope conclusions.
• Do not use this skill when a simpler direct answer is more appropriate than the documented workflow.