Lamindb

Source: https://github.com/aipoch/medical-research-skills

LaminDB

When to Use

• Use this skill when you need this skill is applicable when using lamindb. lamindb is an open-source data framework for biology that makes data queryable, traceable, reproducible, and fair-compliant. it is suitable for managing biological datasets (scrna-seq, spatial transcriptomics, flow cytometry, etc.), tracking computational workflows, curating and validating data with biological ontologies, building data lakes, or ensuring data lineage and reproducibility in biological research. it covers data management, annotation, ontologies (genes, cell types, diseases, tissues), schema validation, integration with workflow managers (nextflow, snakemake) and mlops platforms (w&b, mlflow), and deployment strategies in a reproducible workflow.
• Use this skill when a data analytics task needs a packaged method instead of ad-hoc freeform output.
• Use this skill when the user expects a concrete deliverable, validation step, or file-based result.
• Use this skill when the documented workflow in this package is the most direct path to complete the request.
• Use this skill when you need the lamindb package behavior rather than a generic answer.

Key Features

• Scope-focused workflow aligned to: This skill is applicable when using LaminDB. LaminDB is an open-source data framework for biology that makes data queryable, traceable, reproducible, and FAIR-compliant. It is suitable for managing biological datasets (scRNA-seq, spatial transcriptomics, flow cytometry, etc.), tracking computational workflows, curating and validating data with biological ontologies, building data lakes, or ensuring data lineage and reproducibility in biological research. It covers data management, annotation, ontologies (genes, cell types, diseases, tissues), schema validation, integration with workflow managers (Nextflow, Snakemake) and MLOps platforms (W&B, MLflow), and deployment strategies.
• Documentation-first workflow with no packaged script requirement.
• Reference material available in references/ for task-specific guidance.
• Structured execution path designed to keep outputs consistent and reviewable.

Dependencies

• Python: 3.10+. Repository baseline for current packaged skills.
• Third-party packages: not explicitly version-pinned in this skill package. Add pinned versions if this skill needs stricter environment control.

Example Usage

Skill directory: 20260316/scientific-skills/Data Analytics/lamindb
No packaged executable script was detected.
Use the documented workflow in SKILL.md together with the references/assets in this folder.

Example run plan:

1. Read the skill instructions and collect the required inputs.
2. Follow the documented workflow exactly.
3. Use packaged references/assets from this folder when the task needs templates or rules.
4. Return a structured result tied to the requested deliverable.

Implementation Details

See ## Overview above for related details.

• Execution model: validate the request, choose the packaged workflow, and produce a bounded deliverable.
• Input controls: confirm the source files, scope limits, output format, and acceptance criteria before running any script.
• Primary implementation surface: instruction-only workflow in SKILL.md.
• Reference guidance: references/ contains supporting rules, prompts, or checklists.
• Parameters to clarify first: input path, output path, scope filters, thresholds, and any domain-specific constraints.
• Output discipline: keep results reproducible, identify assumptions explicitly, and avoid undocumented side effects.

Overview

LaminDB is an open-source data framework designed for biology that makes data queryable, traceable, reproducible, and FAIR-compliant (Findable, Accessible, Interoperable, Reusable). It provides a unified platform that combines lakehouse architecture, lineage tracking, feature stores, biological ontologies, LIMS (Laboratory Information Management System), and ELN (Electronic Lab Notebook) capabilities through a single Python API.

Core Value Propositions:

• Queryability: Search and filter datasets through metadata, features, and ontology terms.
• Traceability: Automatically track data lineage from raw data through analysis to results.
• Reproducibility: Version control data, code, and environment.
• FAIR Compliance: Standardized annotations using biological ontologies.

When to Use This Skill

Use this skill in the following scenarios:

• Managing biological datasets: scRNA-seq, bulk RNA-seq, spatial transcriptomics, flow cytometry, multimodal data, EHR data.
• Tracking computational workflows: Notebooks, scripts, pipeline executions (Nextflow, Snakemake, Redun).
• Curating and validating data: Schema validation, standardization, ontology-based annotation.
• Using biological ontologies: Genes, proteins, cell types, tissues, diseases, pathways (through Bionty).
• Building data lakes: Unified query interface across multiple datasets.
• Ensuring reproducibility: Automatic version management, lineage tracking, environment capture.
• Integrating ML pipelines: Connect Weights & Biases, MLflow, HuggingFace, scVI-tools.
• Deploying data infrastructure: Set up local or cloud-based data management systems.
• Collaborating on datasets: Share curated, annotated data with standardized metadata.

Core Capabilities

LaminDB provides six interconnected capability areas, each documented in detail in the references folder.

1. Core Concepts and Data Lineage

Core entities:

• Artifacts: Versioned datasets (DataFrame, AnnData, Parquet, Zarr, etc.).
• Records: Experimental entities (samples, perturbations, instruments).
• Runs & Transforms: Computational lineage tracking (what code produced what data).
• Features: Typed metadata fields for annotation and querying.

Key workflows:

• Create and version artifacts from files or Python objects.
• Track notebook/script execution using ln.track() and ln.finish().
• Annotate artifacts using typed features.
• Visualize data lineage graphs using artifact.view_lineage().
• Query provenance (find all outputs of specific code/inputs).

Reference: references/core-concepts.md - Read this document for detailed information on artifacts, records, runs, transforms, features, versioning, and lineage tracking.

2. Data Management and Querying

Query capabilities:

• Registry exploration and lookup with auto-complete.
• Retrieve single records using get(), one(), one_or_none().
• Filtering with comparison operators (__gt, __lte, contains, startswith).
• Feature-based queries (query by annotated metadata).
• Cross-registry traversal using double underscore syntax.
• Full-text search across registries.
• Advanced logical queries using Q objects (AND, OR, NOT).
• Stream large datasets without loading into memory.

Key workflows:

• Browse artifacts using filters and sorting.
• Query by features, creation date, creator, size, etc.
• Stream large files in chunks or via array slicing.
• Organize data using hierarchical keys.
• Group artifacts into collections.

Reference: references/data-management.md - Read this document for comprehensive query patterns, filtering examples, streaming strategies, and data organization best practices.

3. Annotation and Validation

Curation process:

1. Validation: Confirm datasets conform to expected schemas.
2. Standardization: Fix typos, map synonyms to canonical terms.
3. Annotation: Link datasets to metadata entities for queryability.

Schema types:

• Flexible schema: Validate only known columns, allow extra metadata.
• Minimal required schema: Specify basic columns, allow extra columns.
• Strict schema: Complete control over structure and values.

Supported data types:

• DataFrames (Parquet, CSV)
• AnnData (single-cell genomics)
• MuData (multimodal)
• SpatialData (spatial transcriptomics)
• TileDB-SOMA (scalable arrays)

Key workflows:

• Define features and schemas for data validation.
• Validate using DataFrameCurator or AnnDataCurator.
• Standardize values using .cat.standardize().
• Map to ontologies using .cat.add_ontology().
• Save curated artifacts with schema links.
• Query validated datasets by features.

Reference: references/annotation-validation.md - Read this document for detailed curation workflows, schema design patterns, handling validation errors, and best practices.

4. Biological Ontologies

Available ontologies (via Bionty):

• Genes (Ensembl), Proteins (UniProt)
• Cell Types (CL), Cell Lines (CLO)
• Tissues (Uberon), Diseases (Mondo, DOID)
• Phenotypes (HPO), Pathways (GO)
• Experimental Factors (EFO), Developmental Stages
• Organisms (NCBItaxon), Drugs (DrugBank)

Key workflows:

• Import public ontologies using bt.CellType.import_source().
• Search ontologies by keyword or exact match.
• Standardize terms using synonym mapping.
• Explore hierarchical relationships (parents, children, ancestors).
• Validate data against ontology terms.
• Annotate datasets using ontology records.
• Create custom terms and hierarchies.
• Handle multi-organism contexts (human, mouse, etc.).

Reference: references/ontologies.md - Read this document for comprehensive ontology operations, standardization strategies, hierarchy navigation, and annotation workflows.