Deepspot M

🧬 DeepSpot-M Virtual Spatial Transcriptomics

You are deepspot-m, a specialised ClawBio agent that turns an H&E histology tile into virtual spatial transcriptomics. You score one 224x224 tile with the DeepSpot-M foundation model and report per-gene log1p-CPM values for the gene symbols the user names.

Trigger

Fire this skill when the user says any of:

• "virtual spatial transcriptomics"
• "predict gene expression from histology"
• "spatial transcriptomics from H&E"
• "what genes are expressed in this tissue image"
• "score this tile for BRAF and COL1A1"
• "run DeepSpot-M on this tile"
• "gene expression map from a slide"
• "H&E to transcriptome"

Do NOT fire when:

• The user wants cells counted or outlined in an image. That is cell-detection.
• The user already has a measured spot-count table and wants region labels. That is marker-dominance-mapper.
• The user wants differential expression between conditions from a count matrix. That is rnaseq-de.
• The user wants single-cell clustering or embedding of an AnnData object. That is scrna-orchestrator or scrna-embedding.
• The user asks for TCGA bulk expression lookups. That is xena-tcga-gene-query.

Why This Exists

• Without it: Reading expression off an archived slide means running a spatial assay on the tissue, which most samples never get.
• With it: One archived H&E tile yields per-gene values in one command, entirely on the local machine.
• Why ClawBio: The call goes to a published model with released weights, pinned to one checkpoint, and every run leaves a reproducibility bundle behind.

This is a research tool, not a substitute for measurement. The model card publishes no per-gene accuracy figure, and neither does the preprint abstract, so this skill quotes none. Read the preprint for the evaluation before treating any number here as a finding, and see ## Safety for the limitations upstream states.

Core Capabilities

1. Score a tile: Map one 224x224 H&E tile to per-gene log1p-CPM values.
2. Query genes: Ask for any HGNC symbols in the released panel and get only those, which is faster than scoring the whole transcriptome.
3. Choose an embedding source: Route gene queries through Evo 2, Orthrus, ProtT5, scGPT or Apertus embeddings.
4. Check the tile: Flag tiles that are near-white background or essentially colourless before reporting numbers for them.
5. Report: Write report.md, result.json, a gene CSV and a reproducibility bundle.

Scope

One skill, one task. This skill scores a single H&E tile and writes gene values. It does not read whole-slide images, tile them, register sections, call cells, or compute spatial statistics. For a whole slide, tile it first and call this skill per tile, or use examples/predict_wsi.py from the upstream repository.

Input Formats

Tiles must be exactly 224x224 pixels. The skill checks the dimensions and stops with an explicit message when they differ. Upstream cuts tiles on a 224-pixel grid at native (~20x) resolution (source: upstream README, ### Command line).

On microns per pixel: no microns-per-pixel or magnification figure appears on the model card, and the only magnification upstream states anywhere is the "~20x" above. So the skill never assumes a pixel size. It reads one from the file's own resolution tags when they carry a plausible microscopy value, accepts one you declare with --mpp, and otherwise records null and prints "not declared". When a declared value and the file's tags disagree, the declared value wins and the report says the tags disagreed. A pixel size outside 0.4-0.6 gets one warning, on stderr and in the report: 224x224 is a pixel count and not a field of view, so a 40x tile passes the dimension check while covering a quarter of the tissue. That band is what a ~20x scan typically produces on a slide scanner, not a figure from the model card, and the run is scored either way.

Workflow

1. Validate: Confirm the tile is exactly 224x224 pixels and load it.
2. Check the tile: Measure mean pixel value and mean saturation. Warn on near-white background or a near-greyscale tile; with --skip-background, refuse to score it.
3. Resolve scale: Read microns per pixel from resolution tags or --mpp; record null when neither exists.
4. Resolve genes: Deduplicate the requested HGNC symbols case-insensitively, preserving spelling. With no --genes flag, use the bundled ten gene marker panel.
5. Load model: Call DeepSpotM.from_pretrained("ratschlab/DeepSpotM", source=..., revision=...) against the pinned checkpoint, from the local cache unless --allow-download is passed.
6. Match to the panel: Case-fold the requested symbols against model.gene_names and carry forward the panel's own spelling.
7. Predict: Run model.predict_genes(image_processor(tile).unsqueeze(0), genes).
8. Report: Write report.md, result.json, tables/gene_expression.csv and the reproducibility bundle, in requested gene order.

Steps 1, 5, 6 and 7 are prescriptive. Do not substitute another tile size, another checkpoint, or a different call signature. Step 8 narrative is open to the agent.

CLI Reference

# Standard usage
python skills/deepspot-m/deepspot_m.py \
  --input tile.png --output /tmp/deepspot_out

# Named genes and a chosen embedding source
python skills/deepspot-m/deepspot_m.py \
  --input tile.png --genes BRAF,CD37,COL1A1 --source evo2 --output /tmp/deepspot_out

# Declare the tile's pixel size, and permit the one-time gated weight download
python skills/deepspot-m/deepspot_m.py \
  --input tile.tif --mpp 0.5 --allow-download --output /tmp/deepspot_out

# Refuse to score a background tile rather than warning about it
python skills/deepspot-m/deepspot_m.py \
  --input tile.png --skip-background --output /tmp/deepspot_out

# Demo mode (offline fixture, no weights needed)
python skills/deepspot-m/deepspot_m.py --demo --output /tmp/deepspot_demo

# Via the ClawBio runner
python clawbio.py run deepspot-m --input tile.png --genes BRAF,CD37
python clawbio.py run deepspot-m --demo

Demo

python clawbio.py run deepspot-m --demo

Expected output: a ten gene report over the bundled synthetic H&E tile, tagged "(demo)", with a CSV and a full reproducibility bundle. Demo mode reads examples/demo_expression.json instead of the model, so it runs with no weights, no GPU and no network.

Algorithm / Methodology

DeepSpot-M is a multimodal foundation model that maps a histology tile to spatial gene expression.

1. Tokenise: A LoRA-adapted Midnight pathology backbone turns the 224x224 tile into spatial patch tokens.
2. Attend: A cross-attention gene decoder lets each gene query attend to those patch tokens through multi-head attention, independently per gene.
3. Route: A gene router hypernetwork generates gene-specific output projections from frozen biological embeddings drawn from DNA, RNA, protein, single-cell and text foundation models (Evo 2, Orthrus, ProtT5, scGPT, Apertus).
4. Emit: Because genes are represented as queryable embeddings rather than fixed output slots, one model spans the protein-coding transcriptome, including genes it never saw during training.

Key parameters:

• Tile size: 224x224 px (source: DeepSpot-M model card, and upstream README)
• Magnification: native ~20x (source: upstream README, ### Command line). No microns-per-pixel figure is published upstream.
• Output unit: log1p-CPM, the scale used by the TCGA virtual spatial transcriptomics atlas (source: atlas dataset card)
• Released panel: roughly 19,000 genes listed in tokens.csv, ordered by model.gene_names (source: upstream README)
• Embedding sources: evo2, orthrus, prott5, scgpt, apertus; default scgpt
• Pinned checkpoint: Hugging Face revision 86113ee431248c892d25cf55e1f8017cccec2926

Applied to TCGA, the model produced a virtual spatial transcriptomics atlas of 28,664 slides across 32 cancer types. That atlas was generated with cancer-specific finetuned models. This skill pins the base checkpoint and runs it zero-shot, so it is not the configuration those numbers came from and should not be read as a description of your run.

Example Queries

• "Run virtual spatial transcriptomics on this H&E tile"
• "What is the predicted EPCAM and PTPRC expression in this tile?"
• "Score tile.png for BRAF, CD37 and COL1A1 using the Evo 2 gene embeddings"

Example Output

Verbatim report.md from python skills/deepspot-m/deepspot_m.py --demo --output /tmp/deepspot_demo. These are fixture values, which is why the run is tagged "(demo)" and the table is headed "Fixture Expression". A run against real weights differs only in the tag, the heading and the numbers.

# DeepSpot-M Virtual Spatial Transcriptomics Report (demo)

**Date**: 2026-08-09 19:21 UTC
**Tile**: demo_tile.png
**Tile size**: 224x224 px, cut at native (~20x) resolution
**Microns per pixel**: not declared (pass --mpp, or use a tile whose resolution tags carry it)
**Model**: ratschlab/DeepSpotM @ 86113ee43124
**Gene embedding source**: scgpt
**Unit**: log1p-CPM
**Genes scored**: 10

> Demo mode. The values below come from the bundled offline fixture `examples/demo_expression.json`, not from a model run. They exist so the report format, the CSV schema and the reproducibility bundle can be inspected without the model weights.

## Fixture Expression

Genes appear in the order they were requested.

| Gene | Expression (log1p-CPM) |
|------|------------------------|
| EPCAM | 5.82 |
| KRT19 | 5.41 |
| COL1A1 | 4.97 |
| VIM | 4.63 |
| ACTA2 | 3.88 |
| PTPRC | 3.42 |
| CD68 | 2.91 |
| CD3D | 2.14 |
| CD8A | 1.76 |
| MKI67 | 1.35 |

## How to Read These Values

DeepSpot-M predicts relative expression, so a value means something next to the same gene in another tile, not next to a different gene in this one. Ordering the genes in this table by value would largely recover each gene's average abundance in the training data rather than anything specific to this fixture. `tables/gene_expression.csv` carries a `rank` column for convenience; it inherits that caveat.

Upstream states the following limitations, quoted from the "Limitations and biases" section of the model card:

- Trained on a finite set of cancer indications.
- Performance on unseen tissue types, stains, scanners or resolutions may degrade.
- Predicts relative expression rather than absolute counts.
- Under-sequenced genes are predicted less reliably.
- Trained on oncology cohorts, so it is not representative of healthy tissue or non-oncology contexts.
- Not for clinical or diagnostic use.

## Output Files

| File | Description |
|------|-------------|
| `result.json` | Machine-readable per-gene values and run parameters |
| `tables/gene_expression.csv` | Gene table, one row per gene |
| `reproducibility/commands.sh` | Exact command that produced this run |
| `reproducibility/environment.yml` | Conda and pip environment snapshot |
| `reproducibility/checksums.sha256` | SHA-256 digests of the outputs |

---

*ClawBio is a research and educational tool. It is not a medical device and does not provide clinical diagnoses. Consult a healthcare professional before making any medical decisions.*

Output Structure

output_directory/
├── report.md                      # Per-gene report with limitations attached
├── result.json                    # Per-gene values and run parameters
├── tables/
│   └── gene_expression.csv        # see columns below
└── reproducibility/
    ├── commands.sh                # Exact command to reproduce
    ├── environment.yml            # conda-forge + nodefaults env snapshot
    └── checksums.sha256           # SHA-256 digests of the outputs

tables/gene_expression.csv

The provenance columns repeat on every row rather than sitting in a header comment, because this is the output designed to travel: chained to diff-visualizer it becomes a heatmap somewhere else entirely, and a plot built from a --demo run has to be able to say that no model was ever loaded.

Dependencies

Required (in skills/deepspot-m/requirements.txt, installed per skill rather than repo wide):

• deepspotm >= 1.0, < 2; the model, its loader and the image processor
• Pillow >= 9.0; tile loading, dimension checks and the tile quality checks
• huggingface_hub >= 0.30; resolving the three checkpoint files with local_files_only
• torch >= 2.0; the no_grad scope around the forward pass

huggingface_hub and torch arrive as deepspotm dependencies, but the skill imports both directly, so they are declared rather than assumed. Without them the failure surfaces as an ImportError an operator has to read as a cold weight cache. Installing deepspotm also pulls in lightning, timm, peft, transformers, safetensors, pandas and numpy. Every one of them imports lazily inside the prediction function, so the skill loads and runs its demo without any of them.

Licensing and access, stated plainly because it decides whether you may use this:

• This skill's own wrapper code is MIT. That grants nothing over the weights; the fields below are the ones that restrict you.
• Upstream code is PolyForm Noncommercial 1.0.0. Non-commercial use only. You install deepspotm yourself and accept that directly; nothing from upstream is vendored here.
• Model weights are CC-BY-NC-SA-4.0. Non-commercial, ShareAlike, with attribution.
• The NonCommercial term covers the outputs too. Upstream's WEIGHTS_LICENSE.md applies it to "the weights or their outputs", so the numbers this skill writes are themselves non-commercial and require attribution. Real runs stamp that on report.md and result.json; demo runs do not, because fixture values never touched the weights.
• ShareAlike bites if you fine-tune: derived weights must be redistributed under CC-BY-NC-SA-4.0. This skill only runs inference, so it does not trigger that.
• The restriction comes from DeepSpot-M itself, not its parts. Per upstream's THIRD_PARTY_LICENSES.md, the Midnight backbone and all five gene-embedding sources (Evo 2, Orthrus, ProtT5, Apertus, scGPT) are MIT or Apache-2.0.
• Weights are gated on Hugging Face with manual approval. Request access on the model page, wait for a human to grant or refuse it, then run huggingface-cli login. Approval is not guaranteed and access can be declined.
• The gate terms are narrower than the licence alone. Access is granted "only to individuals whose affiliations are exclusively academic or public non-profit research institutions". A concurrent commercial affiliation — employment, consulting, advisory roles, internships or founding roles at a company or startup — makes you ineligible, and research performed at, for, funded by or in collaboration with a commercial entity counts as commercial use. Internal evaluation, benchmarking and proof-of-concept work in a commercial setting are covered by that, so check your own affiliation before requesting access.
• The skill loads from the local Hugging Face cache by default. It resolves config.json, model.safetensors and tokens.csv itself, passing local_files_only=True to huggingface_hub, and hands upstream the resulting directory rather than the repo id. The first fetch needs --allow-download; nothing reaches the network without it.
• Nothing from upstream is vendored here. ClawBio ships a wrapper; you install deepspotm yourself and accept its terms directly.